tiger analytics interview questions

Описание: In this video, I have talked about data engineering interview question asked in tiger analytics. All of the above question was asked in actual interview of mine. For more queries reach out to me on my below social media handle.

Directly connect with me on:- https://topmate.io/manish_kumar25

Dataset:-
data = [(1,'Arul','Chennai','2023-01-01' ),
(1,'Arul','Bangalore','2023-02-01' ),
(2,'Sam','Chennai','2023-01-01' ),
(3,'manish','patna','2023-01-01' ),
(3,'manish','patna','2023-03-15' ),
(3,'manish','patna','2023-02-27' )]

schema = StructType([ \
StructField("id",IntegerType(),True), \
StructField("name",StringType(),True), \
StructField("location",StringType(),True), \
StructField("date",StringType(),True), \
])

df = spark.createDataFrame(data=data,schema=schema)
df.show()

df.select('id','name','location',to_date(col('date'),'yyyy-MM-dd').alias('date')).createOrReplaceTempView('table')

spark.sql("""
select * from table
""").show()

Second Data:-
data= [(1, 'Arul', 'SQL' ),
(1 ,'Arul' , 'Spark' ),
(2, 'Bhumica' , 'SQL' ),
(2 ,'Bhumica' , 'Spark' )]

schema= ['id','name','course']

df= spark.createDataFrame(data=data,schema=schema)

df.createOrReplaceTempView('table')

spark.sql("""
select * from table
""").show()

Third Data:-
data=[(10 ,'Anil',50000, 18),
(11 ,'Vikas',75000, 16),
(12 ,'Nisha',40000, 18),
(13 ,'Nidhi',60000, 17),
(14 ,'Priya',80000, 18),
(15 ,'Mohit',45000, 18),
(16 ,'Rajesh',90000, 10),
(17 ,'Raman',55000, 16),
(18 ,'Sam',65000, 17)]

schema=['id','name','sal','mngr_id']

manager_df= spark.createDataFrame(data=data,schema=schema)
manager_df.createOrReplaceTempView("manager_tbl")
spark.sql("""
select * from manager_tbl
""").show()


Spark words count:-
sc = spark.sparkContext
file= sc.textFile("/FileStore/tables/word_count.txt")
flat_file = file.flatMap(lambda word : word.split(' '))
flat_file_tuple = flat_file.map(lambda word: (word,1))
final_word_count = flat_file_tuple.reduceByKey(lambda x,y : x+y)
count = final_word_count.collect()
count

My Second Channel --    / @competitivegyan1  

Interview series Playlist:-    • Interview Questions and answers  

Follow me on LinkedIn:-   / manish-kumar-373b86176  
Follow Me On Instagram:-   / competitive_gyan1  
Follow me on Facebook:-   / manish12340  

#dataengineer #interview #interviewquestions
My Gear:-
Rode Mic:-- https://amzn.to/3RekC7a
Boya M1 Mic-- https://amzn.to/3uW0nnn
Wireless Mic:-- https://amzn.to/3TqLRhE
Tripod1 -- https://amzn.to/4avjyF4
Tripod2:-- https://amzn.to/46Y3QPu
camera1:-- https://amzn.to/3GIQlsE
camera2:-- https://amzn.to/46X190P
Pentab (Medium size):-- https://amzn.to/3RgMszQ (Recommended)
Pentab (Small size):-- https://amzn.to/3RpmIS0
Mobile:-- https://amzn.to/47Y8oa4 ( Aapko ye bilkul nahi lena hai)
Laptop -- https://amzn.to/3Ns5Okj
Mouse+keyboard combo -- https://amzn.to/3Ro6GYl
21 inch Monitor-- https://amzn.to/3TvCE7E
27 inch Monitor-- https://amzn.to/47QzXlA
iPad Pencil:-- https://amzn.to/4aiJxiG
iPad 9th Generation:-- https://amzn.to/470I11X
Boom Arm/Swing Arm:-- https://amzn.to/48eH2we

My PC Components:-
intel i7 Processor:-- https://amzn.to/47Svdfe
G.Skill RAM:-- https://amzn.to/47VFffI
Samsung SSD:-- https://amzn.to/3uVSE8W
WD blue HDD:-- https://amzn.to/47Y91QY
RTX 3060Ti Graphic card:- https://amzn.to/3tdLDjn
Gigabyte Motherboard:-- https://amzn.to/3RFUTGl
O11 Dynamic Cabinet:-- https://amzn.to/4avkgSK
Liquid cooler:-- https://amzn.to/472S8mS
Antec Prizm FAN:-- https://amzn.to/48ey4Pj