Pivot function in pyspark

Описание: Pivot function in pyspark #dataengineering #pyspark #pysparktutorial

from pyspark.sql.functions import *
from pyspark.sql.types import *
from pyspark.sql.window import Window

Data=[(101,"HR","Male",1000),
(102,"HR","Male",2000),
(103,"HR","FeMale",3000),
(104,"IT","Male",4000),
(105,"IT","FeMale",5000)]
Schema=["ID","Dept","Gender","Sal"]

df=spark.createDataFrame(Data,Schema)
df.show()

df1=df.groupBy("Dept").pivot("Gender").sum("Sal")
df1.show()