Session 4 - Different ways of creating the data frame in PySpark

Описание: Different ways to create the data frame -

1. When we have python list
df = spark.createDataFrame(data,schema)

2. When file is stored at storage location
df = spark.read\
.format("file_format")\
.schema(dataframe_schema)\
.option("header",True)\
.load("file_path")

3. Creating the dataframe from table
df3 = spark.read.table("employee")

4. Creating the dataframe from table
df4 = spark.table("employee")

5. Creating the dataframe from table
df5 = spark.sql("SELECT * FROM employee")

6. Using range function
df6 = spark.range(6)
df7 = spark.range(12,35)
df8 = spark.range(2,56,4)

#pyspark #apachespark #schéma #learning #spark