08 Working with Strings, Dates and Null | Regex Replace | Convert string to date | Transform NULL

Описание: Video explains - How to use Case When in Spark ? How to manipulate String data in Spark DataFrames? How to cast dates in Spark ? How to extract date portions in Spark ? How to work with NULL data in Spark ?

Chapters
00:00 - Introduction
01:08 - How to use Case When in Spark?
04:30 - String Regex Replace
06:00 - How to convert string to date in Spark?
08:10 - How to add current date or timestamp in Spark ?
10:07 - How to drop NULL records in Spark ?
10:50 - How to transform NULL Columns in Spark ?
12:18 - Fix DataFrame
14:00 - Bonus Tip

Local PySpark Jupyter Lab setup -    • 03 Data Lakehouse | Data Warehousing with ...  
Python Basics - https://www.learnpython.org/
GitHub URL for code - https://github.com/subhamkharwal/pysp...
Documentation Spark Functions - https://spark.apache.org/docs/latest/...
Documentation Date/Timestamp Patterns - https://spark.apache.org/docs/latest/...

The series provides a step-by-step guide to learning PySpark, a popular open-source distributed computing framework that is used for big data processing.

New video in every 3 days ❤️

#spark #pyspark #python #dataengineering