Databricks Full Course for Beginners (2 Hours) - Declarative Pipelines & Lakeflow Designer

Описание: Dive into a new way of building data pipelines in Databricks!

Instead of writing step-by-step Spark code, you define what you want with SQL and Databricks handles execution, dependencies, and orchestration for you.

In this full 2-hour course, you’ll learn how to build end-to-end data pipelines using Spark Declarative Pipelines and Lakeflow Designer.

👉 You’ll build a complete pipeline:
✅ Bronze → Silver → Gold (Medallion architecture)
✅ From raw data to analytics-ready tables
✅ Using SQL instead of complex Spark logic
✅ With automatic dependency handling and orchestration

And on top of that:
✅ Build pipelines visually with Lakeflow Designer
✅ Use Genie (Databricks Assistant) to speed up development
✅ Understand how modern Databricks workflows actually work

⏱️ Timestamps
00:03 – Course Overview & What You Will Build
02:56 – Getting Started in Databricks & Dataset Exploration
16:15 – Delta Lake Essentials (ACID, Time Travel, Versioning)
26:06 – Unity Catalog Basics
33:28 – Introduction to Declarative Pipelines & Medallion Architecture
39:02 – Building the Bronze Layer
50:17 – Building the Silver Layer
1:03:33 – Building the Gold Layer
1:11:06 – Scheduling & Running the Pipeline in Lakeflow Designer
1:17:39 – Creating a Pipeline Inside Lakeflow Designer (Part 1)
1:23:15 – Creating a Pipeline Inside Lakeflow Designer (Part 2)
1:33:19 – Streaming Modes in Declarative Pipelines
1:35:57 – Connecting Databricks to an AWS Message Queue
1:41:53 – Building a Simple Declarative Streaming Pipeline
1:56:38 – Summary & Next Steps

Resources:
► GitHub Repo for this course: https://github.com/andkret/Databricks...
► Databricks Free Edition: https://bit.ly/4eZHHIg
► All free labs & resources: https://learndataengineering.com/p/fr...

Want more?
I also did a livestream where I walk through the lab and explain how everything fits together.
► Watch it here: https://youtube.com/live/7lwKcLICy5g?...


#dataengineering #dataengineeringcourse #databricks #databrickscourse