Apache Spark Internals: Task Scheduling - Execution of a Physical Plan

Описание: In this video we deep dive into the execution of physical plans in Apache Spark. In the previous videos we have explored what a logical plan is and how it is transformed into a physical plan. Now we look at: How is the physical plan executed on our workers ...

00:00 Intro
00:28 Recap: Physical Plan, Logical Plan & Pipelining
02:16 What is a task?
02:54 How are tasks derived from the physical plan?
04:49 How are tasks scheduled on the cluster?
06:31 How are tasks executed on the workers?
08:16 What is a shuffle then?
08:56 Outro