Tech Talk Series Part Four: Continuous Integration and Continuous Delivery with Delta Lake

Описание: Join us for the final in a four part series with Salesforce Engineering.

Link to complete series recording: https://delta.io/news/salesforce-engi...

Abstract:
Abstract: As we build our Engagement Delta Lake on Databricks Workspace, one of the challenges is how to automate the integration testing of our Spark jobs in the CI/CD pipeline. We came up with two designs to tackle the challenge : Namespace Deployment and Scenario Based Testing. In this talk, we will discuss the rationale and implementations of the two designs.

Part 1: Engagement Activity Delta Lake -    • Tech Talk Series Part One: Engagement Acti...  
Part 2: Boost Delta Lake Performance with Data Skipping and Z-Order -    • Tech Talk Series Part Two: Boost Delta Lak...  
Part 3: Global Synchronization and Ordering in Delta Lake -    • Tech Talk Series Part Three: Global Synchr...  


-----------------
Speakers
-----------------

Zhidong Ke, Software Engineer PMTS, Salesforce
Zhidong is passionate in designing distributed systems, real-time/batch data processing and building applications.

Yifeng Liu, Software Engineer LMTS, Salesforce
Yifeng is a software engineer who has extensive experience in big data processing and distributed system, and interested in high volume, high complexity, low latency data pipeline and framework building.

Aaron Zhang, Software Engineering PMTS, Salesforce
Aaron is an experienced software engineering leader with interests and areas of focus in engineering secure, fault-tolerant, high volume systems built on micro services.

Heng Zhang, Software Engineering PMTS, Salesforce
Heng is a software engineer who is interested and specialized in micro services, distributed systems and big data. Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-nam...