End to End Azure Data Engineering Project by Sumit Sir

Описание: Link to Ultimate Data Engineering Program by Sumit Sir - https://trendytech.in/ultimate-big-da...

End to End Azure Data Engineering Project by Sumit Sir
This includes both part1 and part2 (one shot)

Github Link to the Datasets and reference code - https://github.com/bigdatabysumitm/Tr...

Azure Project Timestamps

00:00:00 - Introduction
00:03:23 - Problem Statement and Domain Overview
00:29:48 - Role of a Data Engineer in the Project
00:32:02 - Overview of Datasets
00:56:28 - Solution Architecture Breakdown
01:16:00 - Technologies Used in the Project
01:18:04 - step 1 - Azure Storage Account
01:26:27 - Pipeline Implementation Details
02:35:11 - Summary of the Full Load Process
02:37:03 - Pipeline Design for Incremental Load
02:50:31 - Challenges with the Current Pipeline
02:58:34 - Introduction to Part 2
3:07:30 - Quick start of Part 2
3:08:55 - Recap
3:15:21 - Overview of the Bronze Layer
3:17:39 - Explanation of ICD and CPT Codes
3:20:20 - Transition from Bronze to Silver Layer
3:22:14 - Transition from Silver to Gold Layer
3:23:42 - Discussion on Enhancements
3:30:50 - Setup Overview
3:38:15 - Key Vault Setup and Usage
3:51:10 - Linked Services
3:57:52 - Datasets Overview
3:59:05 - Active and Inactive Flags
4:06:00 - Adding Other Datasets to Bronze Layer
4:29:54 - Bronze to Silver Layer
4:45:08 - Introduction to SCD Type 2
4:46:56 - Explanation of CDM (Common Data Model)
4:52:28 - Quality Checks
4:55:08 - Implementation of SCD Type 2
5:13:33 - Silver to Gold Layer
5:18:00 - Gold Layer Queries
5:21:20 - ADF (Azure Data Factory) Pipeline
5:36:04 - Syncing Code to GitHub

Technology - Azure Data Engineering Stack
Domain - Healthcare Revenue Cycle Management (RCM)

RCM is the process that hospitals use to manage the financial aspects, from the time the patient schedules an appointment till the time the provider gets paid..

In This Project will be heavily using Azure Databricks & Azure Data Factory.

We will be creating a metadata driven pipeline.

Links to understand the KPI's & understand more on Accounts Receivable

https://mdmanagementgroup.com/healthc...

https://gentem.com/blog/revenue-cycle...

difference between ICD codes CPT codes

https://www.simplepractice.com/blog/i...

DO not forget to like the video & share with your friends!