Data Reliability Engineering: Secret Of Our Success | Akshay Karle & Carmen Mardiros | SREday 2023

Описание: SREday 2023 - September 14 & 15 2023 - London, UK
upcoming SREday CFPs: https://www.papercall.io/events?cfps-...
RSVP for the next edition: https://www.sreday.com
—

Data products break the software rules. Schemas shift, data acts up, systems revolt. Yet demand skyrockets. We built solutions. You'll master our playbook—data contracts, testing, monitoring—forcing order upon data chaos. Tame the wild west; lead the data age.

"While applying software engineering principles to build reliable software products is well established, data products pose unique challenges. Schemas and formats constantly evolve. Unexpected data issues arise. Multiple complex systems, including legacy platforms, must be integrated.
This talk shares hard-won lessons from building an enterprise data product that serves a major publishing company with massive data needs. You’ll learn how to tailor software practices specifically for data, employing:
Data contracts: Align teams, systems and uses.
Validation: Ensure correctness of your schemas and data, test automation in CI and at runtime.
Monitoring: Data freshness, completeness, and anomaly detection to address data product challenges.

Discover how to craft highly resilient, scalable data solutions. Tame fluctuating data and unreliable systems. Tackle the data wild west. You’ll leave armed with a blueprint for leadership in the data age.

Carmen:
Data professional with broad experience in the Data space spanning technical, analytical and strategy having held roles from pure BI and Analytics through to Senior Data Engineering and Head of Data. I have deep expertise (and most fun) in data engineering and data modelling.

Akshay:
Software Engineer. Quick Learner. Team Leader. Specializing in data, cloud infrastructure and security. 11+ years in Agile and distributed teams, implementing and coaching teams on Continuous Delivery, DevOps and Data Engineering while developing large scale enterprise webapps.