The 12 Factor App for Data • James Bowkett • GOTO 2023

Описание: This presentation was recorded at GOTO Copenhagen 2023. #GOTOcon #GOTOcph
https://gotocph.com

James Bowkett - Technical Delivery Director at OpenCredo ‪@OpencredoItd‬

RESOURCES
  / techwob  
  / jamesbowkett  

ABSTRACT
Data is everywhere, and so too are data-centric applications. As the world becomes increasingly data-centric, and the volumes of that data increase over time, data engineering will become more and more important. If we're going to be dealing with petabytes of data it will be better to get the fundamentals in place before you start, rather than trying to retrofit best practices onto mountains of data. This only makes a difficult job harder.

The 12-factor app helped to define how we think about and design cloud native applications. In this presentation, I will discuss 12 principles of designing data-centric applications that have helped me over the years across 4 categories : Architecture & Design, Quality & Validation (Observability), Audit & Explainability, Consumption.

This has ultimately led to our teams delivering data platforms that are both testable and well-tested. The 12 factors also enable them to be upgraded in a safe and controlled manner and will help them get deployed quickly, safely and repeatedly.

This talk will be filled with examples and counter examples from the course of my career and the projects that my teams have seen over the years. It will incorporate software engineering best practices and how these apply to data-centric engineering. We hope that you can benefit from some of our experience to create higher quality data-centric applications that scale better and get into production quicker. [...]

TIMECODES
00:00 Intro
01:51 What & why?
06:17 Architecture & design
06:27 No. 1: Data structures as code
10:07 No. 2: Append-only data structures
13:21 No. 3: Optimize for access & retrieval
15:51 No. 4: Separate data from logic
22:32 No. 5: Strongly type your data fields
25:03 Quality & validation
25:14 No. 6: Architect for regression testability
29:19 No. 7: Track changes in your test data
31:59 Audit & explainability
32:08 No. 8: Mind your metadata: Data cataloguing
34:37 No. 9: Mind your metadata: Code traceability
37:02 Consumption
37:28 No. 10: Defined APIs for accessing data
40:08 No. 11: Defined SLAs (& SLOs) for data
41:17 No. 12: Treat data as a product
42:28 Summary
45:32 Outro

Download slides and read the full abstract here:
https://gotocph.com/2023/sessions/2968

RECOMMENDED BOOKS
Zhamak Dehghani • Data Mesh • https://amzn.to/3tTCwAC
Sandeep Uttamchandani • The Self-Service Data Roadmap • https://amzn.to/3wAw5W2
Piethein Strengholt • Data Management at Scale • https://amzn.to/3tya08H

  / gotocon  
  / goto-  
  / goto_con  
  / gotoconferences  
#12FactorApp #Data #DataCentricApp #CloudNative #SoftwareArchitecture #DataPlatforms #DataStructuresAsCode #ContinuousDelivery #SLA #SLO #DataAsAProduct #JamesBowkett

CHANNEL MEMBERSHIP BONUS
Join this channel to get early access to videos & other perks:
   / @goto-  

Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket at https://gotopia.tech
Sign up for updates and specials at https://gotopia.tech/newsletter

SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
https://www.youtube.com/user/GotoConf...