Open Source Spotlight - Kestra - Will Russell

Описание: Open-Source Spotlight is our series where we're discovering open-source tools. This time, we talked about Kestra, a unified orchestrator that processes inputs and generates outputs, supporting both YAML code and a user-friendly UI. It bridges technical and non-technical workflows, integrates with tools like Docker and Git, and features in-editor documentation, autocomplete, and secret management for streamlined and collaborative workflow creation.

00:00 Introduction to Kestra and its features with Python
10:32 Using inputs and outputs in workflows and passing data between tasks
12:37 Managing dependencies and containers in Kestra
14:15 Working with output files and conditional logic
19:02 Handling missing outputs and error management
21:53 Debugging and using the Replay feature in Kestra
23:05 Error handling with notifications
24:48 Automating workflows with triggers
26:54 Backfilling data and using previous dates in Python
28:32 Visualizing workflow logic and task dependencies
28:47 Task sequencing, parallel execution, and dependency management
31:06 Contributing to the Kestra open source project
32:20 Joining the Kestra community and seeking help
33:15 Advice for new open source contributors for new open source contributors

🔗 USEFUL LINKS
Kestra documentation: https://go.kestra.io/docs
GitHub Repository: https://go.kestra.io/github
Slack Community: https://go.kestra.io/slack

🔗 CONNECT WITH Will Russell
X - https://x.com/wrussell1999
Linkedin -   / wrussell1999  
GitHub - https://github.com/wrussell1999
Website - https://wrussell.co.uk/videos

🔗 CONNECT WITH DataTalksClub
Join the community - https://datatalks-club.slack.com/join...
Subscribe to our Google calendar to have all our events in your calendar- https://calendar.google.com/calendar/...
Check other upcoming events - https://lu.ma/dtc-events
LinkedIn -   / datatalks-club  
X - https://x.com/DataTalksClub
Website - https://datatalks.club/

🔗 CONNECT WITH ALEXEY
X: https://x.com/Al_Grigor
Linkedin -   / agrigorev  

📚Check our free online courses
ML Engineering course - http://mlzoomcamp.com
Data Engineering course - https://github.com/DataTalksClub/data... uj7jh1c6xg
MLOps course - https://github.com/DataTalksClub/mlop...
Analytics in Stock Markets - https://github.com/DataTalksClub/stoc...
LLM course - https://github.com/DataTalksClub/llm-...
Read about all our courses in one place - https://datatalks.club/blog/guide-to-...

👋🏼 GET IN TOUCH
If you want to support our community, use this link - https://github.com/sponsors/alexeygri...

If you’re a company, support us at [email protected]