Hunter Owens | Building Your First Data Pipelines
Автор: PyData
Загружено: 2016-10-24
Просмотров: 13461
Описание:
PyData DC 2016
You need a data pipeline. This talk will discuss the lifecycle of projects using Jupyter notebooks & Luigi as a data pipeline management tool for a variety of projects, from greenfield to retrofitting complex systems. It will included a hands on demo.
Data pipelines are hard. Too often we resort to retrofitting janky scripts, relying on keeping a readme up to data, etc.
First, this proposal lays out the variety of tools that are available to build data pipelines. This talk will discuss why you should be using Luigi and how to use it in a variety of common use cases.
Next, we will build a basic exploratory analysis using DC open data and Luigi to demonstrate the power of this concept and how it works with Jupyter.
Finally, we'll retrofit a larger, more complex project to use Luigi to show how you can use it in bigger organizations. 00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.
Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVi...
Повторяем попытку...
Доступные форматы для скачивания:
Скачать видео
-
Информация по загрузке:
