Distributed pub-sub infrastructure with Apache Kafka

Описание: Carl Scheffler
http://www.pyvideo.org/video/3924/dis...
https://za.pycon.org/talks/20/
‹a href="http://kafka.apache.org/"›Apache Kafka‹/a› is great for building a large scale distributed data bus. Even a small cluster will happily accept and store thousands of messages per second, and make them available to consumers with low latency.

Kafka was chosen as the solution to our publish-subscribe infrastructure at ‹a href="http://www.takealot.com/"›Takealot.com‹/a›. It supports our event-driven systems on the website, in the warehouses and in the office, as well as our analytics and machine learning projects.

This talk will

introduce the basic Kafka principles that make things work,
outline how Kafka fits in with the rest of our architecture,
cover some of the practicalities of building Python-based Kafka services,
compare the two main Python libraries for Kafka, namely ‹a href="https://github.com/mumrah/kafka-python"›kafka-python‹/a› and ‹a href="https://github.com/Parsely/pykafka"›pykafka‹/a›,
demonstrate some practical applications at Takealot.com.

Join in if you are interested in scalable distributed infrastructure.