Daniel Schauenberg: Scaling Deployment at Etsy

Описание: At Etsy we have over 100 engineers deploying more than 40 times a day. This culture of continuously deploying small change sets enables us to build and release robust features all while serving over a billion page views per month. In order to make sure we can keep up this pace, we have development and deployment infrastructure in place that makes it comfortable and simple to make changes. So simple that as an engineer at Etsy you deploy the site on your first day - even if you're a dog.

But how is it possible to deploy so frequently among so many engineers and yet maintain a stable system? To answer this I give a high-level overview of the basic application structure to introduce the specific architecture we have at Etsy and how our development environment is set up. For development, every engineer gets their own VM with the full application stack configured. This makes it easy to get started and puts every developer in the same, familiar setup. This is a crucial part in removing confusion and ambiguity about how to work on and deploy changes.

For the actual deployment we use a one-button deploy system - Deployinator - which we developed and opensourced. We integrated our system tightly with our company-wide IRC server and a set of tools we built to foster confidence, fast feedback and easy communication and collaboration between engineers. A detailed overview will be given on how the system works, how we use it and what problems we had to solve while making sure everyone can deploy as easily and fast as possible.

The central source of confidence in our development process is our CI system. We have developed an easy way for developers to run tests on the CI cluster while developing to test changes before going into the deploy pipeline. Additionally tests are run on deploy and initiated by engineers with a tool called 'try' across 200 build machines. Our cluster setup is explained along with what problems we hit with an ever-growing codebase and number of engineers, and how we made our current setup fast.

Continuous Deployment and the ability to quickly make changes are crucial for moving fast and iterating on products. Our deployment system works very well and is keeping up with the demands of a growing site and team. After attending this talk you will understand exactly how we approach Continuous Deployment and Continuous Integration at Etsy and get insight into what went wrong along the way and how we fixed it.