The Internet X-Ray: Diagnosing ECMP failures from the edge.

Описание: Automattic (AS2635) operates an Anycast CDN utilizing common internet transit links for datacenter interconnection (DCI) duties. Availability is handled by having a large variety of DFZ connections at each datacenter.

A huge issue with this strategy is the propensity for the internet to fail in ways that are not immediately obvious due to the heavy utilization of multi-pathing (ECMP, LACP, etc) technologies that can hide congestion and errors in the aggregation of many links by network service providers.

To find and visualize these issues, Automattic has designed and deployed internet monitoring software that successfully enumerates ECMP links on the open internet and allows us to find and quantify single-link failures deep within NSP networks. We have been using this software for some time to work with NSP's to aid them in more rapidly remediating their networks.

This presentation will present the basics of Anycast CDN operations, give a refresher on router flow-hashing and ECMP path selection and then delve into several case studies showing how Automattic's PINGO system is able to visualize ECMP failures deep inside the internet. We'll present several failures and show how we're able to track these issues down before the NSP's themselves.

Find Pingo on Github at https://github.com/automattic/pingo

Speaker: Tyler Leeds