SREcon26 Americas - Reliable OpenTelemetry at Scale: No Queue, No Problem

Описание: Reliable OpenTelemetry at Scale: No Queue, No Problem

Tommy Li and Vlad Seliverstov, ClickHouse

SREs are expected to ingest and analyze massive streams of metrics, logs and traces across multi-tenant Kubernetes environments. Traditional approaches rely on central message queues and/or vendor pipelines. These struggle with cost, reliability and operational overhead at the GB/s scale. In this talk, we present a practical reference architecture for a queue-less OpenTelemetry pipeline built entirely on Kubernetes using the OpenTelemetry collector, operator and OpAMP. We explore how we run a large fleet of collectors, rollout config changes safely without disrupting ingestion and handle failure without data loss. Our system uses backpressure, autoscaling and object storage for overflow to support high throughput without needing Kafka or Pulsar. You’ll learn concrete pipeline configuration, schema design choices and practices we use that make it possible to support trillions of events per day reliably at reasonable cost.

View the full SREcon26 Americas program at https://www.usenix.org/conference/sre...