Enabling End-to-End LLMOps on Michelangelo with Ray

Описание: Generative AI that is driven by LLMs becomes critical for companies to succeed in the modern AI era. As a leading tech company in mobility and delivery, Uber has extensive domain-specific knowledge and many LLM use cases for improving magical user experience and employee productivity.

To achieve this, we are extending Uber's ML platform (Michelangelo) to support end-to-end LLMOps experience. We have established a scalable and interactive development environment capable of utilizing hundreds of A100 GPUs on Ray with great flexibility. By integrating various open-source techniques for LLM training, evaluation and serving, we have significantly enhanced our capability to efficiently develop Uber's custom models based on state-of-the-art LLMs such as LLama2.

Find the slide deck here: https://drive.google.com/file/d/19BPd...


About Anyscale
---
Anyscale is the AI Application Platform for developing, running, and scaling AI.

https://www.anyscale.com/

If you're interested in a managed Ray service, check out:
https://www.anyscale.com/signup/

About Ray
---
Ray is the most popular open source framework for scaling and productionizing AI workloads. From Generative AI and LLMs to computer vision, Ray powers the world’s most ambitious AI workloads.
https://docs.ray.io/en/latest/


#llm #machinelearning #ray #deeplearning #distributedsystems #python #genai