AI Agents 8 - Evaluation, Cost and Scalability
Автор: Prof. Ghassemi Lectures and Tutorials
Загружено: 2025-11-02
Просмотров: 253550
Описание:
In this lecture, Dr. Mohammad Ghassemi explains how to evaluate, optimize, and scale AI agents built with large language models (LLMs). Starting from first principles, he outlines when LLMs should be used, how to select and test models, and how to manage cost-performance tradeoffs. Using the problem of extracting scientific knowledge from 64 million papers since 1996, he demonstrates step-by-step strategies to reduce costs from millions of dollars and centuries of compute to minutes and a few thousand dollars—through parallelization, smaller models, and targeted data retrieval.
Topics include:
Benchmarking LLMs using leaderboards and custom tests
Practical evaluation methods (human, LLM, and metric-based)
Cost modeling and scalability in real systems
Data and tool management via Model Context Protocol (MCP)
Повторяем попытку...
Доступные форматы для скачивания:
Скачать видео
-
Информация по загрузке:
