The AI Memory Wall: Why LLMs & AI Accelerators Hit a Bottleneck

Описание: Learn more about the Memory Wall here → https://ibm.biz/~NjOWdJHJy

More GPU compute doesn't always mean faster AI models.
Kaoutar El Maghraoui explains the memory wall, arithmetic intensity, and why LLM inference can become memory-bound or compute-bound.
Learn how memory bandwidth, batching, and the roofline model impact AI performance.

00:00 Why Faster Chips Don’t Always Make AI Workloads Faster
00:32 The Memory Wall: The Hidden Bottleneck in Modern Computing
01:00 Compute Growth vs Memory Bandwidth Explained
02:39 Arithmetic Intensity: The Metric That Predicts Performance
03:48 Understanding the Roofline Model for AI Optimization
05:27 Compute-Bound vs Memory-Bound Workloads Explained
08:25 LLM Decode vs Prefill: Why Performance Varies
09:48 How Batching Improves AI Inference Efficiency
10:35 Why Leading LLM Serving Systems Rely on Continuous Batching
11:22 Practical Steps to Diagnose and Optimize AI Performance

AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~nnWVjebwV

AI was used in the creation of the transcript and metadata for this video.

#aimodels #aiaccelerator #ai #machinelearning
---------------------------------------------------------------------------------------------------------
Find us on YouTube:
🔵 IBM Technology:    / @ibmtechnology  
🔵 IBM:    / @ibm  
🔵 IBM Developer:    / @ibmdeveloperadvocates