When AI Discovers the Next Transformer — Robert Lange

Описание: Robert Lange, founding researcher at Sakana AI, joins Tim to discuss Shinka Evolve — a framework that combines LLMs with evolutionary algorithms to do open-ended program search. The core claim: systems like AlphaEvolve can optimize solutions to fixed problems, but real scientific progress requires co-evolving the problems themselves.

GTC is coming, the premier AI conference, great opportunity to learn about AI. NVIDIA and partners will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference, exploring the next wave of AI innovation for developers and researchers. Register for virtual GTC for free, using my link and win NVIDIA DGX Spark (https://nvda.ws/4qQ0LMg)

In this episode:

• Why AlphaEvolve gets stuck — it needs a human to hand it the right problem. Shinka tries to invent new problems automatically, drawing on ideas from POET, PowerPlay, and MAP-Elites quality-diversity search.

• The architecture of Shinka: an archive of programs organized as islands, LLMs used as mutation operators, and a UCB bandit that adaptively selects between frontier models (GPT-5, Sonnet 4.5, Gemini) mid-run. The credit-assignment problem across models turns out to be genuinely hard.

• Concrete results — state-of-the-art circle packing with dramatically fewer evaluations, second place in an AtCoder competitive programming challenge, evolved load-balancing loss functions for mixture-of-experts models, and agent scaffolds for AIME math benchmarks.

• Are these systems actually thinking outside the box, or are they parasitic on their starting conditions? When LLMs run autonomously, "nothing interesting happens." Robert pushes back with the stepping-stone argument — evolution doesn't need to extrapolate, just recombine usefully.

• The AI Scientist question: can automated research pipelines produce real science, or just workshop-level slop that passes surface-level review? Robert is honest that the current version is more co-pilot than autonomous researcher.

• Where this lands in 5-20 years — Robert's prediction that scientific research will be fundamentally transformed, and Tim's thought experiment about alien mathematical artifacts that no human could have conceived.

Robert Lange: https://roberttlange.com/

---
TIMESTAMPS:
00:00:00 Introduction: Robert Lange, Sakana AI and Shinka Evolve
00:04:15 AlphaEvolve's Blind Spot: Co-Evolving Problems with Solutions
00:09:05 Unknown Unknowns, POET, and Auto-Curricula for AI Science
00:14:20 MAP-Elites and Quality-Diversity: Shinka's Evolutionary Architecture
00:28:00 UCB Bandits, Mutations and the Vibe Research Vision
00:40:00 Scaling Shinka: Meta-Evolution, Democratisation and the Three-Axis Model
00:47:10 Applications, ARC-AGI and the Future of Work
00:57:00 The AI Scientist and the Human Co-Pilot: Who Steers the Search?
01:06:00 AI Scientist v2, Slop Critique and the Future of Scientific Publishing

---
REFERENCES:
paper:
[00:03:30] ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
https://arxiv.org/abs/2509.19349
[00:04:15] AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery
https://arxiv.org/abs/2506.13131
[00:06:30] Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
https://arxiv.org/abs/2505.22954
[00:09:05] Paired Open-Ended Trailblazer (POET)
https://arxiv.org/abs/1901.01753
[00:10:00] PowerPlay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem
https://arxiv.org/abs/1112.5309
[00:10:40] Automated Capability Discovery via Foundation Model Self-Exploration
https://arxiv.org/abs/2502.07577
[00:15:30] Illuminating Search Spaces by Mapping Elites (MAP-Elites)
https://arxiv.org/abs/1504.04909
[00:47:10] Automated Design of Agentic Systems (ADAS)
https://arxiv.org/abs/2408.08435
[00:49:50] Discovering Preference Optimization Algorithms with and for Large Language Models (DiscoPOP)
https://arxiv.org/abs/2406.08414
[00:57:00] The AI Scientist v2: Automating the Full Research Pipeline
https://arxiv.org/abs/2504.08066
book:
[00:06:48] Why Greatness Cannot Be Planned
https://link.springer.com/book/10.100...
benchmark:
[00:47:10] ALE-Bench: A Benchmark for Long-Horizon Objective-Driven Algorithm Engineering
https://arxiv.org/abs/2506.09050
[00:50:50] On the Measure of Intelligence (ARC-AGI)
https://arxiv.org/abs/1911.01547

---
LINKS:
Download PDF transcript: https://app.rescript.info/api/session...
Full Transcript: https://app.rescript.info/public/shar...