Self-play for Self-driving and where Scaling Reinforcement Learning is Heading with Eugene Vinitsky

Описание: Eugene Vinitsky is a professor a New York University department of Civil and Urban Engineering. He’s one of my original reinforcement learning friends from when we were both doing our Ph.D.’s in RL at UC Berkeley circa 2020. Eugene has extensive experience in self-driving, open endedness, multi-agent reinforcement learning, and self-play with RL.

Full details, paper links, etc: https://www.interconnects.ai/p/interv...

In this conversation we focus on a few key topics:

His latest results on self-play for self-driving and what they say about the future of RL,

Why self-play is confusing and how it relates to the recent takeoff of RL for language models, and

The future of RL in LMs and elsewhere.

This is a conversation where we take the time to distill very cutting edge research directions down into the core essences. I felt like we were learning in real time what recent developments mean for RL, how RL has different scaling laws for deep learning, and what is truly salient about self-play.

The main breakthrough we discuss is scaling up self-play techniques for large-scale, simulated reinforcement learning. Previously, scaling RL in simulation has become economical in single-agent domains. Now, the door is open to complex, multi-agent scenarios where more diversity is needed to find solutions (in this case, that’s what self play does).

00:00:00 Introduction & RL Fundamentals
00:11:27 Self‑Play for Self‑Driving Cars
00:31:57 RL Scaling in Robotics and Other Domains
00:44:23 Language Models and In-Context Preference Learning
00:55:31 Future of RL and Grad School Advice

Get Interconnects (https://www.interconnects.ai/)...
... on YouTube:    / @interconnects  
... on Twitter: https://x.com/interconnectsai
... on Linkedin:   / interconnects-ai  
... on Spotify: https://open.spotify.com/show/2UE6s7w...
… on Apple Podcasts: https://podcasts.apple.com/us/podcast...