How We Trained a Robot to Do 50 Household Tasks in Simulation (BEHAVIOR Challenge 1st place)

Описание: 🏆 BEHAVIOR Challenge (NeurIPS 2025) Winner Deep Dive

Recently, me and my teammates Gleb and Akash won 1st place in the BEHAVIOR Challenge organized by Stanford Vision and Learning Lab (NeurIPS 2025). We already open-sourced the full solution: code, technical report, blog post with rollouts, and all pretrained weights. In this video, I go through the solution in more detail.

BEHAVIOR Challenge is a large-scale robotics benchmark: you get 50 household tasks in high-quality simulation (OmniGibson + NVIDIA Isaac Sim) and 1,200+ hours of teleoperation data. This is the scale where modern VLA policies start to really benefit from transfer learning and generalization. I’ll explain what the challenge is, what we built (architecture, training, inference), and show a lot of real autonomous rollouts (successes and failures).

🎯 What you’ll learn in this video:
✅ What BEHAVIOR is (robot, simulator, dataset, evaluation)
✅ Why we started from Pi0.5 and what we changed (architecture, training, inference):
Task embeddings instead of language prompts
A simple deterministic System 2 progress tracker
Gripper retry heuristic
Correlated noise for flow matching
Trainable VLM layer mixing for the action expert attention
Inference tricks: soft chunk inpainting + action compression
✅ Results: leaderboard, per-task patterns, and failure analysis

🎯 Chapters:
00:00 Intro
01:12 BEHAVIOR overview
04:00 Dataset + task examples
10:57 Solution overview
15:58 System 2
29:01 Actions correlation and flow matching
45:49 Trainable mixed-layer attention
58:12 Inference optimization
1:04:23 Results and autonomous rollouts
1:25:43 Summary and outro

🔗 Open-Source Solution & Resources:
• Code: https://github.com/IliaLarchenko/beha...
• Pretrained Weights: https://huggingface.co/IliaLarchenko/...
• Technical Report: https://arxiv.org/abs/2512.06951
• Blog Post: https://robot-learning-collective.git...

🔗 Main Challenge Info:
• Official Website: https://behavior.stanford.edu/challenge/
• Official Repo: https://github.com/StanfordVL/BEHAVIO...
• Teleoperation Hardware: https://behavior-robot-suite.github.io/

📚 Additional References:
• Pi0: https://www.pi.website/blog/pi0
• Pi0.5: https://www.pi.website/blog/pi05
• Real-Time Action Chunking: https://www.pi.website/research/real_...
• Dot Policy (action speedup): https://github.com/IliaLarchenko/dot_...
• Figure AI: Helix (Sport Mode): https://www.figure.ai/news/helix-logi...
• Russ Tedrake talk (timestamp normalization):    • Stanford Seminar - Multitask Transfer in T...  
• Isaac GR00T: https://github.com/NVIDIA/Isaac-GR00T
• SmolVLA: https://huggingface.co/blog/smolvla
• Attention is All You Need (Transformer): https://arxiv.org/abs/1706.03762
• Mixture of Experts: https://arxiv.org/abs/1701.06538

👥 Team:
• Ilia Larchenko: https://x.com/IliaLarchenko
• Gleb Zarin: https://x.com/zaringleb
• Akash Karnatak: https://x.com/akashkarnatak

---

Some footage and images in this video are from or generated by Stanford VL Lab / BEHAVIOR / OmniGibson / NVIDIA Isaac Sim, used for educational purposes. All rights remain with the original creators.

💬 Have questions or suggestions? Drop them in the comments, I read everything!
⭐ Enjoyed the video? Please like & subscribe if you like this content