Prof. Manling Li: RAGEN: Training Agents by Reinforcing Reasoning

Описание: Talk Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning tasks, but the emerging Large Agent Models (LAMs) faces unique challenges as these models learn to interact with dynamic environments. This talk explores the fundamental framework for understanding and improving agent decision-making across long-horizon multi-round interactions. We begin by formalizing agent reasoning as a Markov Decision Process (MDP) and introduce the Embodied Agent Interface, a standardized framework for studying core agent capabilities including goal interpretation, subgoal decomposition, action sequencing, and transition modeling. Through this lens, we identify long-horizon decision making as a critical bottleneck that requires specialized training approaches. To address this challenge, we present RAGEN, a novel framework that is inspired by the recent success of DeepSeek-R1(Zero) using rule-based reward in reinforcement learning. RAGEN tackles two key challenges in real-world agent scenarios: environmental non-deterministic reward and long-horizon multi-turn interactions. To handle visual states, we introduce VAGEN to formulate the problem as a Partially Observable Markov Decision Process, enabling more robust learning in complex visual states.

Bio: Manling Li is an Assistant Professor at Northwestern University. She was a postdoc at Stanford University and obtained the PhD degree in Computer Science at University of Illinois Urbana-Champaign in 2023. She works on the intersection of language, vision, and robotics. Her work won the ACL’24 Outstanding Paper Award, ACL'20 Best Demo Paper Award, and NAACL'21 Best Demo Paper Award. She was a recipient of Microsoft Research PhD Fellowship in 2021, an EE CS Rising Star in 2022,etc. She served as Organizing Committee of ACL 25, NAACL 25, EMNLP 24, and invited as keynote speaker at Amazon-Illinois Center on AI for Interactive Conversational Experiences. Additional information is available at https://limanling.github.io/.