Cam Allen - The Agent Must Choose the Problem Model

Описание: RL & Agents Reading Group | 24 July 2025

Speaker(s): Cam Allen
Title: The Agent Must Choose the Problem Model

Abstract:
Reinforcement learning agents have it easy. Their problem model comes pre-specified from the first time step of their deployment. Observations, actions, rewards—even the learning algorithm—are pre-arranged, expert-designed, and hand-tuned to help the agent accomplish its task. But models can be wrong. If the agent's problem model turns out to be inadequate, no one is coming to help. Autonomous agents must adapt. How can we build agents that handle such daunting ambiguity? I'll present some initial progress in that direction: an agent that can detect when its observations are incomplete and learn a memory function to compensate. The result is a first step towards agents that choose their own problem models.

Main paper:
Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy (https://camallen.net/files/lambda_dis...)

Supporting papers:
Learning Markov State Abstractions for Deep Reinforcement Learning (https://camallen.net/files/markov_sta...)
Memory as State Abstraction over Trajectories(https://camallen.net/files/traj_abstr...)

Bio:
Cam Allen is a postdoctoral fellow at the Center for Human-Compatible Artificial Intelligence at the University of California, Berkeley. His research centers on problem formalization: what are the right conceptual models to express the problems we are trying to solve, and how can we build agents that help us model and solve those problems? More generally, Cam is interested in the foundations of intelligence—the computations that enable agency, learning, planning, abstraction, and interaction. He completed his PhD in Computer Science at Brown University in 2023, where he studied structured abstractions for general-purpose decision making, and he co-taught Berkeley's introductory AI course in spring 2024.