David Abel - A Definition of Continual Reinforcement Learning

Описание: UoE RL Reading Group | 2 May 2024

Speaker: David Abel (Google DeepMind)

Title: A Definition of Continual Reinforcement Learning

Authors: David Abel, André Barreto, Benjamin Van Roy, Doina Precup, Hado van Hasselt, Satinder Singh

Abstract: In a standard view of the reinforcement learning problem, an agent’s goal is to efficiently identify a policy that maximizes long-term reward. However, this perspective is based on a restricted view of learning as finding a solution, rather than treating learning as endless adaptation. In contrast, continual reinforcement learning refers to the setting in which the best agents never stop learning. Despite the importance of continual reinforcement learning, the community lacks a simple definition of the problem that highlights its commitments and makes its primary concepts precise and clear. To this end, this paper is dedicated to carefully defining the continual reinforcement learning problem. We formalize the notion of agents that “never stop learning” through a new mathematical language for analyzing and cataloging agents. Using this new language, we define a continual learning agent as one that can be understood as carrying out an implicit search process indefinitely, and continual reinforcement learning as the setting in which the best agents are all continual learning agents.

Link: https://arxiv.org/abs/2307.11046

Bio: David Abel is a Senior Research Scientist at DeepMind in the UK, based in Edinburgh. Before that, he completed his Ph.D in Computer Science at Brown University.