Genie: Generative Interactive Environments | Jake Bruce

Описание: ICARL Seminar Series - 2024 Spring

Genie: Generative Interactive Environments
Seminar by Jake Bruce

——————————————————
Abstract:

We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

——————————————————
About the Speaker:

Jake has a background in deep learning for robot navigation from Queensland University of Technology in Brisbane, Australia. He has been working on reinforcement learning and large-scale deep learning at Google DeepMind since 2018, and has been involved in projects on imitation learning in NetHack, generalist agents via Gato, and most recently generative world models in Genie.

——————————————————
Links
Jake Bruce
Site: jakebruce.ca

ICARL
Site: icarl.doc.ic.ac.uk
Twitter: x.com/ic_arl
YouTube: @ICARLSeminars
——————————————————