KAIST-NYU Talk Series on Language Models: He He, James Thorne, Minjoon Seo (Mar 9)

Описание: In this seminar, Prof. He He (NYU), Prof. James Thorne (KAIST), and Prof. Minjoon Seo (KAIST) will give a series of talks about LLM research. In the following panel discussion, three speakers will discuss (1) future directions in LLM research and (2) collaboration ideas between NYU and KAIST.

00:00 Opening
01:20 Evaluating and understanding the in-context learning ability of language models, He He (NYU)
33:31 How do we know what LLMs know?, James Thorne (KAIST)
1:03:19 Updating and Accessing Knowledge in Language Models, Minjoon Seo (KAIST)
1:35:14 Panel Discussion

Evaluating and understanding the in-context learning ability of language models (He He)
Abstract: In-context learning is an emergent ability of large language models that enables adaptation to new tasks without parameter update. Our understanding of how it works and how it arises is still nascent. In this talk, I will first describe a more rigorous evaluation of how well these models can learn to reason in-context through chain-of-thought prompting. Next, I will introduce a simple method to optimize in-context learning using Transformer-based language models as a meta learner. Finally, I will talk about some preliminary work that investigates what properties of the pre-training data impact the resulting in-context learning ability.
Bio: He He is an Assistant Professor of Computer Science and Center for Data Science at New York University. Her current research focuses on improving robustness and truthfulness of NLP models, and building interactive systems that collaborate with humans. Before joining NYU, she obtained her PhD from the University of Maryland, College Park, did a post-doc at Stanford, and spent one year at Amazon.

How do we know what LLMs know? (James Thorne)
Abstract: Recent work highlights a lot of interesting properties about memorization of factual knowledge within large-scale language models. In this short talk, I will give an overview of how we can probe the knowledge stored in LLMs, and how this can be used to help with factual reasoning tasks.
Bio: James Thorne is Assistant Professor the KAIST AI Graduate School where is researching large scale and explainable factual reasoning. James completed his PhD at University of Cambridge with a Graduate Fellowship from Amazon Alexa. There, he founded the FEVER challenge and designed foundational approaches for fact verification.

Updating and Accessing Knowledge in Language Models (Minjoon Seo)
Abstract: As language models become an important interface for us to access the world knowledge, how they updates its internal knowledge as the world changes has become increasingly important. In this talk, I will introduce two recent projects in which we setup a benchmark for this and explore various methods that could attain new knowledge while minimize catastrophic forgetting.
Bio: Minjoon Seo is an Assistant Professor at KAIST AI, where he is the Director of Language & Knowledge Lab. He is interested in how the world knowledge can be encoded (e.g. external memory and language model), accessed (e.g. question answering and dialog), and produced (e.g. reasoning). He obtained PhD in Computer Science at the University of Washington where he was fortunate to be advised by Hannaneh Hajishirzi and Ali Farhadi and supported by Facebook Fellowship and AI2 Key Scientific Challenges Award. Before that, he obtained BS in Electrical Engineering & Computer Science at the University of California, Berkeley.