Deep Representation Learning - Yoshua Bengio (MILA, Canada)

Описание: Abstract: How could humans or machines discover high-level abstract representations which are not directly specified in the data they observe? The original goal of deep learning is to enable learning of such representations in a way that disentangles underlying explanatory factors. Ideally, this would mean that high-level semantic factors could be decoded from top-level representations with simple predictors like a linear classifier, trainable from very few examples. However, there are too many ways of representing the same information, and it is thus necessary to provide additional clues to the learner, which can be thought about as priors. We highlight several such priors. One of those priors is that high-level factors measured at different times (or places) have high mutual information, i.e., can be predicted from each other and contain many bits of information. We present recent work in unsupervised representation learning towards maximizing the mutual information between random variables. Finally, we introduce the novel idea that good representations should be robust under changes in distribution and show that this can, in fact, be used in a meta-learning setup to identify causal variables and how they are causally related.

Bio: Yoshua Bengio is recognized as one of the world’s leading experts in artificial intelligence (AI) and a pioneer in deep learning.

Since 1993, he has been a professor in the Department of Computer Science and Operational Research at the Université de Montréal. Holder of the Canada Research Chair in Statistical Learning Algorithms, he is also the founder and scientific director of Mila, the Quebec Institute of Artificial Intelligence, which is the world’s largest university-based research group in deep learning.

His research contributions have been undeniable. In 2018, Yoshua Bengio collected the largest number of new citations in the world for a computer scientist thanks to his many publications. The following year, he earned the prestigious Killam Prize in computer science from the Canada Council for the Arts and was co-winner of the A.M. Turing Prize, which he received jointly with Geoffrey Hinton and Yann LeCun, as well as the Excellence Awards of the Fonds de recherche du Québec – Nature et technologies.

Concerned about the social impact of AI, he actively contributed to the development of the Montreal Declaration for the Responsible Development of Artificial Intelligence.