MedAI Session 16: Bootstrapped Self-Supervised Representation Learning in Graphs | Shantanu Thakoor

Описание: Title: Bootstrapped Self-Supervised Representation Learning in Graphs

Speaker: Shantanu Thakoor

Abstract:
Self-supervised graph representation learning aims to construct meaningful representations of graph-structured data in the absence of labels. Current state-of-the-art methods are based on contrastive learning, and depend heavily on the construction of augmentations and negative examples. Achieving peak performance requires computation quadratic in the number of nodes, which can be prohibitively expensive. In this talk, we will present Bootstrapped Graph Latents (BGRL) a method for self-supervised graph representation learning that gets rid of this potentially quadratic bottleneck. We show that BGRL outperforms or matches previous methods on several established benchmark datasets, while consuming 2-10x less memory. Moreover, it enables the effective usage of more expressive GNN architectures, allowing us to further improve the state of the art. Finally, we will present our recent results on applying BGRL to the very large-scale data regime, in the OGB-LSC KDD Cup, where it was key to our entry being among the top 3 awardees our track.

Speaker Bio:
Shantanu is a Research Engineer working at DeepMind. His primary research interests are in graph representation learning and reinforcement learning. Prior to this, he received his MS from Stanford University, where he was working on AI safety and neural network verification, and B.Tech. from IIT Bombay, where he worked on program synthesis. Recently, he has been interested in applying graph representation learning methods to large-scale problems, including the OGB Large Scale Challenge.

------

The MedAI Group Exchange Sessions are a platform where we can critically examine key topics in AI and medicine, generate fresh ideas and discussion around their intersection and most importantly, learn from each other.

We will be having weekly sessions where invited speakers will give a talk presenting their work followed by an interactive discussion and Q&A. Our sessions are held every Thursday from 1pm-2pm PST.

To get notifications about upcoming sessions, please join our mailing list: https://mailman.stanford.edu/mailman/...

For more details about MedAI, check out our website: https://medai.stanford.edu. You can follow us on Twitter @MedaiStanford

Organized by members of the Rubin Lab (http://rubinlab.stanford.edu)
Nandita Bhaskhar (https://www.stanford.edu/~nanbhas)
Siyi Tang (https://siyitang.me)