Benefits of understanding Theory of Deep Learning | Dr. Hanie Sedghi

Описание: Hanie is a senior research scientist at‪@Google‬ Brain working on research problems related to understanding and improving deep learning techniques. She works on designing algorithms with theoretical guarantees such that they work efficiently in real world applications. Prior to that she was a research scientist at ‪@allenai‬ and before that she was a Post-Doc fellow at ‪@ucirvine‬. She graduated from USC with a PhD with minors in Mathematics.

Timestamps:
00:00 Introductions
01:07 Can you tell us what kind of research questions you are interested in while working at ‪@Google‬ Brain?
02:45 What was your entry point in deep learning research? What interested you more about the theory of DL vs applied DL research?
04:45 How can theoretical proofs & guarantees help the deep learning community move forward
06:05 What is one thing that surprises/puzzles you about its effectiveness to date?
07:45 What motivates you more about projects, the application associated with them, or the pursuit of theoretical knowledge behind them?
09:10 Can you explain what we mean by over-parameterization and generalization in DNNs?
11:15 What are the advantages vs drawbacks of over-parameterization?
14:00 Exploring the Limits of Large Scale Pre-training: https://arxiv.org/pdf/2110.02095.pdf, intuitive understanding of those results and what that means? And things to keep in mind while working with TL and scaling models.
20:35 choice of upstream and downstream task datasets in order to use transfer learning (TL) efficiently
25:00 What is being transferred in TL?: https://arxiv.org/pdf/2008.11687.pdf, since large models don’t change easily when dealing with small target datasets, should large models be not used with TL? Vs we can’t really do TL with small networks
29:10 Can you give us instance/s of such theoretical investigations and the benefits of those works?
32:00 Vision-Transfomers vs CNNs, differences from a fundamental perspective
34:10 Is causal reasoning related to generalization in anyways? How?
36:50 Working as a researcher in academia/student vs in industry
40:25 Remaining updated to progress in DL research and not being overwhelmed
43:00 What accounts for a good PhD thesis?
48:35 Any piece of advice you have for graduate students trying to explore and find interest in DL research?

Dr. Hanie Sedghi's links
Twitter: https://twitter.com/haniesedghi?ref_s...
Homepage: https://haniesedghi.com/

Also check-out these talks on all available podcast platforms: https://jayshah.buzzsprout.com

About the Host:
Jay is a PhD student at Arizona State University, doing research on building Interpretable AI models for Medical Diagnosis.
Jay Shah:   / shahjay22  

You can reach out to https://www.public.asu.edu/~jgshah1/ for any queries.
Stay tuned for upcoming webinars!

#theoryofmachinelearning #deeplearning #ai #machinelearning #fundamentals

**Disclaimer: The information contained in this video represents the views and opinions of the speaker and does not necessarily represent the views or opinions of any institution. It does not constitute an endorsement by any Institution or its affiliates of such video content.**