How I-JEPA Works

Описание: We dive into the I-JEPA Paper by Meta AI, McGill, Mila, and NYU, a non-generative approach for self-supervised learning from images.

--

Get Oxen 🐂 https://oxen.ai/

Oxen.ai makes versioning your datasets as easy as versioning your code! Even is millions of unstructured images, we quickly handle any type of data so you can build cutting-edge AI.

--

ImageNet 🔢 https://www.oxen.ai/ox/ImageNet-1k

Paper 📜 https://arxiv.org/abs/2301.08243

I-JEPA 📜 https://www.oxen.ai/blog/arxiv-dives-...

Masked Auto Encoder 📜 https://arxiv.org/abs/2111.06377

Latent Space Diagram 📜   / vision-transformer-vit-under-the-magnifyin...  

Join Arxiv Dives 🤿 https://oxen.ai/community

Discord 🗿   / discord  

--

Chapters
0:00 Intro to I-JEPA
2:05 Semantic Image Representations
3:05 Latent Representation
4:30 Invariance Based Pre-Training
5:45 Generative Pre-Training
6:53 What is I-JEPA
9:07 I-JEPA vs. Previous Approaches
11:55 ViT Method
13:00 Sampling Context and Targets
15:35 Prediction and Loss
17:30 Latent Space
19:49 Attention Head
22:50 Evaluation on Image Classification
30:20 Conclusion and Conversation