ViT: повышаем точность. Лекция 26.

Описание: Следующий поток онлайн-школы: осень 2024.
Предварительная регистрация и подробности здесь:
https://razinkov.ai/school

Подпишитесь на телеграм-канал с анонсами стримов:
https://t.me/razinkov_ai

Плейлист "AI: от основ до трансформеров":
   • AI: от основ до трансформеров  

DeiT III: Revenge of the ViT
Touvron, Hugo, Matthieu Cord, and Hervé Jégou. "Deit iii: Revenge of the vit." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.
https://arxiv.org/pdf/2204.07118.pdf

Knowledge Distillation
Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network." arXiv preprint arXiv:1503.02531 (2015).
https://arxiv.org/pdf/1503.02531.pdf

BEiT
Bao, Hangbo, et al. "Beit: Bert pre-training of image transformers." arXiv preprint arXiv:2106.08254 (2021).
https://arxiv.org/pdf/2106.08254.pdf)...

DeiT
Touvron, Hugo, et al. "Training data-efficient image transformers & distillation through attention." International conference on machine learning. PMLR, 2021.
https://arxiv.org/pdf/2012.12877.pdf

LayerScale
Touvron, Hugo, et al. "Going deeper with image transformers." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
https://openaccess.thecvf.com/content...

BYOL
Лекция: https://youtube.com/live/9ld7I_R1o9Y
Grill, Jean-Bastien, et al. "Bootstrap your own latent-a new approach to self-supervised learning." Advances in neural information processing systems 33 (2020): 21271-21284.
https://proceedings.neurips.cc/paper_...