Datasets through the looking glass - S09E02 - Abhishek Singh Sambyal

Описание: Datasets through the L👀king-Glass is a webinar series focused on reflecting on the data-related facets of Machine Learning (ML) methods. We are building a community of enthusiastic researchers who care about understanding the impact that data and ML methods could have on our society. The webinar is part of the “Making MetaDataCount” project and was originally organized by Dr. Veronika Cheplygina and Dr. Amelia Jiménez-Sánchez at the IT University of Copenhagen. The webinar goes beyond the project since Théo Sourget, also affiliated at the IT University of Copenhagen, and Steff Groefsema from the University of Groningen have joined the organizational team.

In this ninth edition, which took place on 2th March, we discuss evaluation metrics.

Abstract: Deep neural networks have achieved impressive performance in medical image classification, yet their confidence estimates are often poorly calibrated, limiting their reliability in clinical practice. In high-risk medical settings, accurate predictions must be accompanied by trustworthy uncertainty estimates. This work investigates how different training strategies, including fully supervised learning and rotation-based self-supervised pretraining with and without transfer learning, influence the calibration behavior of deep neural networks. A comprehensive empirical analysis across multiple medical imaging datasets reveals that self-supervised pretraining can significantly improve confidence reliability while maintaining competitive predictive performance. The findings provide practical insights into the relationship between representation learning, training dynamics, and calibration, highlighting pathways toward building more trustworthy medical AI systems.