Quantifying Foundation Model Robustness: the Robustness Index: Edwin de Jong, 23/06/25

Описание: TIA Centre Seminar Series: Dr Edwin D. de Jong

Full Title: Quantifying Foundation Model Robustness against Medical Center Variation: the Robustness Index

Abstract: Pathology Foundation Models (FMs) hold great promise for healthcare. Their clinical application is challenged by well-documented variations between medical centers, which can potentially lead to biased downstream models. To enable the use of pathology FMs in clinical practice, it is essential to ensure they are robust to such variation. This requires the ability to measure and quantify robustness. We measure how strongly current pathology FMs encode biological features like tissue and cancer type, and how strongly they encode confounding medical center signatures introduced by staining procedure and other differences. The relation between these two factors defines the Robustness Index. This novel robustness metric quantifies to what degree biological features dominate confounding features. We evaluate the robustness of current pathology FMs. We find that all FMs encode medical centers to some degree, and that medical center variation can dominate the organization of the embedding space, depending on the application domain. The influence of this confounding information on the embedding space varies, and significant differences in the robustness index are observed. The robustness index provides a critical new benchmark for evaluating and improving pathology FMs, accelerating progress towards their safe and reliable clinical adoption.

Paper Link: https://arxiv.org/html/2501.18055v1