How to find and measure AI's weak spots [Research, NAACL 2025 Outstanding Paper]
Автор: Jordan Boyd-Graber
Загружено: 2025-04-30
Просмотров: 532
Описание:
Adversarial datasets should validate AI robustness by presenting samples that humans handle well but models struggle with. However, as models advance, these datasets risk becoming obsolete. Assessing whether a dataset remains adversarial is challenging due to the absence of a standardized metric for adversarialness. To address this, we introduce AdvScore, a human-grounded evaluation metric that quantifies a dataset's adversarial nature by accounting for the differing abilities of models and humans while also identifying low-quality examples.
Publication information:
Yoo Yeon Sung, Maharshi Gor, Eve Fleisig, Ishani Mondal, and Jordan Lee Boyd-Graber. ADVSCORE: A Metric for the Evaluation and Creation of Adversarial Benchmarks. North American Association for Computational Linguistics, 2025.
Read the full paper:
http://cs.umd.edu/~jbg//docs/2025_naa...
Take Part in Human-Computer QA Competitions:
http://qanta.org
Повторяем попытку...
![How to find and measure AI's weak spots [Research, NAACL 2025 Outstanding Paper]](https://i.ytimg.com/vi/FHeFVNXeTyg/maxresdefault.jpg)
Доступные форматы для скачивания:
Скачать видео
-
Информация по загрузке:
![Can AI write Questions that it can't Answer? [Research, NAACL 2025]](https://image.4k-video.ru/id-video/TT4Tim2UeRg)
![The bad habit from high school that's making AI dumb [ACL 2025, Research]](https://image.4k-video.ru/id-video/XrA0jGuLIVQ)