Safety Testing of the AI Agent: Vulnerabilities and Attacks Beyond the Chatbot

Описание: Alexander Borodetskiy, VP of Growth, AI Safety, Toloka AI

Abstract:
AI agents are evolving beyond simple chat, browsing the web and interacting with your computer. But how do we ensure they're safe? We'll explore a new safety evaluation framework, revealing how specially crafted web pages, files, and OS environments can be used to expose agent vulnerabilities. See how prompt injections can manipulate agents into leaking sensitive data, how to organize testing of an agent for performing unsafe mistakes - and how they might even be hijacked for malicious activities. This technical deep dive reveals the approach to conduct safety testing of today's leading AI agents and provides insights into building more robust and safe AI systems.