Applying STPA to Improve Safety and Reliability of AI-Enabled Systems

Описание: This talk explores the application of System-Theoretic Process Analysis (STPA) to enhance the safety and reliability of AI-enabled systems. STPA provides a structured and scalable approach to identifying system-level hazards and analyzing risks in highly complex systems, such as those involving large language models (LLMs). By addressing unsafe interactions and potentially hazardous scenarios early in the development lifecycle, STPA enables the design of safer, more robust AI systems.

The effectiveness of this methodology was tested by implementing STPA on an open-source intelligence (OSINT) tool designed for military intelligence applications. The analysis identified a comprehensive set of unsafe control actions (UCAs) that could lead to critical failures. The UCAs were addressed with targeted safety interventions that enabled risk mitigation and improved system reliability, while also ensuring accurate, responsible handling of critical data. The approach described in this presentation extends beyond this use case, offering a roadmap for applying STPA to enhance safety and reliability in a wide range of AI-enabled systems.

Presentation slides are available at: https://softwareexcellencealliance.or...

------------------------------------------------------------------

About the presenter: Timothy Davison is a data scientist at the Software Engineering Institute (SEI), where he works on developing machine learning and data science solutions for cybersecurity challenges, as well as enhancing the safety and reliability of AI systems through methodologies like System-Theoretic Process Analysis (STPA).

At the SEI, Timothy contributes to projects involving computer vision, large language models, and safety analysis for AI systems in military and defense applications. His work blends technical innovation with a focus on system safety, ensuring advanced AI technologies are robust and secure.

Before joining the SEI, Timothy served seven years as an officer in the U.S. Army, where he gained valuable leadership and organizational skills and was among the first to work at the Army Artificial Intelligence Task Force as a data scientist.

Timothy holds a Bachelor of Science in Computer Science from the Virginia Military Institute (VMI) and a Master’s degree in Data Science from Carnegie Mellon University.