M. Vierhauser, M. Islam, A. Agrawal, J. Cleland-Huang, J. Mason: Hazard Analysis for Human-on-the-Loop Interactions in sUAS Systems, European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2021), Athens, Greece, August 23-28, 2021, virtual event, Doi: 10.1145/3468264.3468534
With the rise of new AI technologies, autonomous systems are moving towards a paradigm in which increasing levels of responsibility are shifted from the human to the system, creating a transition from human-in-the-loop systems to human-on-the-loop (HoTL) systems. This has a significant impact on the safety analysis of such systems, as new types of errors occurring at the boundaries of human-machine interactions need to be taken into consideration. Traditional safety analysis typically focuses on system-level hazards with little focus on user-related or user-induced hazards that can cause critical system failures. To address this issue, we construct domain-level safety analysis assets for sUAS (small unmanned aerial systems) applications and describe the process we followed to explicitly, and systematically identify Human Interaction Points (HiPs), mission modes and Hazard Factors from system hazards. We evaluate our approach in two ways. First, by investigating the extent to which recent sUAS incidents reported by the media and by regulatory bodies are covered by our hazard trees, and second by performing a study with six domain experts using our hazard trees to identify and document hazards for sUAS usage scenarios. Our study showed that hazards identified by the literature search provided effective coverage for a wide variety of sUAS application scenarios, and that the hazard trees we constructed were useful for stimulating safety-thinking and helping users to identify and potentially mitigate human-interaction hazards.