The Responsibility Quantification (ResQu) Model of Human Interaction with Automation

Advanced automation is involved in information collection and evaluation, in decision-making and in the implementation of chosen actions. In such systems, human responsibility becomes equivocal, and there may exist a responsibility gap. Understanding human responsibility is particularly important when systems can harm people, as with autonomous vehicles or, most notably, with Autonomous Weapon Systems (AWS). Using Information Theory, we develop a responsibility quantification (ResQu) model of human interaction in automated systems and demonstrate its applications on decisions involving AWS. The analysis reveals that human comparative responsibility is often low, even when major functions are allocated to the human. Thus, broadly stated policies of keeping humans in the loop and having meaningful human control are misleading and cannot truly direct decisions on how to involve humans in advanced automation. Our responsibility model can guide system design decisions and can aid policy and legal decisions regarding human responsibility in highly automated systems.