A Framework for Discussing Trust in Increasingly Autonomous SystemsJune 2017
Topics: Computing Methodologies, Systems Engineering
We are seeing an increase in the complexity, sophistication, and interconnectedness of automation used in a variety of everyday applications from transportation to medical diagnostics. These increasingly autonomous systems depend upon software, data, and networked communications for safe, secure, and efficient operations. They must continue to function appropriately in the face of design defects, unanticipated situations, faulty/missing data, and deliberate attacks because their operational failure could have dire consequences. MITRE recognized that there was a need for a comprehensive framework for discussing trust in these increasingly autonomous systems. This framework shown in the figure on the next page was first documented in a paper presented at the Association for the Advancement of Artificial Intelligence (AAAI) 2014 Spring Symposium.1
We have used this framework to discuss the trust humans place in these cyber-intensive systems and trustworthiness of the systems themselves. The goal of this paper is to introduce this framework and explain its elements and relationships.