Assurances and Machine Self-Confidence for Enhanced Trust in Autonomous Systems

This work investigates a model-based approach to understanding how user trust evolves in systems consisting of a supervising user and an autonomous agent. This model consists of a multivariate model for user trust, and a feedback connection between user and agent. Feedback information is termed assurance, which is also shown to consist of multiple aspects concerning the state of the autonomous agent. It is argued that the closed loop interactions between user and agent can and should be designed to calibrate user trust. In order to develop design principles, it is first necessary to define the terms and salient components of these models to provide a logical framework for their interconnection. Although elements such as trust and assurance are essential in a usable autonomous system, they are also nebulous concepts with multiple meanings [1]. We provide definitions and structure that enables a systematic study of the problem. Our user trust model implies that better assurances can be designed by providing users with better insight into the ‘competency boundaries’ for key decision-making components of the autonomy. One potentially important assurance is a report of the self-confidence (i.e. selftrust) the autonomy has in its own process. We are currently developing formal computational mechanisms for assessing machine self-confidence as an assurance in the context of a probabilistic autonomous route planning.