RADAR - A Proactive Decision Support System for Human-in-the-Loop Planning

Proactive Decision Support (PDS) aims at improving the decision making experience of human decision makers by enhancing both the quality of the decisions and the ease of making them. In this paper, we ask the question what role automated decision making technologies can play in the deliberative process of the human decision maker. Specifically, we focus on expert humans in the loop who now share a detailed, if not complete, model of the domain with the assistant, but may still be unable to compute plans due to cognitive overload. To this end, we propose a PDS framework RADAR based on research in the automated planning community that aids the human decision maker in constructing plans. We will situate our discussion on principles of interface design laid out in the literature on the degrees of automation and its effect on the collaborative decision making process. Also, at the heart of our design is the principle of naturalistic decision making which has been shown to be a necessary requirement of such systems, thus focusing more on providing suggestions rather than enforcing decisions and executing actions. We will demonstrate the different properties of such a system through examples in a fire-fighting domain, where human commanders are involved in building response strategies to mitigate a fire outbreak. The paper is written to serve both as a position paper by motivating requirements of an effective proactive decision support system, and also an emerging application of these ideas in the context of the role of an automated planner in human decision making, in a platform that can prove to be a valuable test bed for research on the same. Human-in-the-loop planning or HILP (Kambhampati and Talamadupula 2015) is a necessary requirement today in many complex decision making or planning environments. In this paper, we consider the case of HILP where the human responsible for making the decisions in complex scenarios are supported by an automated planning system. Highlevel information fusion that characterizes complex longterm situations and support planning of effective responses is considered the greatest need in crisis-response situations (Laskey, Marques, and da Costa 2016). Indeed, automated planning based proactive support was shown to be preferred by humans involved in teaming with robots (Zhang et al. 2015) and the cognitive load of the subjects involved was observed to have been reduced (Narayanan et al. 2015). We note that the humans are in the driver’s seat in generating plans.We investigate the extent to which an automated Figure 1: Planning for decision support involves iterative and the need to consider difference of models between the planner and the human in the loop. planner can support the humans in planning, despite not having access to the complete domain and preference models. This is appropriate in many cases, where the human in the loop is ultimately held responsible for the plan under execution and its results. This is in contrast to earlier work on systems such as TRAINS and MAPGEN (Allen 1994; Ai-Chang et al. 2004), where the planner is in the drivers seat, with the humans ”advising” the planner. It is also a far cry from the earlier work on mixed-initiative planning where humans enter the land of automated planners and manipulate their internal search data structures. In our framework, the planners have to enter the land of humans. An important complication arises due to the fact that the planner and the human can have different (possibly complementary) models of the same domain or knowledge of the problem at hand, as shown in Figure 1. In particular, humans might have additional knowledge about the domain as well as the plan preferences that the automated planner is not privy to. This means that plan suggestions made by the automated planner may not always make sense to the human in the loop, i.e. appear as suboptimal in her domain. This Figure 2: Degrees of automation of the various stages of decision support, and the role of RADAR in it. can occur either when the human or the planner has a faulty model of the world. This is an ideal opportunity to provide model updates or explanations and reconcile this model difference through iterative feedback from the human. This calls for active participation from the human in the loop rather than simply adopting a system generated plan. Though having to deal with an incomplete model is the usual case in many mixed initiative settings, i.e. an automated support component, without a full model, cannot actually generate entire plans from scratch but can sometimes complete or critique existing ones the extent to which a planner can be of help is largely dependent on the nature of the model that is available. Keeping this in mind, in the current paper we focus on scenarios which come with more well-defined protocols or domain models, and illustrate how off-the-shelf planning techniques may be leveraged to provide more sophisticated decision support. Examples where such technologies can be helpful include any complex tasks, especially disaster response or emergency situations, where the mental overload of the human (either due to the complexity of the problem at hand or the sheer volume of data that needs to be considered to make an informed decision) can affect the quality of successful recovery. To this end, we propose a proactive decision support (PDS) system RADAR following some of the design principles laid out in the literature in the human-computer interface community, to demonstrate possible roles that existing automated planning technologies can play in the deliberative process of the human decision maker in terms of the degree of automation of the planning process it affords. Naturalistic Decision Making The proposed proactive decision support system supports naturalistic decision making (NDM), which is a model that aims at formulating how humans make decisions is complex time-critical scenarios (Zsambok and Klein 2014; Klein 2008). It is acknowledged as a necessary element in PDS systems (Morrison et al. 2013). Systems which do not support NDM have been found to have detrimental impact on work flow causing frustration to decision makers (Feigh et al. 2007). At the heart of this concept is, as we discussed before, the requirement of letting the human be in control. This motivates us to build a proactive decision support system, which focuses on aiding and alerting the human in the loop with his/her decisions rather than generate a static plan that may not work in the dynamic worlds that the plan has to execute in. In cases when the human wants the planner to generate complete plans, he still has the authority to ask RADAR to explain its plan when it finds it to be inexplicable (Chakraborti et al. 2017). We postulate that such a system must be augmentable, context sensitive, controllable and adaptive to the humans decisions. Various elements of human-automation interaction such as, adaptive nature and context sensitivity are presented in (Sheridan and Parasuraman 2005). (Warm, Parasuraman, and Matthews 2008) show that vigilance requires hard mental work and is stressful via converging evidence from behavioral, neural and subjective measures. Our system may be considered as a part of such vigilance support thereby reducing the stress for the human. Degrees of Automation One of the seminal works by (Sheridan and Verplank 1978), builds a model that enumerates ten levels of automation in software systems depending on the autonomy of the automated component. Later, in the study of mental workload and situational awareness of humans performing alongside automation software, (Parasuraman 2000) separates automation into four partsInformation Acquisition, Information Analysis, Decision Selection and Action Implementation (see Figure 2). We use this system as an objective basis for deciding which functions for our system should be automated and to what extent so as to reduce human’s mental overload while supporting Naturalistic Decision making. (Parasuraman and Manzey 2010) shows that human use of automation may result in automation bias leading to omission and commission errors, which underlines the importance of reliability of the automation (Parasuraman and Riley 1997). Indeed, it is well known (Wickens et al. 2010), that climbing the automation ladder in Figure 2 might well improve operative performance but drastically decrease the response to failures or mistakes. Hence, to meet