Decision-Centered Testing (DCT): Evaluating Joint Human-Computer Cognitive Work

In order to test the effectiveness of a human operator paired with a decision support system, it is necessary to complement current testing practices addressing software validation, human performance, and usability. Decision Centered Testing (DCT) aims at testing the effectiveness of operators teamed with Decision Support Systems (DSS) in any challenging work domain. DCT is grounded in a Cognitive Systems Engineering (CSE) framework, where the concept of a joint cognitive system (JCS) is central. DCT aims at evaluating the decision-making effectiveness across identified 'error prone' regions in the JCS structure. A description of the DCT Methodology with an illustration taken from an initial application of the methodology is presented. In this application, insights from the DCT methodology enabled the definition of appropriate test metrics and the construction of unique test scenarios to exercise the decision-making effectiveness. From this application, it can be concluded that following the DCT Methodology facilitated the construction of an evaluation framework for assessing JCS net decision-making effectiveness.