Maximal benefits and possible detrimental effects of binary decision aids

Binary decision aids, such as alerts, are a simple and widely used form of automation. The formal analysis of a user's task performance with an aid sees the process as the combination of information from two detectors who both receive input about an event and evaluate it. The user's decisions are based on the output of the aid and on the information, the user obtains independently. We present a simple method for computing the maximal benefits a user can derive from a binary aid as a function of the user's and the aid's sensitivities. Combining the user and the aid often adds little to the performance the better detector could achieve alone. Also, if users assign non-optimal weights to the aid, performance may drop dramatically. Thus, the introduction of a valid aid can actually lower detection performance, compared to a more sensitive user working alone. Similarly, adding a user to a system with high sensitivity may lower its performance. System designers need to consider the potential adverse effects of introducing users or aids into systems.

[1]  Justin G. Hollands,et al.  Target Detection and Identification Performance Using an Automatic Target Detection System , 2017, Hum. Factors.

[2]  Alice F. Healy,et al.  The decision rule in probabilistic categorization: What it is and how it is learned. , 1977 .

[3]  W H Mobley,et al.  The Effects of Payoff on the Visual Processing of Dental Radiographs , 1978, Human factors.

[4]  Soumya Jana,et al.  SIGNAL DETECTION AND ESTIMATION , 2002 .

[5]  Mark R. Lehto,et al.  A Distributed Signal Detection Theory Model: Implications to the Design of Warnings , 1991, 1991 American Control Conference.

[6]  Joachim Meyer,et al.  Mental effort in binary categorization aided by binary cues. , 2013, Journal of experimental psychology. Applied.

[7]  Joachim Meyer,et al.  Use of Warnings in an Attentionally Demanding Detection Task , 2001, Hum. Factors.

[8]  Colin G. Drury,et al.  Do people choose an optimal response criterion in an inspection task , 1998 .

[9]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[10]  Neil A. Macmillan,et al.  Detection Theory , 2021 .

[11]  David Woods,et al.  The alarm problem and directed attention in dynamic fault management , 1995 .

[12]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[13]  M. Kubovy,et al.  Probability matching and the formation of conservative decision rules in a numerical analog of signal detection. , 1981 .

[14]  Michael. Athans,et al.  On optimal distributed decision architectures in a hypothesis testing environment , 1992 .

[15]  Christine M. Haslegrave,et al.  Industrial Inspection Efficiency and the Probability of a Defect Occurring , 1969 .

[16]  David D. Woods,et al.  Systems with Human Monitors: A Signal Detection Analysis , 1985, Hum. Comput. Interact..

[17]  W. Lee,et al.  CATEGORIZING EXTERNALLY DISTRIBUTED STIMULUS SAMPLES FOR THREE CONTINUA. , 1964, Journal of experimental psychology.

[18]  Justin G. Hollands,et al.  Beyond Identity: Incorporating System Reliability Information Into an Automated Combat Identification System , 2011, Hum. Factors.

[19]  Thomas R. Zentall,et al.  Factorial effects in the categorization of externally distributed stimulus samples , 1966 .

[20]  Catherine M. Burns,et al.  There Is More to Monitoring a Nuclear Power Plant than Meets the Eye , 2000, Hum. Factors.

[21]  Z. J. Ulehla,et al.  Optimality of perceptual decision criteria. , 1966, Journal of experimental psychology.

[22]  Joachim Meyer,et al.  Effects of Warning Validity and Proximity on Responses to Warnings , 2001, Hum. Factors.

[23]  J. D. Ingleby,et al.  Further studies of the human observer as statistical decision maker , 1974 .

[24]  I POLLACK,et al.  On the Performance of a Combination of Detectors , 1964, Human factors.

[25]  Graham A. Murrell,et al.  Combination of evidence in a probabilistic visual search and detection task , 1977 .

[26]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[27]  P A Hancock,et al.  Alarm effectiveness in driver-centred collision-warning systems. , 1997, Ergonomics.