Brunswikian Theory and Method as a Foundation for Simulation-Based Research on Clinical Judgment

Alex Kirlik, PhD An article on clinicians’ abilities to accurately and reliably make judgments of Apgar scores by Nadler et al1 (hereafter, “the authors”) published in a recent issue of this journal presented the first use of Egon Brunswik’s2– 4 (also see Refs. 5 and 6) theory of probabilistic functioning and methodology of representative design in Simulation in Healthcare. The authors noted that while the Apgar score has become an internationally recognized, standard method for rating judgments of the physiological status of newborns, little research had been done to date to determine how accurately these clinical judgments are made. Perhaps more importantly, the authors noted that although video recordings of neonatal resuscitations in actual clinical settings have been used for auditing and training, a recent study7 found that “clinicians cannot assign Apgar scores to video recordings of actual neonatal resuscitations with acceptable levels of interobserver reliability, and that observers’ scores had little agreement with the original scores assigned by the clinicians who performed the resuscitation.”1 In an attempt to clarify this state of affairs, the authors posed the question of whether the poor performance of the clinicians in the study just mentioned7 might, in large part, be an artifact of their restricted perceptual access to, or information about, the physiological states of the newborns whose resuscitations were video recorded. As such, the authors generated their own resuscitation videos that they believed to make more perceptually available the actual clinical signs (judgment cues) needed to make accurate Apgar judgments in a clinical context. To do so, they used a representatively designed sample of 51 resuscitation scenarios and a newborn mannequin-simulator to make their own video recordings. In describing the motivation for their research method, the authors wrote “According to Brunswik, experiments—including experiments in laboratories—should have a representative design where participants are exposed to situations that represent the range and distribution of situations and cues (clinical signs) in their natural environment.”1 Using this methodology, the authors found that the clinicians participating in their study made Apgar rating judgments that were highly (0.78 – 0.91) correlated with true Apgar scores programmed into their patient simulator (ie, ground truth). The authors concluded that the relatively high level of clinical judgment performance observed in their study, as compared with the prior study,7 owed largely to a relatively higher degree of representativeness2– 4 in their experimental design. Finally, the authors also concluded that the high level of correlation (0.79 – 0.97) found between clinician’s ratings and individualized linear models created by regressing these ratings against the perceptual cues or clinical signs available from the videotapes indicated that clinicians demonstrated “systematic” judgment. That is, the authors found a high level of consistency in how clinicians executed their cognitive strategies for making these judgments.8 This finding was taken to lend credence to the use of Brunswik’s theory of probabilistic functionalism to provide a useful way to analyze and model clinical judgment in representatively designed experimental conditions.