Cognitive Workload Induced by Information Systems: Introducing an Objective Way of Measuring based on Pupillary Diameter Responses

We give an overview of experimental results having applied a novel method to derive a users’ cognitive workload (CW) intensity based on the CW – pupillary diameter (PD) relationship (Beatty, 1982) using eye-tracking technology. In our first experiment (n=12) we adopted two simple tasks, well-documented in psychology. First, according to (Beatty, 1982), our participants have to memorize and reproduce numbers from three to nine digits. The digit span can be split into a low cognitive demand level (three to five digits) and a medium demand level (six to nine digits). Second, those tested have to solve four arithmetic multiplication problems representing a high cognitive demand level as documented in (Hess & Polt, 1964). As a result, we found coherently significant (p<0.05) differences in PDs between both the low and the medium demand level as well as between the medium and the high demand level. The psychophysiological response was considerable as the effect sizes of the coherently significant pupillary dilations were medium (0.44 ≤ d ≤ 0.54). In addition we observed within the first stage (three to nine digit memorizing/reproducing tasks) tonic pupil dilations of about 0.3 mm pupillary radius between relaxation (no task/pause: 1.3 mm) and cognitive workload (memorizing/reproducing: 1.6 mm). These observations confirm the results of (Beatty, 1982) who also coherently found the 0.3 mm pupillary radius value between relaxation and cognitive workload. During the second experimental stage (arithmetic multiplication task) we observed tonic pupil dilations of about 0.4 mm pupillary radius between relaxation (no task/pause: 1.3 mm) and cognitive workload (multiplication task: 1.7 mm). Thus, we could confirm the results of (Hess & Polt, 1964) who also found a taskevoked pupil dilation of up to 29.5 percent within this specific multiplication task. In order to evaluate the method on a complex widely-used information system with a lot of dynamic web elements, interaction functions and advertising banners, we chose LinkedIn.com as a common social network application and performed a laboratory experiment with a larger sample size (n=125). The participants of our experiment were asked to solve three information search tasks with a varied level of difficulty (varying number of website changes, filter and search mechanisms that have to be applied) inducing different workload levels of the users. These three tasks are to be fulfilled in the same test order (Eckhardt, Maier, & Buettner, 2012): 1. “Ask your contact [given first name, surname] for a letter of recommendation.” (eight steps between login and task fulfillment  high demand level); 2. “Apply to Oracle for a new job of your choice.” (four steps between login and task fulfillment  low demand level); 3. “Identify and name the Global Head of Recruiting at BMW.” (six steps between login and task fulfillment  mid demand level). The results showed that the level of difficulty matched the assumed direction since 81/42/28 of the 125 participants successfully solved the low/mid/high demand level tasks. The PD deviation differences between the low and the medium cognitive demand level as well as the medium and the high cognitive demand level were all significant (p<0.01). The effect sizes of the PD deviation differences were small to medium (0.11 ≤ d ≤ 0.31). In conclusion, we reliably detected the different cognitive workload levels within all the stages of our experiments. Future work should improve the method concerning the individual differences of the CW – PD relationship. Since PDs are age-dependent (Birren, Casperson, & Botwinick, 1950), the potential age-dependency of the CW – PD relationship should be investigated.