An examination of two mental workload measurement approaches to understanding multimedia learning

This study reports on an examination of two measures of mental workload: the NASA-TLX and Paas' Subjective Cognitive Load (SCL) measure. The goal was to assess the relative efficacy of the measures in the design and research of multimedia learning environments. Benchmarks based on the literature as to the goals for mental workload measurement in learning research are established. A multifaceted study was conducted which manipulated various aspects of mental workload in order to study the utility of these two measures in detecting changes in load and their relationship to learning outcomes. The results indicate that the weighted version of the NASA-TLX provided little additional value over the unweighted version of the measure. While both the NASA-TLX and SCL measures were sensitive to changes in both intrinsic and extraneous load, the study revealed differences in the measures based on levels of each of these load factors. The study also concludes that a better understanding of the third factor, germane load, will be needed to both expand the theoretical framework about mental workload in instructional settings and further understand the utility of these two measures.

[1]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[2]  R. Catrambone,et al.  Can learning from molar and modular worked examples be enhanced by providing instructional explanations and prompting self-explanations? * , 2006 .

[3]  F. Paas.  Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. , 1992 .

[4]  F. Paas,et al.  Variability of Worked Examples and Transfer of Geometrical Problem-Solving Skills: A Cognitive-Load Approach , 1994 .

[5]  Richard Catrambone,et al.  Making the abstract concrete: Visualizing mathematical solution procedures , 2006, Comput. Hum. Behav..

[6]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[7]  F. Paas,et al.  Instructional control of cognitive load in the training of complex cognitive tasks , 1994 .

[8]  R. Mayer,et al.  A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory , 1998 .

[9]  Daniel P. J. Bruneau,et al.  A self-analysis of the NASA-TLX workload measure , 2007, Ergonomics.

[10]  Katharina Scheiter,et al.  Explaining the split-attention effect: Is the reduction of extraneous cognitive load accompanied by an increase in germane cognitive load? , 2009, Comput. Hum. Behav..

[11]  James M. LeBreton,et al.  History and Use of Relative Importance Indices in Organizational Research , 2004 .

[12]  Richard E. Mayer,et al.  Elements of a Science of E-Learning , 2003 .

[13]  P. Hancock,et al.  Human Mental Workload , 1988 .

[14]  P. Tsang,et al.  Diagnosticity and multidimensional subjective workload ratings. , 1996, Ergonomics.

[15]  H. Tabbers,et al.  Multimedia instructions and cognitive load theory: effects of modality and cueing. , 2004, The British journal of educational psychology.

[16]  P. Chandler,et al.  The Role of Visual Indicators in Dual Sensory Mode Instruction , 1997 .

[17]  Robert R. Whelan.  The multimedia mind: Measuring cognitive load in multimedia learning , 2006 .

[18]  Slava Kalyuga,et al.  Managing split-attention and redundancy in multimedia instruction , 1999 .

[19]  Richard Catrambone,et al.  Designing Instructional Examples to Reduce Intrinsic Cognitive Load: Molar versus Modular Presentation of Solution Procedures , 2004 .

[20]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[21]  Thomas E. Nygren,et al.  Psychometric Properties of Subjective Workload Measurement Techniques: Implications for Their Use in the Assessment of Perceived Mental Workload , 1991 .

[22]  F. Paas,et al.  Measurement of Cognitive Load in Instructional Research , 1994, Perceptual and motor skills.

[23]  Jefferson M. Koonce,et al.  Comprehension and workload differences for VDT and paper-based reading , 2001 .

[24]  Per Øivind Braarud,et al.  Subjective Task Complexity and Subjective Workload: Criterion Validity for Complex Team Tasks , 2001 .

[25]  Robert E. Ployhart,et al.  A multidimensional approach for evaluating variables in organizational research and practice , 2007 .

[26]  Sharon K Tindall-Ford,et al.  When two sensory modes are better than one , 1997 .

[27]  D. Damos.  Multiple-task performance , 2020 .

[28]  Eric Jamet,et al.  Attention guiding in multimedia learning , 2008 .

[29]  Susana Rubio,et al.  Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA‐TLX, and Workload Profile Methods , 2004 .

[30]  Susan G. Hill,et al.  Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? In A , 1989 .

[31]  Diana G. Oblinger,et al.  Fostering Learning in the Networked World: The Cyberlearning Opportunity and Challenge. A 21st Century Agenda for the National Science Foundation , 2008 .

[32]  Shiow-Yun Chang,et al.  Discriminating relative workload level by data envelopment analysis , 2006 .

[33]  William F. Moroney,et al.  A comparison of two scoring procedures with the NASA task load index in a simulated flight task , 1992, Proceedings of the IEEE 1992 National Aerospace and Electronics Conference@m_NAECON 1992.

[34]  Tamara van Gog,et al.  State of the art research into Cognitive Load Theory , 2009, Comput. Hum. Behav..

[35]  David B. Boles,et al.  Predicting Dual-Task Performance With the Multiple Resources Questionnaire (MRQ) , 2007, Hum. Factors.

[36]  W. Schnotz,et al.  A Reconsideration of Cognitive Load Theory , 2007 .