Validation of the Instructional Materials Motivation Survey (IMMS) in a self-directed instructional setting aimed at working with technology

The ARCS Model of Motivational Design has been used myriad times to design motivational instructions that focus on attention, relevance, confidence and satisfaction in order to motivate students. The Instructional Materials Motivation Survey (IMMS) is a 36-item situational measure of people's reactions to instructional materials in the light of the ARCS model. Although the IMMS has been used often, both as a pretest and a posttest tool serving as either a motivational needs assessment prior to instruction or a measure of people's reactions to instructional materials afterward, the IMMS so far has not been validated extensively, taking statistical and theoretical aspects of the survey into account. This paper describes such an extensive validation study, for which the IMMS was used in a self-directed instructional setting aimed at working with technology (a cellular telephone). Results of structural equation modeling show that the IMMS can be reduced to 12 items. This Reduced Instructional Materials Motivation Survey IMMS (RIMMS) is preferred over the original IMMS. The parsimonious RIMMS measures the four constructs attention, relevance, confidence and satisfaction of the ARCS model well, and reflects its conditional nature.

[1]  James C. Anderson,et al.  STRUCTURAL EQUATION MODELING IN PRACTICE: A REVIEW AND RECOMMENDED TWO-STEP APPROACH , 1988 .

[2]  D. Jonassen,et al.  Enhancing relevance: Embedded ARCS strategies vs. Purpose , 1997 .

[3]  Joyce Karreman,et al.  Adding Motivational Elements to an Instruction Manual for Seniors: Effects on Usability and Motivation , 2007 .

[4]  Joyce Karreman,et al.  Verification Steps and Personal Stories in an Instruction Manual for Seniors: Effects on Confidence, Motivation, and Usability , 2013, IEEE Transactions on Professional Communication.

[5]  Bruce W. Tuckman.  Thinking Out Loud: Do We Dare Not Be Politically Correct? , 1994 .

[6]  Toni Bellon,et al.  Best Practices in Cyberspace: Motivating the Online Learner. , 2002 .

[7]  James C. Anderson,et al.  On the Meaning of Within-Factor Correlated Measurement Errors , 1984 .

[8]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[9]  James Daniel Lehman,et al.  Learning Foreign Language through an Interactive Multimedia Program: An Experimental Study on the Effects of the Relevance Component of the ARCS Model , 2013 .

[10]  Ruth V. Small,et al.  The Relationship of Motivational Conditions to Effective Instructional Attributes: A Magnitude Scaling Approach. , 1994 .

[11]  Scott D. Johnson,et al.  The Effect of Context-Based Video Instruction on Learning and Motivation in Online Courses , 2005 .

[12]  Hermann Astleitner,et al.  The Effects of Situation-Outcome-Expectancies and of ARCS-Strategies on Self-Regulated Learning with Web-Lectures , 2003 .

[13]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[14]  Heidi A. Diefes-Dux,et al.  A preliminary validation of Attention, Relevance, Confidence and Satisfaction model-based Instructional Material Motivational Survey in a computer-based tutorial setting , 2006, Br. J. Educ. Technol..

[15]  J. Keller.  Motivational Design for Learning and Performance , 2010 .

[16]  Timothy R. Hinkin,et al.  A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires , 1998 .

[17]  J. Keller.  The Arcs Model of Motivational Design , 2010 .

[18]  J. Keller.  Motivational Design of Instruction , 1983 .

[19]  J. Tukey,et al.  Multiple-Factor Analysis , 1947 .

[20]  J. Keller.  Using the ARCS Motivational Process in Computer-Based Instruction and Distance Education , 1999 .

[21]  K. Bollen,et al.  An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models , 2008, Sociological methods & research.

[22]  Larry W. Thompson,et al.  The Older Adult , 1992 .

[23]  Bonnie Shellnut,et al.  Applying the ARCS model to the design and development of computer-based modules for manufacturing engineering courses , 1999 .

[24]  B. Thompson,et al.  EFFECTS OF SAMPLE SIZE, ESTIMATION METHODS, AND MODEL SPECIFICATION ON STRUCTURAL EQUATION MODELING FIT INDEXES , 1999 .

[25]  J. Keller.  Motivational Design for Learning and Performance: The ARCS Model Approach , 2009 .

[26]  Samuel Messick,et al.  STANDARDS OF VALIDITY AND THE VALIDITY OF STANDARDS IN PERFORMANCE ASSESSMENT , 2005 .

[27]  J. Keller.  Development and use of the ARCS model of instructional design , 1987 .

[28]  R. L. Holbert,et al.  Structural Equation Modeling in the Communication Sciences, 1995–2000 , 2002 .

[29]  John M. Keller,et al.  Effectiveness of motivationally adaptive computer-assisted instruction on the dynamic aspects of motivation , 2001 .

[30]  Joyce Karreman,et al.  Confidence-increasing elements in user instructions: Seniors' reactions to verification steps and personal stories , 2013 .

[31]  C. Stein,et al.  Structural equation modeling. , 2012, Methods in molecular biology.

[32]  Donald J. Winiecki,et al.  Evaluation of Effective Interventions to Solve the Drop out Problem in Adult Distance Education , 1999 .

[33]  John M. Keller,et al.  The systematic process of motivational design , 1987 .

[34]  A. Panter,et al.  Writing about structural equation models. , 1995 .

[35]  C Loehlin John,et al.  Latent variable models: an introduction to factor, path, and structural analysis , 1986 .

[36]  J. Keller,et al.  An application of the ARCS Model of Motivational Design. , 1987 .

[37]  J. Keller.  Strategies for stimulating the motivation to learn , 1987 .