Personalized messages in a brewery educational simulation: Is the personalization principle less robust than previously thought?

The personalization principle, one of the design principles of multimedia learning, states that people learn better from multimedia presentations when instructions are in a conversational style rather than a formal style, possibly due to learners' increased interest. This principle was shown to be robust in short interventions that could be completed within minutes or a few dozen minutes; however, complex digital simulations and games that support the acquisition of complex mental models usually take longer to complete. In this study, we investigate the personalization principle in a new context: in an interactive simulation on the topic of beer brewing, which lasts 2-3 h. Instructions were presented in the Czech language, either in a personalized style, where learners were addressed conversationally by ''their grandpa, an owner of the family brewery,'' or in a non-personalized, more formal style without the grandpa. In Experiment 1, 26 college students, who interacted with both simulation versions, expressed on average a preference for the personalized version of the simulation. However, some of them worried that personalization could distract them. In Experiment 2 with a between-subject design, the knowledge of 75 predominantly college students was tested by means of retention and transfer tests immediately after completing the simulation and also a month later. Contrary to most previous works, our results showed no difference between the personalized and non-personalized groups in learning achievement, despite the fact that learners who received the personalized treatment voluntarily spent about 20% more time on the simulation. We also applied various measures of the learner's affective state, including Flow Short Scale and PANAS, but - again - no between-group differences were observed. These results indicate that personalization is not always beneficial to learning, which raises important questions for future research. Additional findings suggest that the simulation, no matter the treatment type, was most beneficial to learners with high mathematical abilities and who play computer games frequently, and also to those who liked the simulation more.

[1]  J. D. Fletcher,et al.  Review of research on computer games. , 2011 .

[2]  A. R. Ilersic,et al.  Research methods in social relations , 1961 .

[3]  Delia M. Dowling.  The development of a mathematics confidence scale and its application in the study of confidence in women college students , 1978 .

[4]  Sharon K Tindall-Ford,et al.  Learning by imagining. , 2001, Journal of experimental psychology. Applied.

[5]  John Sweller,et al.  Cognitive load and the imagination effect , 2004 .

[6]  Richard E. Mayer,et al.  A Personalization Effect in Multimedia Learning: Students Learn Better When Words Are in Conversational Style Rather Than Formal Style. , 2004 .

[7]  David A. Pierfy.  Comparative Simulation Game Research , 1977 .

[8]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[9]  R. Mayer,et al.  Personalized messages that promote science learning in virtual environments , 2004 .

[10]  Ning Wang,et al.  The politeness effect: Pedagogical agents and learning outcomes , 2008, Int. J. Hum. Comput. Stud..

[11]  Cyril Brom,et al.  Are educational computer micro-games engaging and effective for knowledge acquisition at high-schools? A quasi-experimental study , 2011, Comput. Educ..

[12]  Larry E. Toothaker,et al.  Multiple Regression: Testing and Interpreting Interactions , 1991 .

[13]  C. Douglas Wetzel,et al.  The Effectiveness of Games for Educational Purposes: A Review of Recent Research , 1992 .

[14]  Krista E. DeLeeuw,et al.  A Comparison of Three Measures of Cognitive Load: Evidence for Separable Measures of Intrinsic, Extraneous, and Germane Load , 2008 .

[15]  Cyril Brom,et al.  Implementing digital game-based learning in schools: augmented learning environment of ‘Europe 2045’ , 2010, Multimedia Systems.

[16]  Seymour Papert,et al.  Mindstorms: Children, Computers, and Powerful Ideas , 1981 .

[17]  Alfred Bork,et al.  Multimedia in Learning , 2001 .

[18]  S. Engeser,et al.  Flow, performance and moderators of challenge-skill balance , 2008 .

[19]  Richard E. Mayer,et al.  Multimedia learning and games. , 2011 .

[20]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[21]  P. Wouters,et al.  A meta-analysis of the cognitive and motivational effects of serious games , 2013 .

[22]  Kenneth R. Koedinger,et al.  Studying the Effects of Personalized Language and Worked Examples in the Context of a Web-Based Intelligent Tutor , 2006, Intelligent Tutoring Systems.

[23]  Jennifer J. Richler,et al.  Effect size estimates: current use, calculations, and interpretation. , 2012, Journal of experimental psychology. General.

[24]  Richard E. Mayer,et al.  Polite web-based intelligent tutors: Can they improve learning in classrooms? , 2011, Comput. Educ..

[25]  James C. Lester,et al.  Lifelike Pedagogical Agents for Mixed-initiative Problem Solving in Constructivist Learning Environments , 2004, User Modeling and User-Adapted Interaction.

[26]  Michelle L. Peters,et al.  EXAMINING THE RELATIONSHIPS AMONG CLASSROOM CLIMATE, SELF-EFFICACY, AND ACHIEVEMENT IN UNDERGRADUATE MATHEMATICS: A MULTI-LEVEL ANALYSIS , 2013 .

[27]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[28]  R. Mayer,et al.  Interactive Multimodal Learning Environments Special Issue on Interactive Learning Environments: Contemporary Issues and Trends , 2007 .

[29]  Richard E. Mayer,et al.  Constructing computer-based tutors that are socially sensitive: Politeness in educational software , 2006, Int. J. Hum. Comput. Stud..

[30]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[31]  P. McCullagh,et al.  Some aspects of analysis of covariance. , 1982, Biometrics.

[32]  Ralf D. Brown,et al.  THE PRONOUNS OF POWER AND SOLIDARITY , 1968 .

[33]  Welch Bl.  THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[34]  N. Betz,et al.  An Exploration of the Mathematics Self-Efficacy/Mathematics Performance Correspondence , 1989 .

[35]  Phillip L. Ackerman,et al.  Cognitive and Noncognitive Determinants and Consequences of Complex Skill Acquisition , 2004 .

[36]  Eliot R. Smith,et al.  Research methods in social relations , 1962 .

[37]  G. Kartal.  Does language matter in multimedia learning? Personalization principle revisited. , 2010 .

[38]  Richard E. Mayer,et al.  A politeness effect in learning with web-based intelligent tutors , 2011, Int. J. Hum. Comput. Stud..

[39]  Peter E. Doolittle.  The Effects of Segmentation and Personalization on Superficial and Comprehensive Strategy Instruction in Multimedia Learning Environments , 2010 .

[40]  Nancy E. Betz,et al.  The relationship of mathematics self-efficacy expectations to the selection of science-based college majors , 1983 .

[41]  P. Johnson-Laird.  Mental models , 1989 .

[42]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[43]  R. Mayer,et al.  Engaging students in active learning: The case for personalized multimedia messages. , 2000 .

[44]  Regina Vollmeyer,et al.  Die Erfassung des Flow-Erlebens , 2003 .

[45]  Michael J. Padilla,et al.  The Construction and Validation of the Test of Graphing in Science (TOGS). , 1986 .

[46]  Margaret G. McKeown,et al.  Questioning the Author: A Yearlong Classroom Implementation to Engage Students with Text , 1996, The Elementary School Journal.