Assessing learning, quality and engagement in learning objects: the Learning Object Evaluation Scale for Students (LOES-S)

Learning objects are interactive web-based tools that support the learning of specific concepts by enhancing, amplifying, and/or guiding the cognitive processes of learners. Research on the impact, effectiveness, and usefulness of learning objects is limited, partially because comprehensive, theoretically based, reliable, and valid evaluation tools are scarce, particularly in the K-12 environment. The purpose of the following study was to investigate a Learning Object Evaluation Scale for Students (LOES-S) based on three key constructs gleaned from 10 years of learning object research: learning, quality or instructional design, and engagement. Tested on over 1100 middle and secondary school students, the data generated using the LOES-S showed acceptable internal reliability, face validity, construct validity, convergent validity and predictive validity.

[1]  Rory McGreal,et al.  Learning objects: A practical definition , 2004 .

[2]  Katrina Strampel,et al.  Learning Objects in Use: 'Lite' Assessment for Field Studies , 2005 .

[3]  Nicole A. Buzzetto-More,et al.  Guidelines and Standards for the Development of Fully Online Learning Objects , 2006 .

[4]  Mario A. Pei.  or Does It , 1946 .

[5]  Derek Hoy,et al.  eLearning techniques supporting problem based learning in clinical simulation , 2005, Int. J. Medical Informatics.

[6]  Allison Druin,et al.  The design of children's technology , 1998 .

[7]  T. Andre,et al.  Using computer simulations to enhance conceptual change: the roles of constructivist instruction and student epistemological beliefs , 1996 .

[8]  Roger Mundell,et al.  Experiences with reusable E-learning objects: From theory to practice , 2002, Internet High. Educ..

[9]  Lev Vygotsky.  Mind in society , 1978 .

[10]  Yavuz Akpinar,et al.  Designing interactive learning environments , 1996 .

[11]  Patrick E. Parrish,et al.  The trouble with learning objects , 2004 .

[12]  Fernando Alonso,et al.  An instructional model for web-based e-learning education with a blended learning process approach , 2005, Br. J. Educ. Technol..

[13]  Liesbeth Kester,et al.  Just-in-time, schematic supportive information presentation during cognitive skill acquisition , 2006, Comput. Hum. Behav..

[14]  Ron Oliver,et al.  Curriculum and Learning-Resources Issues Arising From the Use of Web-Based Course Support Systems , 1999 .

[15]  Lúcia Blondet Baruque,et al.  Learning Theory and Instructional Design Using Learning Object , 2022 .

[16]  Marianne Acovelli,et al.  A Coaching Agent for Learners Using Multimedia Simulations. , 1997 .

[17]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[18]  Tomi Jaakkola,et al.  Effectiveness of learning objects in various instructional settings , 2006 .

[19]  Andy Field,et al.  Discovering statistics using SPSS, 2nd ed. , 2005 .

[20]  Katina Zammit,et al.  Computer Icons: A Picture Says a Thousand Words. Or Does It? , 2000 .

[21]  Andrew R. Brown,et al.  Elements of Effective e-Learning Design , 2005 .

[22]  Gabriel López-Morteo,et al.  Computer support for learning mathematics: A learning environment based on recreational learning objects , 2007, Comput. Educ..

[23]  Terry Anderson,et al.  Interaction for lifelong learning , 2004, Br. J. Educ. Technol..

[24]  Barbara Wasson,et al.  Coordinating Collaborative Knowledge Building , 2003 .

[25]  Orit Zeichner,et al.  Using Technology to Enhance Mathematical Reasoning: Effects of Feedback and Self-Regulation Learning , 2001 .

[26]  W. Velicer,et al.  Relation of sample size to the stability of component patterns. , 1988, Psychological bulletin.

[27]  Allison Littlejohn,et al.  Issues in reusing online resources , 2003 .

[28]  David M. Kennedy,et al.  Design elements for interactive multimedia , 1997 .

[29]  Ralph Cafolla,et al.  Project MERLOT: Bringing Peer Review to Web-Based Educational Resources , 2006 .

[30]  P. Chandler,et al.  Cognitive Load Theory and the Format of Instruction , 1991 .

[31]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[32]  Erla Morales,et al.  Learning Object Evaluation , 2008 .

[33]  George Gadanidis,et al.  Factors Mediating the use of Online Applets in the Lesson Planning of Pre-service Mathematics Teachers , 2003 .

[34]  Terrie Lynn Thompson,et al.  Addressing the eLearning Contradiction: A Collaborative Approach for Developing a Conceptual Framework Learning Object , 2005 .

[35]  Robin Kay,et al.  Developing Learning Objects for Secondary School Students: A Multi-Component Model , 2005 .

[36]  Michelene T. H. Chi,et al.  Learning from Examples Via Self-Explanations , 1988, Knowing, Learning, and Instruction.

[37]  M. Albanese,et al.  Problem‐based Learning: A Review of Literature on Its Outcomes and Implementation Issues , 1993, Academic medicine : journal of the Association of American Medical Colleges.

[38]  Madhumita,et al.  Twenty-one Guidelines for Effective Instructional Design. , 1995 .

[39]  J. Bruner.  Child's Talk: Learning to Use Language , 1985 .

[40]  J. Merriënboer,et al.  Research on cognitive load theory and its design implications for e-learning , 2005 .

[41]  Marshall G. Jones,et al.  Using Metacognitive Theories to Design User Interfaces for Computer-Based Learning. , 1995 .

[42]  Marek Hatala,et al.  Edusource: Canada's Learning Object Repository Network , 2004 .

[43]  Claire Bradley,et al.  The Design, Development, and Use of Multimedia Learning Objects , 2004 .

[44]  David A. Wiley,et al.  The Instructional Use of Learning Objects: Online Version , 2000 .

[45]  Tomi Jaakkola,et al.  Promises and pitfalls of learning objects , 2006 .

[46]  Elena Maceviciute,et al.  Review of: Einspruch, Eric L. An introductory guide to SPSS® for Windows®. 2nd ed. Thousand Oaks, CA: Sage Publications, 2005 , 2006, Inf. Res..

[47]  Jude T. Lubega,et al.  The Effectiveness of Assessment Learning Objects Produced Using Pair Programming , 2004 .

[48]  Ann L. Brown,et al.  Guided, Cooperative Learning and Individual Knowledge Acquisition , 2018, Knowing, Learning, and Instruction.

[49]  Todd Michael Ohl.  An Interaction-Centric Learning Model , 2001 .

[50]  Ronald J. Chenail,et al.  When Disney meets the research park: Metaphors and models for engineering an online learning community of tomorrow , 2004, Internet High. Educ..

[51]  Tony Koppi,et al.  Learning objects, repositories, sharing and reusability , 2005 .

[52]  Mohamed Ally,et al.  A Study of the Design and Evaluation of a Learning Object and Implications for Content Development , 2005 .

[53]  Vittorio Marengo,et al.  The OSEL Taxonomy for the Classification of Learning Objects , 2006 .

[54]  Siu Cheung Kong,et al.  A cognitive tool for teaching the addition/subtraction of common fractions: a model of affordances , 2005, Comput. Educ..

[55]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[56]  Alex Koohang,et al.  Architecting Usability Properties in the E-Learning Instructional Design Process , 2004 .

[57]  M. Cole,et al.  Mind in society: The development of higher psychological processes. L. S. Vygotsky. , 1978 .

[58]  George P Max Schell,et al.  Merlot : A Repository of e-Learning Objects for Higher Education , 2002 .

[59]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[60]  John C. Nesbit,et al.  A Convergent Participation Model for Evaluation of Learning Objects , 2002 .

[61]  Thomas Brush,et al.  The Use of Embedded Scaffolds with Hypermedia-Supported Student-Centered Learning , 2001 .

[62]  Mike Sosteric,et al.  When is a Learning Object not an Object: A first step towards a theory of learning objects , 2002 .

[63]  K. Scheiter,et al.  Learner Control in Hypermedia Environments , 2007 .

[64]  K. Scherer,et al.  How Seductive Details Do Their Damage : A Theory of Cognitive Interest in Science Learning , 2004 .

[65]  J. Stevens.  Applied Multivariate Statistics for the Social Sciences , 1986 .

[66]  John C. Nesbit,et al.  Learning Object Evaluation: Computer-Mediated Collaboration And Inter-Rater Reliability , 2003 .

[67]  Patricia McGee,et al.  Transformative power of the learning object debate , 2005 .

[68]  Norm Friesen,et al.  A New Learning Object Repository for Language Learning: Methods and Possible Outcomes , 2006 .

[69]  Thomas Cochrane,et al.  Interactive QuickTime: Developing and Evaluating Multimedia Learning Objects to Enhance Both Face-To-Face and Distance E-Learning Environments , 2004 .

[70]  Mustafa Baser.  Promoting conceptual change through active learning using open source software for physics simulations , 2006 .

[71]  Min Liu,et al.  An analysis of cognitive tool use patterns in a hypermedia learning environment , 2005 .

[72]  Fred Paas,et al.  Cognitive Aging and Computer-Based Instructional Design: Where Do We Go From Here? , 2006 .

[73]  R. Mayer,et al.  How Seductive Details Do Their Damage: A Theory of Cognitive Interest in Science Learning , 1998 .

[74]  Richard F. Kenny,et al.  Towards Guidelines for the Design of Interactive Multimedia Instruction: Fostering the Reflective D , 1999 .

[75]  R. Mayer,et al.  Aids to computer-based multimedia learning , 2002 .

[76]  Andreas Holzinger,et al.  Rapid prototyping for a virtual medical campus interface , 2004, IEEE Software.

[77]  Sean W. M. Siqueira,et al.  Increasing the Semantics of Learning Objects , 2004, Int. J. Comput. Process. Orient. Lang..

[78]  David Wiley,et al.  Overcoming the Limitations of Learning Objects , 2004 .

[79]  Hai-Ning Liang,et al.  Designing online mathematical investigation , 2004 .

[80]  P. Kline.  The handbook of psychological testing, 2nd ed. , 1993 .

[81]  Hai-Ning Liang,et al.  Interactivity of Visual Mathematical Representations: Factors Affecting Learning and Cognitive Processes , 2006 .

[82]  J. Bruner.  Actual minds, possible worlds , 1985 .

[83]  Rory McGreal.  Online Education Using Learning Objects , 2004 .

[84]  Cameron Richards,et al.  Developing Interactive Learning Objects for a Computing Mathematics Module , 2006 .

[85]  Licia Calvi.  Navigation and disorientation: a case study , 1997 .

[86]  Robin Kay,et al.  Evaluating the learning in learning objects , 2007 .

[87]  Arend J. Visscher,et al.  Formative Evaluation in Educational Computing Research and Development , 1999 .

[88]  Margaret Haughey,et al.  Evaluating Learning Objects for Schools , 2005 .

[89]  David H. Jonassen,et al.  Is There a Learning Orientation in Learning Objects , 2004 .

[90]  Dick Botteldooren,et al.  Implementation and Evaluation of a Course Concept Based on Reusable Learning Objects , 2003 .

[91]  Andy P. Field,et al.  Discovering Statistics Using SPSS , 2000 .

[92]  Harald Vacik,et al.  The use of COCOON in teaching silviculture , 2006, Comput. Educ..

[93]  Aleck C. H. Lin,et al.  Designing Websites for Learning and Enjoyment: A study of museum experiences , 2006 .