Differences in the Educational Software Evaluation Process for Experts and Novice Students

This comparative case study investigated the educational software evaluation processes of both experts and novices in conjunction with a software evaluation checklist. Twenty novice elementary education students, divided into groups of five, and three experts participated. Each novice group and the three experts evaluated educational software selected by the novice groups in accordance with the checklist. Data were collected through focus group interviews, classroom observations and document analysis. Evaluation processes were analysed through thematic comparisons. The results showed that the expert-novice agreement rate was as low as 48%, with novice students tending to grade the software higher. While the experts used a systematic approach, including understanding and assessing each criterion, supporting the process with literature and evaluating the software as a whole, novice students lacked such methods, indicating a need for additional training and development.

[1]  Hatice Sancar Tokmak,et al.  ICT Pre-service Teachers' Opinions about the Contribution of Initial Teacher Training to Teaching Practice , 2011 .

[2]  R. Sternberg.  Advances in the psychology of human intelligence , 1982 .

[3]  Herbert J. Walberg,et al.  The International encyclopedia of educational evaluation , 1990 .

[4]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[5]  Peter Baumgartner,et al.  Abstract (1996): Learning as action: A social science approach to the evaluation of interactive media , 1996 .

[6]  Joseph A. Maxwell,et al.  Qualitative Research Design: An Interactive Approach , 1996 .

[7]  K. Knafl,et al.  Within-Case and Across-Case Approaches to Qualitative Data Analysis , 2003, Qualitative health research.

[8]  Robert Heinich,et al.  Instructional Media and Technologies for Learning , 1996 .

[9]  Lieven Verschaffel,et al.  Computers and Learning , 1994 .

[10]  Michael W Crossley,et al.  Case‐Study Research Methods and Comparative Education , 1984 .

[11]  Hilda Borko,et al.  High school mathematics review lessons: Expert-novice distinctions. , 1990 .

[12]  Peter J. Fadde.  Expertise-Based Training: Getting More Learners Over the Bar in Less Time , 2009 .

[13]  Ruth de Villiers,et al.  Usability evaluation of an e-learning tutorial: criteria, questions and case study , 2004 .

[14]  Dale S. Niederhauser,et al.  Teachers’ instructional perspectives and use of educational software , 2001 .

[15]  T. Husén,et al.  The International Encyclopedia of Education , 1994 .

[16]  R. Kozma,et al.  Multimedia and understanding: Expert and novice responses to different representations of chemical phenomena , 1997 .

[17]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[18]  H. Swanson,et al.  An Information Processing Analysis of Expert and Novice Teachers’ Problem Solving , 1990 .

[19]  Marco Antonio Chávez Arcega.  Instructional technology and media for learning , 2010 .

[20]  Edward A. Silver,et al.  The Role of Visual Representations in Advanced Mathematical Problem Solving: An Examination of Expert-Novice Similarities and Differences , 2004 .

[21]  Helen Meyer.  Novice and expert teachers' conceptions of learners' prior knowledge , 2004 .

[22]  Robert A. Reiser,et al.  Evaluating instructional software: A review and critique of current methods , 1994 .

[23]  Maria Virvou,et al.  Combining Software Games with Education: Evaluation of its Educational Effectiveness , 2005, J. Educ. Technol. Soc..

[24]  Jack R. Fraenkel,et al.  Educational Research: A Guide to the Process , 1974 .

[25]  Terri L. Kurz,et al.  A Taxonomy of Software for Mathematics Instruction. , 2005 .

[26]  Jennifer Preece,et al.  Usability and learning: Evaluating the potential of educational software , 1996, Comput. Educ..

[27]  Michelene T. H. Chi,et al.  Expertise in Problem Solving. , 1981 .