Assessing the applicability of the structured expert evaluation method (SEEM) for a wider age group

This paper describes a study which examines whether a predictive evaluation method (SEEM) is suitable to assess products for an older age group than for which the evaluation method was originally developed. SEEM stands for Structured Expert Evaluation Method and is an analytical evaluation method especially developed for assessing the fun and usability of young children's educational computer games (children from 5 to 7 years old). In the present study SEEM was applied to assess educational computer games for children between 9 and 11 years old. Outcomes on scores for thoroughness (whether SEEM finds all problems), validity (whether SEEM makes predictions that are likely to be true) and appropriateness (whether SEEM is applied correctly) were compared. The results show that the trends for the thoroughness and the validity are the same for the two different age groups; however SEEM scores a bit better for the oldest age group. The appropriateness scores are about the same for the two age groups. The results indicate that SEEM can also be applied for assessing educational computer games for children between 9 and 11 years old.

[1]  Thomas W. Malone,et al.  What Makes Things Fun to Learn? A Study of Intrinsically Motivating Computer Games. , 1981 .

[2]  Wolmet Barendregt,et al.  Empirical evaluation of usability and fun in computer games for children , 2003 .

[3]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[4]  Jacqueline Brodie,et al.  Applying user testing data to UEM performance metrics , 2004, CHI EA '04.

[5]  Panos Markopoulos,et al.  A comparison of think-aloud and post-task interview for usability testing with children , 2004, IDC '04.

[6]  Kori Inkpen,et al.  Three Important Research Agendas for Educational Multimedia: Learning, Children, and Gender , 1998 .

[7]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[8]  Robert C. Williges,et al.  Criteria For Evaluating Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[9]  Wolmet Barendregt,et al.  A Structured Expert Evaluation Method for the Evaluation of Children's Computer Games , 2005, INTERACT.

[10]  Gilbert Cockton,et al.  Understanding Inspection Methods: Lessons from an Assessment of Heuristic Evaluation , 2001, BCS HCI/IHM.

[11]  Wolmet Barendregt,et al.  Predicting effectiveness of children participants in user testing based on personality characteristics , 2007, Behav. Inf. Technol..

[12]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[13]  Malcolm P. Atkinson,et al.  Comparison of evaluation methods using structured usability problem reports , 1997, Behav. Inf. Technol..

[14]  J. Piaget.  Science of education and the psychology of the child , 1970 .

[15]  Andrew Sears,et al.  Heuristic Walkthroughs: Finding the Problems Without the Noise , 1997, Int. J. Hum. Comput. Interact..