Evaluating First Experiences with an Educational Computer Game: A multi-Method Approach

This paper presents our evaluation approach for a specific case study, namely the evaluation of an early prototype of an educational game with children aged between 12 and 14 years. The main goal of this initial evaluation study was to explore children’s first impressions and experiences of the game on the one hand and to assess the students’ ideas and wishes for the further development of the game on the other hand. The main challenge for the evaluation activities was the selection of the appropriate methodological approach, taking into account children as a special user group. We opted for a combination of different, mainly qualitative and explorative methods that were reported beneficial for work with children in the human-computer interaction (HCI) field. By presenting our multi-method approach, in particular the different steps and procedure within our study, other researchers can get inspirations for follow up activities when evaluating games with children as well as benefit from our experiences in exploring more collaborative methods and methodological combinations.

[1]  David R. Michael,et al.  Serious Games: Games That Educate, Train, and Inform , 2005 .

[2]  R. Seagram,et al.  Educational game models: conceptualization and evaluation , 2004 .

[3]  Arnold P. O. S. Vermeeren,et al.  Assessing usability evaluation methods on their effectiveness to elicit verbal comments from children subjects , 2003, IDC '03.

[4]  Berry Eggen,et al.  Comparing early design methods for children , 2007, IDC.

[5]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[6]  Evangelos Karapanos,et al.  User experience over time , 2008, CHI Extended Abstracts.

[7]  Paula Kotzé,et al.  Ten design lessons from the literature on child development and children's use of technology , 2009, IDC.

[8]  Manfred Tscheligi,et al.  Evaluating user experiences in games , 2008, CHI Extended Abstracts.

[9]  Lloyd P. Rieber,et al.  Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games , 1996 .

[10]  Rachel Benedyk,et al.  A comparison of usability evaluation methods for child participants in a school setting , 2007, IDC.

[11]  Greg Walsh,et al.  Wii can do it: using co-design for creating an instructional game , 2009, CHI Extended Abstracts.

[12]  Mikael B. Skov,et al.  Comparison of think-aloud and constructive interaction in usability testing with children , 2005, IDC '05.

[13]  Alessandro De Gloria,et al.  Enhancing the educational value of video games , 2009, CIE.

[14]  Perttu Hämäläinen,et al.  Using peer tutoring in evaluating the usability of a physically interactive computer game with children , 2003, Interact. Comput..

[15]  Mikael B. Skov,et al.  A review of research methods in children's technology design , 2005, IDC '05.

[16]  Manfred Tscheligi,et al.  Facial expressions as game input with different emotional feedback conditions , 2008, ACE '08.

[17]  Kirsten Risden,et al.  Evaluating computer game concepts with children , 2004, IDC '04.

[18]  Jung-Joo Lee,et al.  "It has to be a group work!": co-design with children , 2009, IDC.