Lessons Learned from Testing a Children's Educational Game through Web Deployment

Beanstalk is an educational game that teaches balance-fulcrum principles, targeting children ages 5-11. Four versions of the game were deployed on the web through the educational portal Learning.com. Two of the versions incorporate non-player characters that offer opportunities for socio-emotional learning (SEL). Two of the versions include a scientific process of "predict-observe-hypothesize-explain" (POHE), which in effect is in-game testing. This paper reports on a first look into the game logs collected from child players via the portal, allowing for a 2x2 SEL crossed with POHE analysis. Lessons learned from testing variants of an educational game are shared, as well as conclusions drawn from the logs. Surprisingly, the version with both SEL and POHE caused children to stop playing the game sooner. The POHE performance was weakest for the players in the SEL treatment. Rather than help with the science educational goals, SEL may have diminished the experience.