Problem roulette: Studying introductory physics in the cloud

We introduce Problem Roulette (PR), a web-based study service at the University of Michigan that offers random-within-topic access to a large library of past exam problems in introductory physics courses. Built on public-private cloud infrastructure, PR served nearly 1000 students during Fall 2012 term, delivering more than 60,000 problem pages. The service complements that of commercial publishing houses by offering problems authored by local professors and by explicitly aligning topics with exam content. We describe the service architecture, including reporting and analytical capabilities, and present an initial evaluation of the impact of its use. Among roughly 500 students studying electromagnetism, we find that the 229 students who worked fifty or more problems over the term outperformed their complement by 0.40 grade points (on a 4.0 scale). This improvement partly reflects a selection bias that academically stronger students used the service more frequently. Adjusting for this selection bias, we find a grade point improvement of 0.22, significantly above the random noise level of 0.04. The simple message to students is that working five or more additional problems per week can lead to a quarter-letter grade improvement in introductory physics. Student comments emphasize the importance of randomness in helping them to synthesize concepts. The PR source code is publicly available.

[1]  L. Medsker,et al.  New Approach to Analyzing Physics Problems: A Taxonomy of Introductory Physics Problems. , 2013 .

[2]  Rasil Warnakulasooriya,et al.  Measuring student learning with item response theory , 2008 .

[3]  Dorothea P. Simon,et al.  Expert and Novice Performance in Solving Physics Problems , 1980, Science.

[4]  Antonija Mitrovic,et al.  Evaluating the Effects of Open Student Models on Learning , 2002, AH.

[5]  Rasil Warnakulasooriya,et al.  Time to completion of web-based physics problems with tutoring. , 2007, Journal of the experimental analysis of behavior.

[6]  Jose P. Mestre,et al.  Impact of a short intervention on novices’ categorization criteria , 2012 .

[7]  Mitchell J. Nathan,et al.  Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology , 2012 .

[8]  Carl E. Wieman,et al.  Why Not Try a Scientific Approach to Science Education? , 2007 .

[9]  Robert J. Dufresne,et al.  Promoting skilled problem‐solving behavior among beginning physics students , 1993 .

[10]  Lynn S. Aaron,et al.  Teaching, Learning, and Collaborating in the Cloud: Applications of Cloud Computing for Educators in Post-Secondary Institutions , 2011 .

[11]  Jiri Lallimo,et al.  Assessing applications for collaboration: from collaboratively usable applications to collaborative technology , 2004, Br. J. Educ. Technol..

[12]  Carlos Delgado Kloos,et al.  Sending Learning Pills to Mobile Devices in Class to Enhance Student Performance and Motivation in Network Services Configuration Courses , 2012, IEEE Transactions on Education.