Do we know how difficult the rainfall problem is?

The programming task known as the Rainfall Problem has developed a reputation for being surprisingly difficult for introductory-level (CS1) students. We contribute a survey of studies of the problem as well as a new study of students' solutions collected at three institutions. In all three CS1s, at least about half of the students were able to fully solve the problem and the large majority were at least close. Failure to handle invalid or missing input accounted for most bugs. Our survey and study together suggest that the Rainfall Problem is not necessarily overwhelmingly difficult: Success rates vary and some reasonably good results have been achieved under multiple programming paradigms. We provide a breakdown of confounding factors and suggest improvements and hypotheses for future studies of the Rainfall Problem.

[1]  Ulf Olsson,et al.  A Measure of Agreement for Interval or Nominal Multivariate Observations , 2001 .

[2]  Françoise Détienne,et al.  Expert Programming Knowledge: a Schema-Based Approach , 2007, ArXiv.

[3]  Cynthia Taylor,et al.  Computer science concept inventories: past and future , 2014, Comput. Sci. Educ..

[4]  Allison Elliott Tew,et al.  A fresh look at novice programmers' performance and their teachers' expectations , 2013, ITiCSE -WGR '13.

[5]  Anna Eckerdal,et al.  On the Bimodality in an Introductory Programming Course: An Analysis of Student Performance Factors , 2015, 2015 International Conference on Learning and Teaching in Computing and Engineering.

[6]  Elliot Soloway,et al.  Learning to program = learning to construct mechanisms and explanations , 1986, CACM.

[7]  Raymond Lister,et al.  Predicting student success using fine grain clicker data , 2014, ICER '14.

[8]  J. Reitman,et al.  Knowledge organization and skill differences in computer programmers , 1981, Cognitive Psychology.

[9]  Simon P. Davies,et al.  The Role of Notation and Knowledge Representation in the Determination of Programming Strategy: A Framework for Integrating Models of Programming Behavior , 1991, Cogn. Sci..

[10]  Robert S. Rist.  Schema Creation in Programming , 1989, Cogn. Sci..

[11]  Elliot Soloway,et al.  A Goal/Plan Analysis of Buggy Pascal Programs , 1985, Hum. Comput. Interact..

[12]  Kathi Fisler,et al.  The recurring rainfall problem , 2014, ICER '14.

[13]  Alireza Ebrahimi,et al.  Novice programmer errors: language constructs and plan composition , 1994, Int. J. Hum. Comput. Stud..

[14]  Matthias Felleisen,et al.  The structure and interpretation of the computer science curriculum , 2004, J. Funct. Program..

[15]  Simon,et al.  Soloway's Rainfall Problem Has Become Harder , 2013, 2013 Learning and Teaching in Computing and Engineering.

[16]  Michael de Raadt.  Teaching Programming Strategies Explicitly to Novice Programmers: Can the way we teach strategies improve novice outcomes? , 2009 .

[17]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[18]  Mark Guzdial,et al.  From science to engineering , 2011, Commun. ACM.

[19]  P. Pirolli.  Effects of Examples and Their Explanations in a Lesson n Recursion: A Production System Analysis , 1991 .

[20]  Anne Venables,et al.  A closer look at tracing, explaining and code writing skills in the novice programmer , 2009, ICER '09.

[21]  Elliot Soloway,et al.  Marcel: Simulating the Novice Programmer , 1992 .

[22]  Mark Guzdial,et al.  Exploring hypotheses about media computation , 2013, ICER.

[23]  Kate Ehrlich,et al.  Knowledge and processes in the comprehension of computer programs. , 1988 .

[24]  Slava Kalyuga.  Cognitive Load Theory: Schema Acquisition and Sources of Cognitive Load , 2010 .

[25]  Kate Ehrlich,et al.  Cognitive strategies and looping constructs: an empirical study , 1983, CACM.

[26]  Elliot Soloway,et al.  PROUST: An automatic debugger for Pascal programs , 1985 .