Skill Progression Demonstrated by Users in the Scratch Animation Environment

The Scratch environment exemplifies a tool+community approach to teaching elementary programming skills, as it includes a website where users can publish, discuss, and organize animations that are programs. To explore this environment's effectiveness for helping people to develop programming skills, a quantitative analysis of 250 randomly selected users' data, including more than 1,000 of their animations, was performed. Skill based on 4 models that had proven useful in prior empirical studies was measured. Overall, mixed results about the environment's effectiveness were found. Among users who do not drop out, an increasing progression in social skills was found. However, an extremely high drop-out rate was also observed. Moreover, a flat or decreasing level of demonstrated skill was observed on virtually every measure. These results call into question whether simply combining an animation tool and an online community is sufficient for keeping people engaged long enough to learn elementary programming skills.

[1]  Ellis Horowitz,et al.  Software Cost Estimation with COCOMO II , 2000 .

[2]  Malcolm Munro,et al.  Modelling and measuring end user sophistication , 1992, SIGCPR '92.

[3]  Capers Jones.  Backfiring: Converting Lines of Code to Function Points , 1995, Computer.

[4]  Stephen Cooper.  Developing Algorithmic Thinking with Alice , 2000 .

[5]  Stephen Cooper,et al.  Evaluating the effectiveness of a new instructional approach , 2004, SIGCSE.

[6]  David Garlan,et al.  A programming system for children that is designed for usability , 2002 .

[7]  Eric Rosenbaum,et al.  Scratch: programming for all , 2009, Commun. ACM.

[8]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[9]  Larry Tesler,et al.  Novice Programming Comes of Age , 2001, Your Wish is My Command.

[10]  Minou Ella Mebane,et al.  Evaluation of the Efficacy of Affective Education Online Training in Promoting Academic and Professional Learning and Social Capital , 2008, Int. J. Hum. Comput. Interact..

[11]  Gerry Stahl,et al.  Internet repositories for collaborative learning: supporting both students and teachers , 1995, CSCL.

[12]  Volker Wulf,et al.  Building Communities among Software Engineers: The ViSEK Approach to Intra- and Inter-Organizational Learning , 2002, LSO.

[13]  Robert D. Macredie,et al.  Cognitive Modeling of Student Learning in Web-Based Instructional Programs , 2004, Int. J. Hum. Comput. Interact..

[14]  M. Resnick.  Sowing the Seeds for a more Creative Society , 2009, CHI 2009.

[15]  Michael J. Hannafin,et al.  Scaffolding in technology-enhanced learning environments , 2007, Interact. Learn. Environ..

[16]  Joseph Krajcik,et al.  The design of guided learner-adaptable scaffolding in interactive learning environments , 1996, CHI.

[17]  Mary Shaw,et al.  Dimensions Characterizing Programming Feature Usage by Information Workers , 2006, Visual Languages and Human-Centric Computing (VL/HCC'06).

[18]  Ramsey F. Hamade,et al.  Learning Theory as Applied to Mechanical CAD Training of Novices , 2006, Int. J. Hum. Comput. Interact..

[19]  Minjuan Wang,et al.  Promoting Online Collaborative Learning Experiences for Teenagers , 2001 .

[20]  Mitchel Resnick,et al.  Empowering kids to create and share programmable media , 2008, Interactions.

[21]  Barry W. Boehm,et al.  Understanding and Controlling Software Costs , 1988, IEEE Trans. Software Eng..

[22]  Alexander Repenning,et al.  Agentsheets: a tool for building domain-oriented dynamic, visual environments , 1993 .

[23]  Mary Shaw,et al.  An approach for categorizing end user programmers to guide software engineering research , 2005, ACM SIGSOFT Softw. Eng. Notes.

[24]  Kouichi Kishida,et al.  Toward an understanding of the motivation of open source software developers , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[25]  Alexander Repenning,et al.  Towards the Automatic Recognition of Computational Thinking for Adaptive Visual Language Learning , 2010, 2010 IEEE Symposium on Visual Languages and Human-Centric Computing.

[26]  Ned Gulley,et al.  In praise of tweaking: a wiki-like programming contest , 2004, INTR.

[27]  Tom DeMarco,et al.  Peopleware: Productive Projects and Teams , 1987 .

[28]  Malcolm Munro,et al.  Understanding and measuring user competence , 1997, Inf. Manag..

[29]  Kouichi Kishida,et al.  Evolution patterns of open-source software systems and communities , 2002, IWPSE '02.

[30]  Mitchel Resnick,et al.  Programming by choice: urban youth learning programming with scratch , 2008, SIGCSE '08.

[31]  Julia Eichmann,et al.  Making Software - What Really Works, and Why We Believe It , 2011, Making Software.

[32]  Alexander Repenning.  Agentsheets: a tool for building domain-oriented visual programming environments , 1993, CHI '93.

[33]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[34]  Yan Zhang,et al.  A qualitative study of animation programming in the wild , 2010, ESEM '10.