A Survey of Successful Evaluations of Program Visualization and Algorithm Animation Systems

This article reviews successful educational experiences in using program and algorithm visualizations (PAVs). First, we survey a total of 18 PAV systems that were subject to 33 evaluations. We found that half of the systems have only been tested for usability, and those were shallow inspections. The rest were evaluated with respect to their educational effectiveness. Script-based systems seem to be well suited for the viewing, responding, and changing engagement levels, while compiler-based systems do well for the construction and presenting engagement levels. Finally, we analyze additional PAV features of successful evaluations and hypothesize that they are relevant.

[1]  John T. Stasko,et al.  Do algorithm animations assist learning?: an empirical study and analysis , 1993, INTERCHI.

[2]  Andreas Kerren,et al.  Human-Centered Visualization Environments , 2008 .

[3]  Pierluigi Crescenzi,et al.  Fully integrating algorithm visualization into a cs2 course.: a two-year experience , 2007, ITiCSE.

[4]  Lauri Malmi,et al.  Matrix A Framework for Interactive Software Visualization , 2004 .

[5]  Christopher D. Hundhausen,et al.  Integrating algorithm visualization technology into an undergraduate algorithms course: ethnographic studies of a social constructivist approach , 2002, Comput. Educ..

[6]  Thomas L. Naps,et al.  A testbed for pedagogical requirements in algorithm visualizations , 2002, ITiCSE '02.

[7]  T. Dean Hendrix,et al.  Experimental evaluation of animated-verifying object viewers for Java , 2006, SoftVis '06.

[8]  Jaime Urquiza Fuentes.  Generación Semiautomática de Animaciones de Programas Funcionales con Fines Educativos , 2007 .

[9]  John T. Stasko,et al.  Evaluating animations as student aids in learning computer algorithms , 1999, Comput. Educ..

[10]  Christopher D. Hundhausen,et al.  Low-Fidelity Algorithm Visualization , 2002, J. Vis. Lang. Comput..

[11]  Mordechai Ben-Ari,et al.  The Jeliot 2000 program animation system , 2003, Comput. Educ..

[12]  Amruth N. Kumar.  Results from the evaluation of the effectiveness of an online tutor on expression evaluation , 2005, SIGCSE '05.

[13]  Barbara M. Moskal,et al.  Evaluating the effectiveness of a new instructional approach , 2004 .

[14]  Lauri Malmi,et al.  Exploring the role of visualization and engagement in computer science education , 2003, ACM SIGCSE Bull..

[15]  T. Dean Hendrix,et al.  Dynamic object viewers for data structures , 2007, SIGCSE '07.

[16]  T. Dean Hendrix,et al.  An extensible framework for providing dynamic data structure visualizations in a lightweight IDE , 2004, SIGCSE.

[17]  M.P. Bruce-Lockhart,et al.  Lifting the hood of the computer: program animation with the Teaching Machine , 2000, 2000 Canadian Conference on Electrical and Computer Engineering. Conference Proceedings. Navigating to a New Era (Cat. No.00TH8492).

[18]  Stephen Cooper,et al.  Evaluating the effectiveness of a new instructional approach , 2004, SIGCSE.

[19]  Christopher D. Hundhausen,et al.  Designing, visualizing, and discussing algorithms within a CS 1 studio experience: An empirical study , 2008, Comput. Educ..

[20]  John T. Stasko,et al.  Empirically evaluating the use of animations to teach algorithms , 1994, Proceedings of 1994 IEEE Symposium on Visual Languages.

[21]  Essi Lahtinen,et al.  Visualizations in Preparing for Programming Exercise Sessions , 2007, Electron. Notes Theor. Comput. Sci..

[22]  Theodore S. Norvell,et al.  Interactive embedded examples: a demonstration , 2006, ITICSE '06.

[23]  Lauri Malmi,et al.  Visual Algorithm Simulation Exercise System with Automatic Assessment: TRAKLA2 , 2004, Informatics Educ..

[24]  Susan H. Rodger,et al.  Web-based animation of data structures using JAWAA , 1998, SIGCSE '98.

[25]  Thomas L. Naps.  JHAVE: supporting algorithm visualization , 2005, IEEE Computer Graphics and Applications.

[26]  Tapio Salakoski,et al.  Multi-Perspective Study of Novice Learners Adopting the Visual Algorithm Simulation Exercise System TRAKLA2 , 2005, Informatics Educ..

[27]  John T. Stasko,et al.  Please address correspondence to , 2000 .

[28]  Rachelle S. Heller,et al.  Integrating algorithm animation into a learning environment , 1997, Comput. Educ..

[29]  Christopher D. Hundhausen,et al.  What you see is what you code: a radically dynamic algorithm visualization development model for novice learners , 2005, 2005 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC'05).

[30]  Guido Rößling,et al.  A Framework for Generating AV Content on-the-fly , 2007, Electron. Notes Theor. Comput. Sci..

[31]  A. W. Lawrence,et al.  Empirical studies of the value of algorithm animation in algorithm understanding , 1993 .

[32]  N. Hari Narayanan,et al.  Dancing hamsters and marble statues: characterizing student visualizations of algorithms , 2003, SoftVis '03.

[33]  John Stasko.  Animating algorithms with XTANGO , 1992, SIGA.

[34]  Jaime Urquiza-Fuentes,et al.  An Evaluation of the Effortless Approach to Build Algorithm Animations with WinHIPE , 2007, Electron. Notes Theor. Comput. Sci..

[35]  John T. Stasko,et al.  Rethinking the evaluation of algorithm animations as learning aids: an observational study , 2001, Int. J. Hum. Comput. Stud..

[36]  Scott Grissom,et al.  Algorithm visualization in CS education: comparing levels of student engagement , 2003, SoftVis '03.

[37]  Jaime Urquiza-Fuentes,et al.  An approach to effortless construction of program animations , 2008, Comput. Educ..

[38]  Ilan Shimshoni,et al.  MAVIS: A multi-level algorithm visualization system within a collaborative distance learning environment , 2008, J. Vis. Lang. Comput..

[39]  Susan H. Rodger,et al.  JAWAA: easy web-based animation from CS 0 to advanced CS courses , 2003, SIGCSE.

[40]  Theodore S. Norvell,et al.  Program and Algorithm Visualization in Engineering and Physics , 2007, Electron. Notes Theor. Comput. Sci..

[41]  Erkki Sutinen,et al.  Visualizing programs with Jeliot 3 , 2004, AVI.

[42]  Martha E. Crosby,et al.  From multimedia instruction to multimedia evaluation , 1995 .

[43]  Rossella Petreschi,et al.  Reversible Execution and Visualization of Programs with LEONARDO , 2000, J. Vis. Lang. Comput..

[44]  Jaime Urquiza-Fuentes,et al.  SRec: an animation system of recursion for algorithm courses , 2008, SIGCSE 2008.

[45]  Jaime Urquiza-Fuentes,et al.  WinHIPE: an IDE for functional programming based on rewriting and visualization , 2007, SIGP.

[46]  John T. Stasko,et al.  Using student-built algorithm animations as learning aids , 1997, SIGCSE '97.

[47]  Thomas L. Naps,et al.  JHAVÉ - More Visualizers (and Visualizations) Needed , 2007, Electron. Notes Theor. Comput. Sci..

[48]  J. Hamer.  A Lightweight Visualizer for Java , 2004 .

[49]  Niko Myller.  Automatic Generation of Prediction Questions during Program Visualization , 2007, Electron. Notes Theor. Comput. Sci..

[50]  Thomas L. Naps,et al.  Exploring the role of visualization and engagement in computer science education , 2003, ITiCSE-WGR '02.

[51]  Bernd Freisleben,et al.  The ANIMAL algorithm animation tool , 2000, ITiCSE '00.

[52]  Guido Rößling,et al.  Animalipse - An Eclipse Plugin for AnimalScript , 2009, Electron. Notes Theor. Comput. Sci..

[53]  Jaime Urquiza-Fuentes,et al.  SRec: an animation system of recursion for algorithm courses , 2008, ITiCSE.

[54]  Camil Demetrescu,et al.  Visual editing of animated algorithms: the Leonardo Web builder , 2006, AVI '06.

[55]  Jaime Urquiza-Fuentes,et al.  Human-Centered Aspects , 2006, Human-Centered Visualization Environments.

[56]  Clifford A. Shaffer,et al.  Using the Swan data structure visualization system for computer science education , 1996, SIGCSE '96.