Assessing the benefits of interactivity and the influence of learning styles on the effectiveness of algorithm animation using web-based data structures courseware

Assessing the Benefits of Interactivity and the Influence of Learning Styles on the Effectiveness of Algorithm Animation using Web-based Data Structures Courseware by Duane Jeffrey Jarc Directed by Professor Michael Bliss Feldman This study used educational software—courseware—that contained algorithm animations and data structure visualizations that were implemented with the programming language Java, and were embedded in a collection of World Wide Web pages. This courseware differs from most previous algorithm animation systems because it is hosted on the Web and it incorporates a greater degree of interactivity than these earlier systems. The objective of this study was to determine whether the interactivity provided by this courseware produced a significant learning advantage and whether a student's learning style influenced its effectiveness. The courseware was used for two years at The George Washington University in the course, CSci 131 Data Structures, that was taught using the programming language Ada. Two controlled experiments were conducted during the second year to determine the effect of interactivity and the influence of students’ learning style. In addition, students evaluated the courseware each semester to uncover problems in the design and method of its use. Log file data was collected to analyze patterns of use, and selected students were recorded thinking-aloud while using the courseware to determine the understandability of the animations. The results of this study demonstrated that the students who used the interactive version of the courseware spent significantly more time using it than those who used the noninteractive version. Students who used the interactive version scored better on several of the questions that tested the more difficult lessons, but performed more poorly overall. None of the differences were statistically significant. No significant differences were observed between students of different learning styles. Student assessment of its effectiveness was uniformly high each semester that it was evaluated, which indicates that courseware of this kind is an effective component of an introductory data structures course.

[1]  Lloyd P. Rieber,et al.  The effects of computer animation on adult learning and retrieval tasks , 1990 .

[2]  John T. Stasko.  Using direct manipulation to build algorithm animations by demonstration , 1991, CHI '91.

[3]  John T. Stasko,et al.  Do algorithm animations assist learning?: an empirical study and analysis , 1993, INTERCHI.

[4]  Claire M. Fletcher-Flinn,et al.  The Efficacy of Computer Assisted Instruction (CAI): A Meta-Analysis , 1995 .

[5]  Marc H. Brown,et al.  Exploring algorithms using Balsa-II , 1988, Computer.

[6]  R. Felder,et al.  Learning and Teaching Styles in Engineering Education. , 1988 .

[7]  Michael B. Feldman,et al.  Validating a Demonstration Tool for Graphics-Assisted Debugging of Ada Concurrent Programs , 1989, IEEE Trans. Software Eng..

[8]  Barbara J. Cordell,et al.  A study of learning styles and computer-assisted instruction , 1991 .

[9]  Lesley Allinson,et al.  Learning Styles and Computer-Based Learning Environments , 1992, ICCAL.

[10]  Michael B. Feldman.  Software Construction and Data Structures with Ada 95 , 1996 .

[11]  Wilhelmina C. Savenye,et al.  How Do Learning Styles Relate to Performance in a Computer Applications Course , 1992 .

[12]  Stephen J. Hartley.  A hypermedia lab manual for operating systems: using the network to teach , 1996, ITiCSE '96.

[13]  John T. Stasko,et al.  Using student-built algorithm animations as learning aids , 1997, SIGCSE '97.

[14]  John T. Stasko,et al.  Using Animations to Learn about Algorithms: An Ethnographic Case Study , 1996 .

[15]  John T. Stasko,et al.  Visualizing program executions on large data sets , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[16]  Peter A. Gloor,et al.  AACE-algorithm animation for computer science education , 1992, Proceedings IEEE Workshop on Visual Languages.

[17]  R. Mayer,et al.  Multimedia learning: Are we asking the right questions? , 1997 .

[18]  Gruia-Catalin Roman,et al.  A taxonomy of program visualization systems , 1993, Computer.

[19]  Rachelle S. Heller,et al.  The Role of Hypermedia in Education: A Look at the Research Issues. , 1990 .

[20]  Susan H. Rodger,et al.  An overview of visualization: its use and design: report of the working group in visualization , 1996, ITiCSE '96.

[21]  C. Martin,et al.  A Media Taxonomy , 1995, IEEE Multim..

[22]  Bunny J. Tjaden.  Do lab modules in CS actually help students?: an empirical study , 1998, SIGCSE '98.

[23]  R. Kozma.  Will media influence learning? Reframing the debate , 1994 .

[24]  John T. Stasko.  Tango: A Framework and System for Algorithm Animation , 1990, Computer.

[25]  Blaine A. Price,et al.  A Principled Taxonomy of Software Visualization , 1993, J. Vis. Lang. Comput..

[26]  N. Entwistle.  Styles of learning and teaching: An integrated outline of educational psychology for students , 1988 .

[27]  James Bosco,et al.  An analysis of evaluations of interactive video , 1986 .

[28]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[29]  Gary R. Morrison,et al.  The media effects question: “Unresolvable” or asking the right question , 1994 .

[30]  Thomas G. West.  Visual Thinkers, Mental Models and Computer Visualization , 1992, Interactive Learning Through Visualization.

[31]  R. Clark.  Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses , 1985 .

[32]  Ephraim P. Glinert,et al.  An experiment into the use of auditory cues to reduce visual workload , 1989, CHI '89.

[33]  Marc H. Brown,et al.  Zeus: a system for algorithm animation and multi-view editing , 1991, Proceedings 1991 IEEE Workshop on Visual Languages.

[34]  T. K. Glennan,et al.  Fostering the Use of Educational Technology: Elements of a National Strategy , 1996 .

[35]  Christine M. Byrne.  Water on tap: the use of virtual reality as an educational tool , 1996 .

[36]  Jon C. Marshall,et al.  Examination of a learning style topology , 1987 .

[37]  John T. Stasko,et al.  Assessing Program Visualization Systems as Instructional Aids , 1992, ICCAL.

[38]  Gruia-Catalin Roman,et al.  Program visualization: the art of mapping programs to pictures , 1992, International Conference on Software Engineering.

[39]  Gruia-Catalin Roman,et al.  Pavane: a system for declarative visualization of concurrent computations , 1992, J. Vis. Lang. Comput..

[40]  Rachelle S. Heller.  Evaluating software: a review of the options , 1991 .

[41]  Albert T. Corbett,et al.  Assessing dynamics in computer-based instruction , 1996, CHI.

[42]  James A. Kulik,et al.  Programmed Instruction in Secondary Education: A Meta-Analysis of Evaluation Findings , 1982 .

[43]  Robert A. Reiser.  Clark's invitation to the dance: An instructional Designer's response , 1994 .

[44]  Ann E. Barron,et al.  The effectiveness of digital audio in computer-based training , 1991 .

[45]  Don P. Ester.  CAI, Lecture, and Student Learning Style: The Differential Effects of Instructional Method , 1994 .

[46]  Lawrence J. Najjar.  Multimedia information and learning , 1996 .

[47]  Bohdan Alexander Nebesh.  Using the World Wide Web for experimentation in reusable component comprehension , 1997 .

[48]  Richard E. Clark,et al.  Research and Theory on Multi-Media Learning Effects , 1992 .

[49]  Robert B. Kozma,et al.  A reply: Media and methods , 1994 .

[50]  Meera M. Blattner,et al.  Communicating and Learning Through Non-speech Audio , 1992 .

[51]  Robert Duisberg.  Visual Programming of Program Visualizations , 1990 .

[52]  R. Clark.  Reconsidering Research on Learning from Media , 1983 .

[53]  Helen C. Purchase,et al.  Which Aesthetic has the Greatest Effect on Human Understanding? , 1997, GD.

[54]  Martha E. Crosby,et al.  From multimedia instruction to multimedia evaluation , 1995 .

[55]  D. D. Wheeler,et al.  The Myers-Briggs Personality Type and Its Relationship to Computer Programming , 1994 .

[56]  Chen-Lin C. Kulik,et al.  Effectiveness of computer-based instruction: An updated analysis. , 1991 .

[57]  William W. Gaver,et al.  AUDITORY ICONS IN LARGE-SCALE COLLABORATIVE ENVIRONMENTS , 1990, SGCH.

[58]  Elliot Soloway.  No one is making money in educational software , 1998, CACM.

[59]  David W. Brooks.  Web-Teaching: A Guide to Designing Interactive Teaching for the World Wide Web , 1997 .

[60]  Curtis A. Carver,et al.  Felder's learning styles, Bloom's taxonomy, and the Kolb learning cycle: tying it all together in the CS2 course , 1996, SIGCSE '96.

[61]  Edward R. Tufte.  Visual explanations: images and quantities, evidence and narrative , 1997 .

[62]  Michael W. Cronin,et al.  The effects of visuals versus no visuals on learning outcomes from interactive multimedia instruction , 1997, J. Comput. High. Educ..

[63]  John T. Stasko,et al.  Rethinking the evaluation of algorithm animations as learning aids: an observational study , 2001, Int. J. Hum. Comput. Stud..

[64]  Adrianne Bonham.  Learning Style Instruments: Let the Buyer Beware. , 1988 .

[65]  Lawrence S. Corman,et al.  Cognitive style, personality type, and learning ability as factors in predicting the success of the beginning programming student , 1986, SGCS.

[66]  C. Claxton,et al.  Learning Styles: Implications for Improving Educational Practices. ASHE-ERIC Higher Education Report No. 4, 1987. , 1987 .

[67]  Marc Najork,et al.  A Java-based implementation of collaborative active textbooks , 1997, Proceedings. 1997 IEEE Symposium on Visual Languages (Cat. No.97TB100180).

[68]  Renée A. McCauley.  Warning! instructional animation tools abound on the Web , 1998, SGCS.

[69]  Wayne Citrin,et al.  Testing effectiveness of algorithm animation , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[70]  Satoshi Matsuoka,et al.  A framework for constructing animations via declarative mapping rules , 1994, Proceedings of 1994 IEEE Symposium on Visual Languages.

[71]  R. Mayer,et al.  Animations need narrations : an experimental test of a dual-coding hypothesis , 1991 .

[72]  Susan H. Rodger,et al.  Web-based animation of data structures using JAWAA , 1998, SIGCSE '98.

[73]  Thomas L. Naps.  Algorithm visualization served off the World Wide Web: why and how , 1996, ITiCSE.

[74]  John T. Stasko.  Simplifying algorithm animation with Tango , 1990, Proceedings of the 1990 IEEE Workshop on Visual Languages.

[75]  Steven C. Glassman.  A Turbo environment for producing algorithm animations , 1993, Proceedings 1993 IEEE Symposium on Visual Languages.

[76]  John T. Stasko,et al.  Empirically evaluating the use of animations to teach algorithms , 1994, Proceedings of 1994 IEEE Symposium on Visual Languages.

[77]  Verzekeren Naar Sparen,et al.  Cambridge , 1969, Humphrey Burton: In My Own Time.

[78]  Chen-Lin C. Kulik,et al.  Effectiveness of computer-based education in colleges , 1986 .

[79]  R. Clark.  Media will never influence learning , 1994 .

[80]  Yuen-Kuang Liao,et al.  Effects of Computer-Assisted Instruction on Cognitive Outcomes: A Meta-Analysis , 1992 .

[81]  Paul Týma.  Why are we using Java again? , 1998, CACM.

[82]  Helen L. Carlson.  Learning style and program design in interactive multimedia , 1991 .

[83]  Diane M. Billings,et al.  Effects of learning style preferences, attitude and GPA on learner achievement using computer assisted interactive videodisc instruction , 1992 .

[84]  H. W. Gordon,et al.  Cognitive Asymmetry, Computer Science Students, and Professional Programmers , 1990 .

[85]  James W. Keefe.  Profiling & Utilizing Learning Style. , 1988 .

[86]  John T. Stasko,et al.  WWW interactive learning environments for computer science education , 1996, SIGCSE '96.

[87]  Khalili Ahmad,et al.  The Effectiveness of Computer Applications , 1994 .

[88]  A. Ellis.  Research on educational innovations , 1993 .

[89]  Cheng-Chih Wu,et al.  Conceptual models and cognitive learning styles in teaching recursion , 1998, SIGCSE '98.

[90]  J. D. Fletcher.  The effectiveness and cost of interactive videodisc instruction , 1989 .

[91]  Rauf Yildiz,et al.  Evaluating multimedia applications , 1993 .

[92]  Thomas L. Naps,et al.  A multi-windowed environment for simultaneous visualization of related algorithms on the World Wide Web , 1998, SIGCSE '98.

[93]  William W. Gaver.  Synthesizing auditory icons , 1993, INTERCHI.

[94]  Aulikki Hyrskykari,et al.  Principles of Aladdin and Other Algorithm Animation Systems , 1990 .

[95]  Katherine D. Blake.  To San Francisco , 1911 .

[96]  A. David Marshall,et al.  Interactive hypermedia courseware for the World Wide Web , 1996, ITiCSE '96.

[97]  T. P. Cavaiani.  Cognitive style and diagnostic skills of student programmers , 1989 .

[98]  John Hershberger,et al.  Color and sound in algorithm animation , 1992, Computer.

[99]  Marc Najork,et al.  Collaborative Active Textbooks: a Web-based algorithm animation system for an electronic classroom , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[100]  Rosane Minghim,et al.  An illustrated analysis of sonification for scientific visualisation , 1995, Proceedings Visualization '95.

[101]  Xuemin Lin,et al.  How to draw a directed graph , 1989, [Proceedings] 1989 IEEE Workshop on Visual Languages.

[102]  L. Curry.  An Organization of Learning Styles Theory and Constructs. , 1983 .

[103]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[104]  Gruia-Catalin Roman,et al.  Abstraction in algorithm animation , 1992, Proceedings IEEE Workshop on Visual Languages.

[105]  Marc H. Brown,et al.  Algorithm animation , 1988 .

[106]  Shelley V. Goldman.  Sorting out sorting , 1982 .

[107]  Erkki Sutinen,et al.  Animation of user algorithms on the Web , 1997, Proceedings. 1997 IEEE Symposium on Visual Languages (Cat. No.97TB100180).

[108]  C. Claxton,et al.  Learning Styles: Implications for Improving Educational Practices , 1987 .

[109]  R. Mayer,et al.  The instructive animation: helping students build connections between words and pictures in multimedia learning , 1992 .

[110]  Lloyd P. Rieber,et al.  Animation in computer-based instruction , 1990 .

[111]  Alistair G. Sutcliffe,et al.  Designing effective multimedia presentations , 1997, CHI.

[112]  Amir Michail,et al.  Teaching binary tree algorithms through visual programming , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[113]  Jeffrey D. McWhirter.  AlgorithmExplorer: a student centered algorithm animation system , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[114]  Herbert L. Dershem,et al.  Tools for Web-based sorting animation , 1998, SIGCSE '98.

[115]  Rachelle S. Heller.  Different logo teaching styles: do the really matter , 1986 .

[116]  Ronald E. Larsen.  Relationship of learning style to the effectiveness and acceptance of interactive video instruction , 1992 .

[117]  Lorne Olfman,et al.  The Importance of Learning Style in End-User Training , 1990, MIS Q..

[118]  Rachelle S. Heller,et al.  Integrating algorithm animation into a learning environment , 1997, Comput. Educ..