The Design Implementation Framework: Iterative Design from the Lab to the Classroom

This chapter explores three broad principles of user-centered design methodologies: participatory design, iteration, and usability considerations. The authors highlight the importance of considering teachers as a prominent type of ITS end user, by describing the barriers teachers face as users and their role in educational technology design. To exemplify these points, they draw upon their experiences in developing the Writing Pal—an ITS for writing strategy instruction. The chapter concludes by offering a design approach—the Design Implementation Framework (DIF)—that builds upon existing cyclical design methods but is tailored to the design of ITSs and other educational technologies.

[1]  J. Preece,et al.  User-Centered Design , 2004 .

[2]  Chih-Hung Wu,et al.  The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability , 2014, Comput. Hum. Behav..

[3]  Leonard A. Annetta,et al.  Science Teacher Efficacy and Extrinsic Factors Toward Professional Development Using Video Games in a Design-Based Research Model: The Next Generation of STEM Learning , 2013 .

[4]  Arthur C. Graesser,et al.  Feeling, Thinking, and Computing with Affect-Aware Learning Technologies , 2015 .

[5]  Neil T. Heffernan,et al.  The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching , 2014, International Journal of Artificial Intelligence in Education.

[6]  Eva Brandt,et al.  Designing exploratory design games: a framework for participation in Participatory Design? , 2006, PDC '06.

[7]  P. Paulus,et al.  Idea Generation in Groups : A Basis for Creativity in Organizations , 1994 .

[8]  Laura K. Allen,et al.  The Writing Pal Intelligent Tutoring System: Usability Testing and Development , 2014 .

[9]  Brian C. Nelson,et al.  Design-based research strategies for studying situated learning in a multi-user virtual environment , 2004 .

[10]  Laura K. Allen,et al.  Does agency matter?: Exploring the impact of controlled behaviors within a game-based environment , 2015, Comput. Educ..

[11]  Rod D. Roscoe,et al.  Writing pal: Feasibility of an intelligent writing strategy tutor in the high school classroom , 2013 .

[12]  Susan McKenney,et al.  Educational design research: the value of variety , 2006 .

[13]  Kasia Muldner,et al.  Comparing Learning from Observing and from Human Tutoring. , 2014 .

[14]  Matej Cebecauer,et al.  Using OpenStreetMap to improve population grids in Europe , 2017 .

[15]  W. Penuel,et al.  Design-Based Implementation Research: An Emerging Model for Transforming the Relationship of Research and Practice , 2013 .

[16]  Joan Greenbaum A Design of One’s Own: Towards Participatory Design in the United States , 2017 .

[17]  Elizabeth Boling,et al.  Instructional Design Models , 2014 .

[18]  Henry Jay Becker,et al.  How Exemplary Computer-Using Teachers Differ From Other Teachers: Implications for Realizing the Po , 1994 .

[19]  Alexander Maedche,et al.  Exploring principles of user-centered agile software development: A literature review , 2015, Inf. Softw. Technol..

[20]  T. Seidel,et al.  Participatory design of learning environments: integrating perspectives of students, teachers, and designers , 2014 .

[21]  Michael J. Muller,et al.  Participatory design: the third space in HCI , 2002 .

[22]  Linda Darling-Hammond,et al.  Powerful Learning: What We Know About Teaching for Understanding , 2008 .

[23]  Matthew J. Koehler,et al.  Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge , 2006, Teachers College Record: The Voice of Scholarship in Education.

[24]  Brian C. Nelson,et al.  The Development of River City, a Multi-User Virtual Environment-Based Scientific Inquiry Curriculum: Historical and Design Evolutions , 2010 .

[25]  David Lebow,et al.  Constructivist values for instructional systems design: Five principles toward a new mindset , 1993 .

[26]  Erica L. Snow,et al.  iSTART-2: A reading comprehension and strategy instruction tutor , 2016 .

[27]  Arthur C. Graesser,et al.  Learning by Communicating in Natural Language With Conversational Agents , 2014 .

[28]  M. Goos Researcher–teacher relationships and models for teaching development in mathematics education , 2014 .

[29]  Frank Maurer,et al.  Agile Methods and User-Centered Design: How These Two Methodologies are Being Successfully Integrated in Industry , 2008, Agile 2008 Conference.

[30]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[31]  Rebecca Clemente,et al.  Instructional systems design and public schools , 1990 .

[32]  Danielle S. McNamara,et al.  The Writing-Pal tutoring system: Development and design , 2010 .

[33]  Danielle S. McNamara,et al.  Lost in Springdale: An interactive narrative for adult literacy learners , 2016 .

[34]  Asimina Vasalou,et al.  Bridging serious games and participatory design , 2014, Int. J. Child Comput. Interact..

[35]  Danielle S McNamara,et al.  Natural language processing in an intelligent writing strategy tutoring system , 2012, Behavior Research Methods.

[36]  Michael Molenda,et al.  In Search of the Elusive ADDIE Model , 2003 .

[37]  Nian-Shing Chen,et al.  Usability of affective interfaces for a digital arts tutoring system , 2014, Behav. Inf. Technol..

[38]  Heather K. Holden,et al.  Understanding the Influence of Perceived Usability and Technology Self-Efficacy on Teachers’ Technology Acceptance , 2011 .

[39]  H. Cooper,et al.  A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students’ mathematical learning. , 2013 .

[40]  Fethi A. Inan,et al.  Factors affecting technology integration in K-12 classrooms: a path model , 2010 .

[41]  William Sugar,et al.  Applying Human-Centered Design to Technology Integration , 2002 .

[42]  Arthur C. Graesser,et al.  AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring , 2014, International Journal of Artificial Intelligence in Education.

[43]  K. VanLehn The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[44]  Daniel F. McCaffrey,et al.  Effectiveness of Cognitive Tutor Algebra I at Scale , 2014 .

[45]  Peggy A. Ertmer Addressing first- and second-order barriers to change: Strategies for technology integration , 1999 .

[46]  Rod D. Roscoe,et al.  Presentation, expectations, and experience: Sources of student perceptions of automated writing evaluation , 2017, Comput. Hum. Behav..

[47]  Laura K. Allen,et al.  {ENTER}ing the Time Series {SPACE}: Uncovering the Writing Process through Keystroke Analyses , 2016, EDM.

[48]  Mica R. Endsley,et al.  Designing for Situation Awareness : An Approach to User-Centered Design , 2003 .

[49]  Kurt VanLehn,et al.  The Behavior of Tutoring Systems , 2006, Int. J. Artif. Intell. Educ..

[50]  Thomas C. Reeves,et al.  The implications of the differences between design research and instructional systems design for educational technology researchers and practitioners , 2010 .

[51]  Olusola O. Adesope,et al.  Intelligent tutoring systems and learning outcomes: A meta-analysis , 2014 .

[52]  Lizanne DeStefano,et al.  Impact of a Student-Teacher-Scientist Partnership on Students' and Teachers' Content Knowledge, Attitudes toward Science, and Pedagogical Practices. , 2014 .

[53]  Yanghee Kim,et al.  Research-Based Design of Pedagogical Agent Roles: a Review, Progress, and Recommendations , 2016, International Journal of Artificial Intelligence in Education.

[54]  Robert M. Gagné,et al.  Instructional Technology: Foundations , 1988 .

[55]  Heather Leary,et al.  Mathematical Tasks as Boundary Objects in Design-Based Implementation Research , 2014 .

[56]  Travis Lowdermilk User-Centered Design , 2013 .

[57]  Cheryl I. Johnson,et al.  Revising the Redundancy Principle in Multimedia Learning. , 2008 .

[58]  Michelene T. H. Chi,et al.  Active-Constructive-Interactive: A Conceptual Framework for Differentiating Learning Activities , 2009, Top. Cogn. Sci..

[59]  David Wallace,et al.  Assessing the Quality of Ideas From Prolific, Early-Stage Product Ideation , 2010 .

[60]  David Engel,et al.  Intelligent Tutoring Systems Evolutions In Design , 2016 .

[61]  Peggy A. Ertmer,et al.  Teacher beliefs and technology integration practices: A critical relationship , 2012, Comput. Educ..

[62]  Danielle S. McNamara,et al.  Partial Verbal Redundancy in Multimedia Presentations for Writing Strategy Instruction. , 2015 .

[63]  Min Kyu Kim,et al.  Teacher Beliefs and Technology Integration. , 2013 .

[64]  Allan Collins,et al.  Toward a Design Science of Education , 1992 .

[65]  Johan van Braak,et al.  The impact of primary school teachers' educational beliefs on the classroom use of computers , 2008, Comput. Educ..

[66]  Martha Stone Wiske,et al.  Teaching for understanding : linking research with practice , 1998 .

[67]  Danielle S. McNamara,et al.  Game-Based Writing Strategy Practice with the Writing Pal , 2013 .

[68]  Danielle S. McNamara,et al.  Internal Usability Testing of Automated Essay Feedback in an Intelligent Writing Tutor , 2011, FLAIRS Conference.

[69]  Robert E. Roth,et al.  Wireframing for interactive & web-based geographic visualization: designing the NOAA Lake Level Viewer , 2017 .

[70]  Abbie Brown,et al.  Design experiments: Theoretical and methodological challenges in creating complex interventions in c , 1992 .

[71]  D. B. Danbjørg,et al.  Participatory design methods in telemedicine , 2016 .

[72]  Richard E. Mayer,et al.  Affective impact of navigational and signaling aids to e-learning , 2012, Comput. Hum. Behav..

[73]  R. Mayer,et al.  Benefits of emotional design in multimedia instruction , 2014 .

[74]  Scotty D. Craig,et al.  Usability evaluation of intelligent tutoring system , 2015 .

[75]  ELLIOTT SEIF 이해를 위한 교수모형(Teaching for Understanding)의 적용에 관한 연구 , 2001 .

[76]  Albert T. Corbett,et al.  Intelligent Tutoring Systems , 1985, Science.

[77]  K. Sheingold,et al.  Commonalities and Distinctive Patterns in Teachers' Integration of Computers , 1993, American Journal of Education.

[78]  Finn Kensing,et al.  Participatory Design: Issues and Concerns , 2004, Computer Supported Cooperative Work (CSCW).

[79]  Matthew J. Koehler,et al.  What Happens When Teachers Design Educational Technology? The Development of Technological Pedagogical Content Knowledge , 2005 .

[80]  Laura K. Allen,et al.  L2 writing practice: Game enjoyment as a key to engagement , 2014 .

[81]  Eugenio Bravo The Hazards of Leaving Out the Users , 2017 .

[82]  Laura K. Allen,et al.  Higher order thinking in comprehension , 2015 .

[83]  R. Lambert,et al.  The Association between Teachers' Beliefs, Enacted Practices, and Student Learning in Mathematics , 2013 .

[84]  Arthur C. Graesser,et al.  The Writing-Pal: Natural Language Algorithms to Support Intelligent Tutoring on Writing Strategies , 2012 .