Evolutionary development: a model for the design, implementation, and evaluation of ICT for education programmes

The impact of information and communication technology (ICT) in primary and secondary education is still an open question. Following review of the available literature, we classify the causes of the lack of impact on students' attainment in four dimensions: (1) the design and implementation of ICT in educational settings; (2) the evaluation of its impact; (3) the scaling up of these kinds of innovations; and (4) the cost-effectiveness of technology-enhanced learning environments. Based on this evidence, we proposed the evolutionary development model (EDM), which aims to produce a cost-effective and sustainable ICT for education (ICT4E) programme in three steps: efficacy, effectiveness, and efficiency. In each step, one component of the programme is built and validated in real educational settings. Therefore, the resultant ICT4E programme is ready to be replicated across the school system. We also show how the EDM guided the development of a programme based on mobile computer supported collaborative learning, known as Eduinnova. Finally, we discuss how EDM can serve as an analysis tool for researchers and policy makers. © 2012 Wiley Periodicals, Inc.

[1]  Penni Tearle,et al.  A theoretical and instrumental framework for implementing change in ICT in education , 2004 .

[2]  Robert E. Slavin,et al.  Perspectives on Evidence-Based Research in Education—What Works? Issues in Synthesizing Educational Program Evaluations , 2008 .

[3]  Robert E. Slavin,et al.  The Relationship Between Sample Sizes and Effect Sizes in Systematic Reviews in Education , 2009 .

[4]  ImpaCT 2 The Impact of Information and Communication Technologies on Pupil Learning and Attainment Summary , 2002 .

[5]  Martyn Wild,et al.  Computer education: new perspectives , 1994 .

[6]  J. Voogt,et al.  International handbook of information technology in primary and secondary education , 2008 .

[7]  Penni Tearle,et al.  ICT implementation: what makes the difference? , 2003, Br. J. Educ. Technol..

[8]  Tom Gilb,et al.  Evolutionary development , 1981, SOEN.

[9]  Henry M. Levin,et al.  Cost-Effectiveness: A Primer , 1985 .

[10]  Thomas C. Reeves,et al.  Enhancing the Worth of Instructional Technology Research through "Design Experiments" and Other Development Research Strategies , 2000 .

[11]  M. Cox,et al.  ICT and pedagogy : a review of the research literature : a report to the DfES (ISBN: 1844781356) , 2003 .

[12]  Chris Dede,et al.  The Cambridge Handbook of the Learning Sciences: Scaling Up , 2005 .

[13]  Miguel Nussbaum,et al.  A constructivist mobile learning environment supported by a wireless handheld network , 2004, J. Comput. Assist. Learn..

[14]  Victor R. Basili,et al.  Iterative and incremental developments. a brief history , 2003, Computer.

[15]  L. Walter.  Research in Education , 1895, Nature.

[16]  J. Enrique Hinostroza,et al.  ICT in Chilean Schools: Students' and Teachers' Access and Use of ICT , 2005 .

[17]  Jaime Sánchez,et al.  ICT & learning in Chilean schools: Lessons learned , 2008, Comput. Educ..

[18]  Miguel Nussbaum,et al.  A conceptual framework based on Activity Theory for mobile CSCL , 2007, Br. J. Educ. Technol..

[19]  Jacques Wainer,et al.  Too much computer and Internet use is bad for your grades, especially if you are young and poor: Results from the 2001 Brazilian SAEB , 2008, Comput. Educ..

[20]  Margaret D. Roblyer,et al.  Educational Technology Research That Makes a Difference: Series Introduction , 2005 .

[21]  Miguel Nussbaum,et al.  Computer supported collaborative learning using wirelessly interconnected handheld computers , 2004, Comput. Educ..

[22]  Rosemary Luckin,et al.  The learner centric ecology of resources: A framework for using technology to scaffold learning , 2008, Comput. Educ..

[23]  Thomas C. Reeves,et al.  Design research from a technology perspective , 2006 .

[24]  Venessa Keesler,et al.  Scaling-Up Exemplary Interventions , 2006 .

[25]  Robert C. Martin,et al.  Iterative and Incremental development , 1999 .

[26]  Camila Cortez,et al.  Teachers' support with ad-hoc collaborative networks , 2005, J. Comput. Assist. Learn..

[27]  Jonathan Miller,et al.  Monitoring and Evaluation of ICT in Education Projects A Handbook for Developing Countries , 2005 .

[28]  Roberto Agodini,et al.  The Effectiveness of Educational Technology: Issues and Recommendations for the National Study. , 2003 .

[29]  Michael Trucano,et al.  Knowledge Maps: ICTs in Education , 2005 .

[30]  Kwok-Wing Lai,et al.  ICT Supporting the Learning Process: The Premise, Reality, and Promise , 2008 .

[31]  James A. Kulik,et al.  Effects of Using Instructional Technology in Colleges and Universities: What Controlled Evaluation Studies Say , 2003 .

[32]  Leigh L. Linden,et al.  Colombia - The use and misuse of computers in education : evidence from a randomized controlled trial of a language arts program , 2013 .

[33]  Katerine Bielaczyc,et al.  Designing Social Infrastructure: Critical Issues in Creating Learning Environments With Technology , 2006 .

[34]  WainerJacques,et al.  Too much computer and Internet use is bad for your grades, especially if you are young and poor , 2008 .

[35]  Miguel Nussbaum,et al.  Dynamic Grouping in Collaborative Learning Supported by Wireless Handhelds , 2005, J. Educ. Technol. Soc..

[36]  W M,et al.  Table of Contents , 2020 .

[37]  Mary Simpson,et al.  Silicon Valley versus Silicon Glen: the impact of computers upon teaching and learning: a comparative study , 2003, Br. J. Educ. Technol..

[38]  R. Kozma.  Technology and Classroom Practices : An International Study , 2003 .

[39]  S. Raudenbush.  Advancing Educational Policy by Advancing Research on Instruction , 2008 .

[40]  Pierre Dillenbourg,et al.  Collaborative Learning: Cognitive and Computational Approaches , 1999 .

[41]  Rafi Nachmias,et al.  Factors Involved in the Implementation of Pedagogical Innovations Using Technology , 2004, Education and Information Technologies.

[42]  R. Earle.  The Integration of Instructional Technology into Public Education: Promises and Challenges , 2002 .

[43]  Ed Smeets,et al.  Modelling and supporting ICT implementation in secondary schools , 2001, Comput. Educ..

[44]  Miguel Nussbaum,et al.  A Monitoring and Evaluation Scheme for an ICT-Supported Education Program in Schools , 2010, J. Educ. Technol. Soc..

[45]  Miguel Nussbaum,et al.  Using multiple choice questions as a pedagogic model for face‐to‐face CSCL , 2009, Comput. Appl. Eng. Educ..

[46]  Roberto Agodini,et al.  Effectiveness of Reading and Mathematics Software Products: Findings From Two Student Cohorts. NCEE 2009-4041. , 2009 .

[47]  Susan McKenney,et al.  Educational design research , 2014 .

[48]  Miguel Nussbaum,et al.  Methodology for evaluating a novel education technology: a case study of handheld video games in Chile , 2006, Comput. Educ..

[49]  Jef Moonen,et al.  Policy from a Global Perspective , 2008 .

[50]  R. Sawyer.  The Cambridge Handbook of the Learning Sciences: Introduction , 2014 .

[51]  T. Plomp,et al.  Curriculum and staff development for ICT in education , 2003 .

[52]  Miguel Nussbaum,et al.  Co-located Collaborative Learning Video Game with Single Display Groupware Co-located Collaborative Learning Video Game with Single Display Groupware , 2022 .

[53]  Xitao Fan,et al.  Statistical Significance and Effect Size in Education Research: Two Sides of a Coin , 2001 .

[54]  Debra N. A. Hayes,et al.  ICT and learning: Lessons from Australian classrooms , 2007, Comput. Educ..

[55]  Ellen B. Mandinach,et al.  A Retrospective on Twenty Years of Education Technology Policy , 2005 .

[56]  Chris Dede,et al.  Scaling Up: Evolving Innovations beyond Ideal Settings to Challenging Contexts of Practice , 2005 .

[57]  John Robertson,et al.  The ambiguous embrace: twenty years of IT (ICT) in UK primary schools , 2002, Br. J. Educ. Technol..

[58]  Laurie A. Williams,et al.  Evolving beyond requirements creep: a risk-based evolutionary prototyping model , 2001, Proceedings Fifth IEEE International Symposium on Requirements Engineering.

[59]  Johan van Braak,et al.  Curricula and the use of ICT in education: Two worlds apart? , 2007, Br. J. Educ. Technol..

[60]  William R. Penuel,et al.  Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort , 2007 .

[61]  Margaret J. Cox,et al.  Effects of ICT: Do we know what we should know? , 2007, Education and Information Technologies.

[62]  Miguel Nussbaum,et al.  Multiple Mice based collaborative one-to-one learning , 2009, Comput. Educ..

[63]  J. Akker.  Curriculum Perspectives: An Introduction , 2004 .

[64]  Margaret Cox,et al.  RESEARCH METHODS: THEIR DESIGN, APPLICABILITY AND RELIABILITY , 2008 .

[65]  Anne McDougall,et al.  Researching IT in education , 2013 .

[66]  O. Bertelsen,et al.  Activity Theory , 2003 .

[67]  Douglas N. Harris,et al.  Toward Policy-Relevant Benchmarks for Interpreting Effect Sizes: Combining Effects With Costs , 2009 .

[68]  A. Sfard,et al.  Telling Identities: In Search of an Analytic Tool for Investigating Learning as a Culturally Shaped Activity , 2005 .

[69]  Alan M. Davis,et al.  A Strategy for Comparing Alternative Software Development Life Cycle Models , 1988, IEEE Trans. Software Eng..

[70]  S. Hennessy,et al.  Teacher perspectives on integrating ICT into subject teaching: commitment, constraints, caution, and change , 2005 .

[71]  D. Laurillard.  Modelling benefits-oriented costs for technology enhanced learning , 2007 .

[72]  Deryn Watson,et al.  Pedagogy before Technology: Re-thinking the Relationship between ICT and Teaching , 2001, Education and Information Technologies.

[73]  C. Coburn,et al.  Rethinking Scale: Moving Beyond Numbers to Deep and Lasting Change , 2003 .

[74]  Madhabi Chatterji,et al.  Evidence on “What Works”: An Argument for Extended-Term Mixed-Method (ETMM) Evaluation Designs , 2004 .

[75]  R. Kozma.  Learning with Media , 1991 .

[76]  Miguel Nussbaum,et al.  Scaffolding group explanation and feedback with handheld technology: impact on students’ mathematics learning , 2010 .

[77]  Rosemary Luckin.  The Ecology of Resources Model of Context: A Unifying Representation for the Development of Learning-Oriented Technologies , 2010, 2010 Second International Conference on Mobile, Hybrid, and On-Line Learning.

[78]  Kathryn Moyle.  Total Cost of Ownership and Total Value of Ownership , 2008 .

[79]  M. Cox,et al.  ICT and attainment: a review of the research literature , 2003 .

[80]  Jennifer Preece,et al.  Predicting quality in educational software: Evaluating For Learning, Usability and the Synergy between Them , 1999, Interact. Comput..

[81]  Colin Kirkman.  A model for the effective management of information and communications technology development in schools derived from six contrasting case studies , 2000 .

[82]  Joyojeet Pal,et al.  Multiple Mice for Computers in Education in Developing Countries , 2006, 2006 International Conference on Information and Communication Technologies and Development.

[83]  L. Lipponen,et al.  Patterns of participation and discourse in elementary students’ computer-supported collaborative learning , 2003 .

[84]  Thomas C. Reeves,et al.  Evaluation of the Design and Development of IT Tools in Education , 2008 .

[85]  A. McFarlane.  Perspectives on the relationships between ICT and assessment , 2001, J. Comput. Assist. Learn..

[86]  Rosa Maria Bottino,et al.  Activity theory: A framework for design and reporting on research projects based on ICT , 1999, Education and Information Technologies.

[87]  Miguel Nussbaum,et al.  Technology as small group face-to-face Collaborative Scaffolding , 2009, Comput. Educ..

[88]  P. Dillenbourg.  Integrating technologies into educational ecosystems , 2008 .

[89]  Rafi Nachmias,et al.  Analysis Schema for the Study of Domains and Levels of Pedagogical Innovation in Schools Using ICT , 2003, Education and Information Technologies.

[90]  Wilmad Kuiper,et al.  Curriculum landscapes and trends , 2003 .

[91]  Rachel Pilkington,et al.  Measuring the Impact of Information Technology on Students’ Learning , 2008 .

[92]  R. Kozma.  Technology and Classroom Practices , 2003 .

[93]  Anja Balanskat,et al.  The The ICT impact report , 2006 .

[94]  Miguel Nussbaum,et al.  An experimental study of the inclusion of technology in higher education , 2009, Comput. Appl. Eng. Educ..

[95]  Nwy Law,et al.  Cross-National Information and Communication Technology Policies and Practices in Education (Revised Second Edition) , 2009 .

[96]  Cher Ping Lim.  A theoretical framework for the study of ICT in schools: a proposal , 2002, Br. J. Educ. Technol..

[97]  David Reynolds,et al.  ICT - the hopes and the reality , 2003, Br. J. Educ. Technol..

[98]  Nancy Law,et al.  SITES2006–International comparative survey of pedagogical practices and ICT in education , 2007, Education and Information Technologies.

[99]  Yungwei Hao,et al.  Large-Scale Studies and Quantitative Methods , 2008 .