Assessment of 21st century ICT skills in Chile: Test design and results from high school level students

This paper describes a study that evaluates fifteen-year-old Chilean students Information and Communication Technology (ICT) skills. The paper presents an operational definition of ICT skills, an instrument measuring these skills as well as the students' results in the test. The definition of ICT skills used considers Chile's curricular framework, functional and cognitive skills. Specifically, ICT skills were defined as the capacity to solve problems of information, communication and knowledge in digital environments. A performance-based assessment was designed in a virtual environment to measure these skills. The analysis of the results showed that the majority of students were able to solve tasks related to the use of information as consumers, i.e., approximately three quarters of the students were able to search for information and half of them were also able to organize and manage digital information. Additionally, they show that very few students were able to succeed in tasks related to the use of information as producers, i.e., only one third of the students were able to develop their own ideas in a digital environment and less than one fifth were able to refine digital information and create a representation in a digital environment. Socioeconomic group, access, daily use and confidence in doing ICT-related activities were all positively associated with higher scores, showing the need to implement strategies to compensate this inequality, possibly by explicitly defining these aims in the national curriculum.

[1]  Ben Williamson,et al.  Digital participation, digital literacy, and school subjects: A review of the policies, literature and evidence , 2009 .

[2]  John Kirriemuir,et al.  Literature Review in Games and Learning , 2004 .

[3]  Gergory J. Cizek,et al.  Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests , 2006 .

[4]  Ximena López,et al.  Beyond Nintendo: design and assessment of educational video games for first and second grade students , 2003, Comput. Educ..

[5]  Kurt Squire,et al.  Video games in education , 2003, Int. J. Intell. Games Simul..

[6]  Youngkyun Baek,et al.  Not just fun, but serious strategies: Using meta-cognitive strategies in game-based learning , 2009, Comput. Educ..

[7]  Anton Beguin,et al.  Using Classical Test Theory in Combination with Item Response Theory , 2003 .

[8]  Miguel Nussbaum,et al.  Is the use of information and communication technology related to performance in working memory tasks? Evidence from seventh-grade students , 2011, Comput. Educ..

[9]  Julian Sefton-Green,et al.  Literature Review in Informal Learning with Technology Outside School , 2004 .

[10]  Donald A. Norman Worsening the Knowledge Gap * , 1984 .

[11]  Seugnet Blignaut,et al.  ICT in education policy and practice in developing countries: South Africa and Chile compared through SITES 2006 , 2010, Comput. Educ..

[12]  B. Silverman Density estimation for statistics and data analysis , 1986 .

[13]  John A. Meskauskas,et al.  Evaluation Models for Criterion-Referenced Testing: Views Regarding Mastery and Standard-Setting , 1976 .

[14]  Francis Tuerlinckx,et al.  A Hierarchical IRT Model for Criterion-Referenced Measurement , 2000 .

[15]  Miguel Nussbaum,et al.  Computer supported collaborative learning using wirelessly interconnected handheld computers , 2004, Comput. Educ..

[16]  Robert L. Linn,et al.  Educational Assessment: Expanded Expectations and Challenges , 1993 .

[17]  Jan-Eric Gustafsson,et al.  Measuring Socioeconomic Status at Individual and Collective Levels , 2004 .

[18]  R. Catts,et al.  Towards information literacy indicators , 2008 .

[19]  Marina Umaschi Bers,et al.  Let the Games Begin: Civic Playing on High-Tech Consoles , 2010 .

[20]  J. Fraillon,et al.  National Assessment Program--ICT Literacy Years 6 & 10 Report, 2005. , 2007 .

[21]  Michael C. McKenna,et al.  Handbook of Literacy and Technology : Transformations in A Post-typographic World , 1998 .

[22]  Olga Perfilieva 2nd session of the OECD Education Policy Committee , 2007 .

[23]  Urs Gasser,et al.  Working Towards a Deeper Understanding of Digital Safety for Children and Young People in Developing Nations , 2010 .

[24]  Samuel A. Livingston,et al.  Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. , 1982 .

[25]  Francis Tuerlinckx,et al.  Identified Parameters, Parameters of Interest and Their Relationships , 2009 .

[26]  I. Spence,et al.  Video Games and Spatial Cognition , 2010 .

[27]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[28]  Philip C. Abrami,et al.  What Forty Years of Research Says About the Impact of Technology on Learning , 2011 .

[29]  Myron C. Tuman Word Perfect: Literacy In The Computer Age , 1992 .

[30]  C. Dede Comments on Greenhow, Robelia, and Hughes: Technologies That Facilitate Generating Knowledge and Possibly Wisdom , 2009 .

[31]  Jaime Sánchez,et al.  ICT & learning in Chilean schools: Lessons learned , 2008, Comput. Educ..

[32]  Paul De Boeck,et al.  On the Relationships between Sum Score Based Estimation and Joint Maximum Likelihood Estimation , 2008 .

[33]  E. Dieterle,et al.  Neomillennial Learning Styles and River City , 2023, Children, Youth and Environments.

[34]  Richard J. Murnane,et al.  How Computerized Work and Globalization Shape Human Skill Demands , 2006 .

[35]  Henry Jenkins Confronting the Challenges of Participatory Culture: Media Education for the 21st Century , 2006 .

[36]  Vincenzo Spiezia,et al.  Does Computer Use Increase Educational Achievements? Student-level Evidence from PISA , 2011 .

[37]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .