An Assessment Instrument of Technological Literacies in Makerspaces and FabLabs

Background As the maker movement is increasingly adopted into K-12 schools, students are developing new competences in exploration and fabrication technologies. This study assesses learning with these technologies in K-12 makerspaces and FabLabs. Purpose Our study describes the iterative process of developing an assessment instrument for this new technological literacy, the Exploration and Fabrication Technologies Instrument, and presents findings from implementations at five schools in three countries. Our index is generalizable and psychometrically sound, and permits comparison between student confidence and performance. Design/Method Our evaluation of distinct technology skills separates general computing, information and communication technology (ICT), and exploration and fabrication technologies (EFTs) into nonoverlapping areas of technological expertise required to perform their respective sets of tasks. The instrument also tracks student confidence in EFT skills and assesses how that confidence relates to actual task performance. Results Exploration and fabrication technologies constitute a new and distinct set of technology literacies arising from fabrication settings. The EFT instrument compares students' self-reported confidence with their performance on complex design tasks and demonstrates that, for students, exposure to general computing and ICT tools differs from exposure to EFT tools. Conclusion The EFT instrument captures a new and distinct set of technology literacies that arise within fabrication settings and are independent of both general computing and digital content production skills.

[1]  Joël Lebeaume,et al.  How do the Interactive White Board and the Radio Frequency Identification and tracking system work? Exploration of pupilsand#8217; spontaneous knowledge and didactical proposals for Technology Education , 2012 .

[2]  Greg Pearson Assessment of technological literacy in the United States , 2008 .

[3]  S. Martins,et al.  Media literacy worldwide. Similarities and differences of theoretical and practical approaches , 2011, 2011 IEEE Global Engineering Education Conference (EDUCON).

[4]  Marie Kraska,et al.  Factors Affecting Master Sergeants' Completion of Community College of the Air Force AAS Degree Requirements. , 1999 .

[5]  Werner A. Stahel,et al.  Sharpening Wald-type inference in robust regression for small samples , 2011, Comput. Stat. Data Anal..

[6]  Albert D. Ritzhaupt,et al.  Measuring Information and Communication Technology Literacy using a performance assessment: Validation of the Student Tool for Technology Literacy (ST2L) , 2014, Comput. Educ..

[7]  Ebru Gençtürk,et al.  Reliability and validity study of the technology proficiency self-assessment scale , 2010 .

[8]  B. Zimmerman Academic studing and the development of personal skill: A self-regulatory perspective , 1998 .

[9]  Jill Denner,et al.  Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? , 2012, Comput. Educ..

[10]  Jeffrey Alan Greene,et al.  Measuring critical components of digital literacy and their relationships with learning , 2014, Comput. Educ..

[11]  Eden Litt,et al.  Measuring users’ internet skills: A review of past assessments and a look toward the future , 2013, New Media Soc..

[12]  D. A. Fields What do Students Gain from a Week at Science Camp? Youth perceptions and the design of an immersive, research‐oriented astronomy camp , 2009 .

[13]  Brigid Barron Interest and Self-Sustained Learning as Catalysts of Development: A Learning Ecology Perspective , 2006, Human Development.

[14]  Kylie Peppler,et al.  Stitching Circuits: Learning About Circuitry Through E-textile Materials , 2013 .

[15]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[16]  Marcelo Worsley,et al.  Children Are Not Hackers: Building a Culture of Powerful Ideas, Deep Learning, and Equity in the Maker Movement. , 2016 .

[17]  Allan S. Cohen,et al.  Threats to the Valid Use of Assessments , 1996 .

[18]  Jean Piaget,et al.  To Understand Is to Invent , 1973 .

[19]  Pm Jenkinson,et al.  Cognitive , 2020, Definitions.

[20]  S. Livingstone,et al.  Converging traditions of research on media and information literacies: disciplinary, critical, and methodological issues , 2008 .

[21]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[22]  Lev Vygotsky Mind in society , 1978 .

[23]  Ji Shen,et al.  Digital Technologies and Assessment in the Twenty-First-Century Schooling , 2014 .

[24]  So Yoon Yoon,et al.  Validation of the Teaching Engineering Self‐Efficacy Scale for K‐12 Teachers: A Structural Equation Modeling Approach , 2014 .

[25]  D. Borsboom,et al.  The concept of validity. , 2004, Psychological review.

[26]  Jill Denner,et al.  Measuring Computational Thinking in Middle School using Game Programming , 2011 .

[27]  Hyun Kyoung Ro,et al.  Validity of the Contextual Competence Scale for Engineering Students , 2015 .

[28]  Kendra Spence Cheruvelil,et al.  Assessment of Student Learning Associated with Tree Thinking in an Undergraduate Introductory Organismal Biology Course , 2013, CBE life sciences education.

[29]  Eszter Hargittai,et al.  Survey Measures of Web-Oriented Digital Literacy , 2005 .

[30]  Seymour Papert,et al.  Mindstorms: Children, Computers, and Powerful Ideas , 1981 .

[31]  Jill M. Aldridge,et al.  Development and Validation of an Instrument to Measure Students’ Motivation and Self‐Regulation in Science Learning , 2011 .

[32]  Mike Petrich,et al.  It Looks Like Fun, but Are They Learning? , 2013 .

[33]  J. Piaget,et al.  The Child's Conception of the World , 1971 .

[34]  Blanca García Riaza,et al.  Informational Literacy And Information And Communication Technologies Use By Secondary Education Students In Spain: A Descriptive Study , 2011 .

[35]  Neil Gershenfeld,et al.  FAB: The Coming Revolution on Your Desktop--from Personal Computers to Personal Fabrication , 2005 .

[36]  R. Sawyer The Cambridge Handbook of the Learning Sciences: Introduction , 2014 .

[37]  J. Dewey Democracy and education : an introduction to philosophy of education / John Dewey , 1916 .

[38]  Marcelo Worsley,et al.  Analyzing Engineering Design through the Lens of Computation , 2014, J. Learn. Anal..

[39]  T. Lewis,et al.  Problem Posing--Adding a Creative Increment to Technological Problem Solving. , 1998 .

[40]  S. Chu,et al.  Using collaborative teaching and inquiry project-based learning to help primary school students develop information literacy and information skills , 2011 .

[41]  K. Peppler,et al.  Youth, Technology, and DIY , 2011 .

[42]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .