Development and Preliminary Validation of the Assessment of Computing for Elementary Students (ACES)

As reliance on technology increases in practically every aspect of life, all students deserve the opportunity to learn to think computationally from early in their educational experience. To support the kinds of computer science curriculum and instruction that makes this possible, there is an urgent need to develop and validate computational thinking (CT) assessments for elementary-aged students. We developed the Assessment of Computing for Elementary Students (ACES) to measure the CT concepts of loops and sequences for students in grades 3-5. The ACES includes block-based coding questions as well as non-programming, Bebras-style questions. We conducted cognitive interviews to understand student perspectives while taking the ACES. We piloted the assessment with 57 4th grade students who had completed a CT curriculum. Preliminary analyses indicate acceptable reliability and appropriate difficulty and discrimination among assessment items. The significance of this paper is to present a new CT measure for upper elementary students and to share its intentional development process.

[1]  Julie Sarama,et al.  Evaluation of a learning trajectory for length in the early years , 2011 .

[2]  Monica McGill,et al.  A Topical Review of Evaluation Instruments for Computing Education , 2019, SIGCSE.

[3]  C. Knox-Quinn.  International Society for Technology in Education. , 1992 .

[4]  Jere Confrey,et al.  Learning trajectories: a framework for connecting standards with curriculum , 2014 .

[5]  Jean Salac,et al.  TIPP&SEE: A Learning Strategy to Guide Students through Use - Modify Scratch Activities , 2020, SIGCSE.

[6]  Chris Stephenson,et al.  Running on Empty: the Failure to Teach K--12 Computer Science in the Digital Age , 2010 .

[7]  David Weintrop,et al.  Using Commutative Assessments to Compare Conceptual Understanding in Blocks-based and Text-based Programs , 2015, ICER.

[8]  Marcos Romn-Gonzlez,et al.  Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test , 2017, Comput. Hum. Behav..

[9]  James C. Lester,et al.  Development and Validation of the Middle Grades Computer Science Concept Inventory (MG-CSCI) Assessment , 2020, EURASIA Journal of Mathematics, Science and Technology Education.

[10]  Jirí Vanícek,et al.  Bebras Informatics Contest: Criteria for Good Tasks Revised , 2014, ISSEP.

[11]  M. Román González,et al.  COMPUTATIONAL THINKING TEST: DESIGN GUIDELINES AND CONTENT VALIDATION , 2015 .

[12]  Joyce Malyn-Smith,et al.  Computational thinking for youth in practice , 2011, INROADS.

[13]  Jacqueline P. Leighton.  Using Think-Aloud Interviews and Cognitive Labs in Educational Research , 2017 .

[14]  C. Lance,et al.  The Sources of Four Commonly Reported Cutoff Criteria , 2006 .

[15]  E. McColl.  Cognitive Interviewing. A Tool for Improving Questionnaire Design , 2006, Quality of Life Research.

[16]  Mark Guzdial,et al.  Developing a validated assessment of fundamental CS1 concepts , 2010, SIGCSE.

[17]  Sue Sentance,et al.  It's Computational Thinking! Bebras Tasks in the Curriculum , 2016, ISSEP.

[18]  Valentina Dagiene,et al.  Bebras International Contest on Informatics and Computer Literacy: Criteria for Good Tasks , 2008, ISSEP.

[19]  Marcos Román-González,et al.  Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test , 2017, Comput. Hum. Behav..

[20]  M. Battista.  Conceptualizations and Issues related to Learning Progressions, Learning Trajectories, and Levels of Sophistication , 2011, The Mathematics Enthusiast.

[21]  David Hammer,et al.  Implications of Complexity for Research on Learning Progressions , 2015 .

[22]  Diana Franklin,et al.  K-8 Learning Trajectories Derived from Research Literature: Sequence, Repetition, Conditionals , 2017, ICER.

[23]  Kristen Miller,et al.  Cognitive interviewing methodology , 2014 .

[24]  Guanhua Chen,et al.  Assessing elementary students' computational thinking in everyday reasoning and robotics programming , 2017, Comput. Educ..

[25]  Kristy Elizabeth Boyer,et al.  Development of a Lean Computational Thinking Abilities Assessment for Middle Grades Students , 2019, SIGCSE.

[26]  Martin A. Simon.  Reconstructing Mathematics Pedagogy from a Constructivist Perspective. , 1995 .