Toward A Framework for Formative Assessment of Conceptual Learning in K-12 Computer Science Classrooms

Unlike summative assessment that is aimed at grading students at the end of a unit or academic term, formative assessment is assessment for learning, aimed at monitoring ongoing student learning to provide feedback to both student and teacher, so that learning gaps can be addressed during the learning process. Education research points to formative assessment as a crucial vehicle for improving student learning. Formative assessment in K-12 CS and programming classrooms remains a crucial unaddressed need. Given that assessment for learning is closely tied to teacher pedagogical con- tent knowledge, formative assessment literacy needs to also be a topic of CS teacher PD. This position paper addresses the broad need to understand formative assessment and build a framework to understand the what, why, and how of formative assessment of introductory programming in K-12 CS. It shares specific pro- gramming examples to articulate the cycle of formative assessment, diagnostic evaluation, feedback, and action. The design of formative assessment items is informed by CS research on assessment design, albeit related largely to summative assessment and in CS1 contexts, and learning of programming, especially student misconceptions. It describes what teacher formative assessment literacy PD should entail and how to catalyze assessment-focused collaboration among K-12 CS teachers through assessment platforms and repositories.

[1]  Tony Clear,et al.  An introduction to program comprehension for computer science educators , 2010, ITiCSE-WGR '10.

[2]  P. Black,et al.  Assessment and Classroom Learning , 1998 .

[3]  Cynthia Collins Selby,et al.  Relationships: computational thinking, pedagogy of programming, and Bloom's Taxonomy , 2015, WiPSCE.

[4]  Roy D. Pea,et al.  Promoting active learning & leveraging dashboards for curriculum assessment in an OpenEdX introductory CS course for middle school , 2014, L@S.

[5]  Cruz Izu,et al.  Fostering Program Comprehension in Novice Programmers - Learning Activities and Learning Trajectories , 2019, ITiCSE-WGR.

[6]  Shuchi Grover,et al.  What We Can Learn About Student Learning From Open-Ended Programming Projects in Middle School Computer Science , 2018, SIGCSE.

[7]  H. Everson,et al.  Using Formative Assessment and Metacognition to Improve Student Achievement , 2013 .

[8]  Michal Armoni Spiral thinking: K--12 computer science education as part of holistic computing education , 2014, INROADS.

[9]  Errol Thompson,et al.  Bloom's taxonomy for CS assessment , 2008, ACE '08.

[10]  C. Wylie,et al.  Reaping the benefits of assessment for learning: achievement, identity, and equity , 2018, ZDM.

[11]  Shuchi Grover,et al.  Measuring Student Learning in Introductory Block-Based Programming: Examining Misconceptions of Loops, Variables, and Boolean Logic , 2017, SIGCSE.

[12]  Alexander Repenning,et al.  Closing The Cyberlearning Loop: Enabling Teachers To Formatively Assess Student Programming Projects , 2015, SIGCSE.

[13]  D. Wiliam Assessment in Education: Principles, Policy & Practice , 2008 .

[14]  Rebecca Vivian,et al.  A survey of Australian teachers' self-efficacy and assessment approaches for the K-12 digital technologies curriculum , 2018, WiPSCE.

[15]  Daniela Giordano,et al.  New Horizons in the Assessment of Computer Science at School and Beyond: Leveraging on the ViVA Platform , 2015, ITiCSE-WGR.

[16]  Elliot Soloway,et al.  Studying the Novice Programmer , 1988 .

[17]  Beth Simon,et al.  Evaluating a new exam question: Parsons problems , 2008, ICER '08.

[18]  C. Hoadley What is a Community of Practice and How Can We Support It , 2011 .

[19]  M. Román González,et al.  COMPUTATIONAL THINKING TEST: DESIGN GUIDELINES AND CONTENT VALIDATION , 2015 .

[20]  Barbara Boucher Owens,et al.  CSTA K--12 Computer Science Standards: Revised 2011 , 2011 .

[21]  Nicholas Lytle,et al.  Creation and validation of low-stakes rubrics for K-12 computer science , 2018, ITiCSE.

[22]  Richard T. White,et al.  Book Review: Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome) , 1983 .

[23]  Shuchi Grover,et al.  Designing an Assessment for Introductory Programming Concepts in Middle School Computer Science , 2020, SIGCSE.

[24]  Maya Israel,et al.  Empowering K–12 Students With Disabilities to Learn Computational Thinking and Computer Programming , 2015 .

[25]  Peter Hubwieser,et al.  Design and First Results of a Psychometric Test for Measuring Basic Programming Abilities , 2015, WiPSCE.

[26]  P. Black,et al.  Developing the theory of formative assessment , 2009 .

[27]  Dylan Wiliam,et al.  Formative Assessment: Getting the Focus Right , 2006 .

[28]  Diana Franklin,et al.  K-8 Learning Trajectories Derived from Research Literature: Sequence, Repetition, Conditionals , 2017, ICER.

[29]  Barbara Ericson,et al.  Solving parsons problems versus fixing and writing code , 2017, Koli Calling.

[30]  Patricia Haden,et al.  Parson's programming puzzles: a fun and effective learning tool for first programming courses , 2006 .

[31]  Angela Carbone,et al.  Reliably classifying novice programmer exam responses using the SOLO taxonomy , 2008 .

[32]  Yuning Xu,et al.  Principled Assessment of Student Learning in High School Computer Science , 2017, ICER.

[33]  W. James Popham,et al.  Assessment Literacy for Teachers: Faddish or Fundamental? , 2009 .

[34]  Susan M. Brookhart,et al.  Developing Measurement Theory for Classroom Assessment Purposes and Uses , 2005 .

[35]  Neil Brown,et al.  Repositories of teaching material and communities of use: nifty assignments and the greenroom , 2010, ICER '10.

[36]  Stephen H. Edwards,et al.  The Canterbury QuestionBank: building a repository of multiple-choice CS1 and CS2 questions , 2013, ITiCSE -WGR '13.

[37]  Kristy Elizabeth Boyer,et al.  Development of a Lean Computational Thinking Abilities Assessment for Middle Grades Students , 2019, SIGCSE.

[38]  D. Sadler Formative assessment and the design of instructional systems , 1989 .

[39]  Carsten Schulte,et al.  Block Model: an educational model of program comprehension as a tool for a scholarly approach to teaching , 2008, ICER '08.

[40]  Rebecca Vivian,et al.  Evaluation and Assessment Needs of Computing Education in Primary Grades , 2020, ITiCSE.

[41]  Raymond Lister,et al.  Not seeing the forest for the trees: novice programmers and the SOLO taxonomy , 2006, ITICSE '06.

[42]  Gregorio Robles,et al.  Dr. Scratch: Automatic Analysis of Scratch Projects to Assess and Foster Computational Thinking , 2015 .

[43]  Ulemu Luhanga,et al.  Teachers’ approaches to classroom assessment: a large-scale survey , 2016 .

[44]  Jean Salac,et al.  If They Build It, Will They Understand It? Exploring the Relationship between Student Code and Performance , 2020, ITiCSE.

[45]  Nathalia da Cruz Alves,et al.  CodeMaster - Automatic Assessment and Grading of App Inventor and Snap! Programs , 2018, Informatics Educ..

[46]  Shuchi Grover,et al.  Assessing Algorithmic and Computational Thinking in K-12: Lessons from a Middle School Classroom , 2017, Emerging Research, Practice, and Policy on Computational Thinking.

[47]  Raymond Lister One Small Step Toward a Culture of Peer Review and Multi- Institutional Sharing of Educational Resources: A Multiple Choice Exam for First Semester Programming Students , 2005, ACE.

[48]  Dylan Wiliam,et al.  Integrating Assessment with Learning: What Will It Take to Make It Work? , 2007 .