The Practical Details of Building a CS Concept Inventory

Concept inventories (CIs) allow researchers and practitioners to measure student conceptual learning within a course or topic area. While they have enabled meaningful pedagogical change in other disciplines, there are relatively few CIs in computer science. In this paper, we report on our experiences as recent developers of a CI for basic data structures. We discuss each step along the route to a CI and offer tips based on what we have learned. We encourage others to create CIs, and we hope that this paper will serve as a practical guide through the process.

[1]  Cynthia Taylor,et al.  Computer science concept inventories: past and future , 2014, Comput. Sci. Educ..

[2]  Janelle M. Bailey,et al.  Development Of A Concept Inventory To Assess Students’ Understanding And Reasoning Difficulties About The Properties And Formation Of Stars , 2006 .

[3]  Cameron H. G. Wright,et al.  The signals and systems concept inventory , 2002, IEEE Transactions on Education.

[4]  Leilani Arthurs,et al.  The Oceanography Concept Inventory: A Semicustomizable Assessment for Measuring Student Understanding of Oceanography , 2015 .

[5]  Wendy K. Adams,et al.  Development and Validation of Instruments to Measure Learning of Expert‐Like Thinking , 2011 .

[6]  Charlene D'Avanzo,et al.  Biology Concept Inventories: Overview, Status, and Next Steps , 2008 .

[7]  M.J. Pavelich,et al.  Development, testing, and application of a chemistry concept inventory , 2004, 34th Annual Frontiers in Education, 2004. FIE 2004..

[8]  Jeremy V. Ernst,et al.  Towards a Concept Inventory for Algorithm Analysis Topics , 2017, SIGCSE.

[9]  Michael C. Loui,et al.  Identifying important and difficult concepts in introductory computing courses using a delphi process , 2008, SIGCSE '08.

[10]  Katherine K. Perkins,et al.  Colorado Upper-Division Electrostatics diagnostic: A conceptual assessment for the junior level , 2012 .

[11]  Hung-Wei Tseng,et al.  Evaluating student understanding of core concepts in computer architecture , 2013, ITiCSE '13.

[12]  Kirsty Young,et al.  Direct from the source: the value of 'think-aloud' data in understanding learning , 2009 .

[13]  M. Zeilik,et al.  Birth of the Astronomy Diagnostic Test: Prototest Evolution , 2001 .

[14]  R. Hake.  Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses , 1998 .

[15]  L. Ward,et al.  Faculty Performance on the Genomic Nursing Concept Inventory. , 2016, Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing.

[16]  E. Mazur,et al.  Peer Instruction: Ten years of experience and results , 2001 .

[17]  D. Hestenes,et al.  Force concept inventory , 1992 .

[18]  Min Li,et al.  An Item Response Theory Evaluation of a Language-Independent CS1 Knowledge Assessment , 2019, SIGCSE.

[19]  G. Norman,et al.  Development and Evaluation of the Conceptual Inventory of Natural Selection , 2002 .

[20]  Mark Guzdial,et al.  Developing a validated assessment of fundamental CS1 concepts , 2010, SIGCSE.

[21]  Philip M. Sadler,et al.  The Astronomy and Space Science Concept Inventory: Development and Validation of Assessment Instruments Aligned with the K-12 National Science Standards , 2009 .

[22]  Jennifer K Knight,et al.  The Genetics Concept Assessment: a new concept inventory for gauging student understanding of genetics. , 2008, CBE life sciences education.

[23]  F. J. Kelly.  The Kansas Silent Reading Tests. , 1916 .

[24]  Kirk Allen,et al.  The statistics concept inventory: a pilot study , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[25]  Steven A. Wolfman,et al.  Misconceptions and concept inventory questions for binary search trees and hash tables , 2014, SIGCSE.

[26]  Jan Vahrenhold,et al.  Detecting and understanding students' misconceptions related to algorithms and data structures , 2012, SIGCSE '12.

[27]  Timothy F. Slater,et al.  Development and Validation of the Light and Spectroscopy Concept Inventory , 2006 .

[28]  Jerome Epstein,et al.  Development and Validation of the Calculus Concept Inventory , 2007 .

[29]  Cynthia Taylor,et al.  Identifying Student Difficulties with Basic Data Structures , 2018, ICER.

[30]  Mark Guzdial,et al.  Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment , 2016, ICER.

[31]  Cynthia Taylor,et al.  Developing Course-Level Learning Goals for Basic Data Structures in CS2 , 2018, SIGCSE.

[32]  ZillesCraig,et al.  Identifying important and difficult concepts in introductory computing courses using a delphi process , 2008 .

[33]  Cynthia Taylor,et al.  Developing a pre- and post-course concept inventory to gauge operating systems learning , 2014, SIGCSE.

[34]  Emily M. Geraghty Ward,et al.  Revisiting the Geoscience Concept Inventory: A call to the community , 2011 .

[35]  Cynthia Bailey Lee,et al.  BDSI: A Validated Concept Inventory for Basic Data Structures , 2019, ICER.

[36]  Sally Hamouda,et al.  A basic recursion concept inventory , 2017, Comput. Sci. Educ..

[37]  Michael C. Loui,et al.  A psychometric evaluation of the digital logic concept inventory , 2014, Comput. Sci. Educ..