Measuring Student Competences in German Upper Secondary Computer Science Education

Within the interdisciplinary research project “Measurement Procedure for Informatics in Secondary Education (MoKoM)”, conducted at the Universities of Paderborn and Siegen with help from Peter Hubwieser, Technical University of Munich, researchers aimed to develop a theoretically and empirically sound competence model for the domains of system comprehension and system modelling, alongside an evaluated measurement instrument to assess competences of students in upper secondary computer science education in German schools. The competence model was thoroughly developed by conducting several theoretical and empirical steps. Based on the model measurement items were constructed and compiled into an instrument with 87 items of varying complexity. The instrument was divided into six booklets and distributed to over 800 computer science students in three German states. The returned 583 tests were analysed by means of the multidimensional item response theory, to assess the item difficulty on the one hand and the student abilities on the other. The results are used in several ways: to evaluate the competence model, to revise the measurement instrument and to define proficiency levels in a competence level model by using the method of scale anchoring.

[1]  Margaret Wu,et al.  ACER conquest: generalised item response modelling software , 1998 .

[2]  Johannes Magenheim,et al.  Informatics Systems and Modelling - Case Studies of Expert Interviews , 2010, Key Competencies in the Knowledge Society.

[3]  Johannes Magenheim,et al.  Competencies for informatics systems and modeling: Results of qualitative content analysis of expert interviews , 2010, IEEE EDUCON 2010 Conference.

[4]  Johannes Magenheim,et al.  Competence model for informatics modelling and system comprehension , 2013, 2013 IEEE Global Engineering Education Conference (EDUCON).

[5]  Joint Task Force on Computing Curricula.  Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science , 2013 .

[6]  Johannes Hartig,et al.  Modellierung von Kompetenzen mit mehrdimensionalen IRT-Modellen. Projekt MIRT , 2010 .

[7]  Roland T. Mittermeir,et al.  Informatics Education - Supporting Computational Thinking, Third International Conference on Informatics in Secondary Schools - Evolution and Perspectives, ISSEP 2008, Torun, Poland, July 1-4, 2008, Proceedings , 2008, ISSEP.

[8]  Johannes Hartig,et al.  Skalierung und Definition von Kompetenzniveaus , 2007 .

[9]  Eckhard Klieme,et al.  Reporting results of large-scale assessment in psychologically and educationally meaningful terms: Construct validation and proficiency scaling in TIMSS. , 2002 .

[10]  June E. Downey,et al.  Les r?v?lations de l'?criture d'apr?s un contr?le scientifique. , 1908 .

[11]  Peter Hubwieser,et al.  Computer Science Education in Secondary Schools -- The Introduction of a New Compulsory Subject , 2012, TOCE.

[12]  Robert E. Ployhart,et al.  Situational Judgment Tests : Theory, Measurement, and Application , 2013 .

[13]  Johannes Hartig,et al.  Multidimensional IRT models for the assessment of competencies , 2009 .

[14]  Peter Hubwieser,et al.  Analysis of Learning Objectives in Object Oriented Programming , 2008, ISSEP.

[15]  Sally Fincher,et al.  Computer Science Curricula 2013 , 2013 .

[16]  Thomas Rhode.  Entwicklung und Erprobung eines Instruments zur Messung informatischer Modellierungskompetenz im fachdidaktischen Kontext , 2013 .

[17]  P. Mayring.  Qualitative Content Analysis , 2000 .

[18]  J. C. Flanagan.  Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[19]  Dana L. Kelly.  Interpreting the Third International Mathematics and Science Study (TIMSS) achievement scales using scale anchoring , 1999 .

[20]  Eckhard Klieme,et al.  Sprachliche Kompetenzen. Konzepte und Messung. DESI-Studie (Deutsch Englisch Schülerleistungen International) , 2007 .

[21]  H. Moosbrugger,et al.  Testtheorie und Fragebogenkonstruktion , 2008 .

[22]  Detlev Leutner,et al.  Kompetenzmodellierung: Zwischenbilanz des DFG-Schwerpunktprogramms und Perspektiven des Forschungsansatzes , 2010 .

[23]  正弘 玉木,et al.  PISA(Programme for International Student Assessment)で求められている読解力教育の重要性 , 2016 .

[24]  J. Rost,et al.  Lehrbuch Testtheorie, Testkonstruktion , 1999 .

[25]  Philip Osteen.  An Introduction to Using Multidimensional Item Response Theory to Assess Latent Factor Structures , 2010, Journal of the Society for Social Work and Research.

[26]  Nicholas Reynolds,et al.  Key Competencies in the Knowledge Society - IFIP TC 3 International Conference, KCKS 2010, Held as Part of WCC 2010, Brisbane, Australia, September 20-23, 2010. Proceedings , 2010, Key Competencies in the Knowledge Society.

[27]  Allen B. Tucker,et al.  A Model Curriculum for K--12 Computer Science: Final Report of the ACM K--12 Task Force Curriculum Committee , 2003 .

[28]  Wolfgang Wagner,et al.  Methodische Grundlagen der Messung und Erklärung sprachlicher Kompetenzen , 2008 .