Assuring Quality in Competence Assessments: The Value Added of Applying Different Assessment Approaches to Professional Education

Due to their multidimensional nature and their development in group settings, the assessment of competencies acquired in professional education goes beyond typical methodological difficulties related to any assessment of behaviour, personality or capability. Competencies are complex constructs for which it is challenging to develop reliable and valid assessments because classical assumptions such as unidimensionality, which means that components of an assessment should reflect only one underlying dimension, and independence of the individuals assessed are violated.

[1]  Martin Mulder,et al.  The concept of competence in the development of vocational education and training in selected EU member states: a critical analysis , 2007 .

[2]  J. Gustafsson,et al.  Beyond dichotomies: Competence viewed as a continuum , 2015 .

[3]  L. Cronbach,et al.  THEORY OF GENERALIZABILITY: A LIBERALIZATION OF RELIABILITY THEORY† , 1963 .

[4]  M. Holodynski,et al.  Evaluation eines videobasierten Trainingsseminars zur Förderung der professionellen Wahrnehmung von Klassenführung im Grundschulunterricht , 2013 .

[5]  Franz Emanuel Weinert,et al.  Concept of competence: A conceptual clarification , 2001 .

[6]  Mark Wilson,et al.  Seeking a Balance Between the Statistical and Scientific Elements in Psychometrics , 2013, Psychometrika.

[7]  Richard J. Shavelson,et al.  On an Approach to Testing and Modeling Competence , 2013 .

[8]  Sigrid Blömeke,et al.  Effects of Mathematics Content Knowledge on Pre-school Teachers’ Performance: a Video-Based Assessment of Perception and Planning Abilities in Informal Learning Situations , 2015 .

[9]  N. Schmitt.  Uses and abuses of coefficient alpha. , 1996 .

[10]  R. Hambleton,et al.  An NCME Instructional Module on Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development. , 2005 .

[11]  Cornelis A.W. Glas,et al.  Computerized adaptive testing : theory and practice , 2000 .

[12]  Lyle M. Spencer,et al.  Competence at Work: Models for Superior Performance , 1993 .

[13]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[14]  Francis Tuerlinckx,et al.  A nonlinear mixed model framework for item response theory. , 2003, Psychological methods.

[15]  Jan-Eric Gustafsson,et al.  Unidimensionality and interpretability of psychological instruments , 2009 .

[16]  Gabriele Kaiser,et al.  Family Background, Entry Selectivity and Opportunities to Learn: What Matters in Primary Teacher Education? An International Comparison of Fifteen Countries , 2012 .

[17]  Sigrid Blömeke,et al.  Future teachers’ general pedagogical knowledge from a comparative perspective: does school experience matter? , 2012 .

[18]  Robert J. Mislevy,et al.  Implications of Evidence‐Centered Design for Educational Testing , 2007 .

[19]  Mark Wilson,et al.  From Principles to Practice: An Embedded Assessment System , 2000 .

[20]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[21]  R. Bosker.  Boekbespreking van "A.S. Bryk & S.W. Raudenbusch - Hierarchical linear models: Applications and data analysis methods" : Sage Publications, Newbury Parki, London/New Delhi 1992 , 1995 .

[22]  Gian-Carlo Rota,et al.  History of Computing in the Twentieth Century , 1980 .

[23]  Aiso Heinze,et al.  Measuring Mathematical Competences of Engineering Students at the Beginning of Their Studies , 2015 .

[24]  William B. Walstad,et al.  Validating test score interpretations by cross-national comparison: Comparing the results of students from Japan and Germany on an American test of economic knowledge in higher education. , 2015 .

[25]  Richard C Gershon,et al.  Computer adaptive testing , 1994 .

[26]  M. Reckase.  Multidimensional Item Response Theory , 2009 .

[27]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[28]  R. Shavelson.  Performance Assessments: Political Rhetoric and Measurement Reality , 1992 .

[29]  Miriam Gamoran Sherin,et al.  How Different Video Club Designs Support Teachers in “Learning to Notice” , 2006 .

[30]  J. Bruner.  Toward a Theory of Instruction , 1966 .

[31]  Stephen W. Raudenbush,et al.  1. Ecometrics: Toward a Science of Assessing Ecological Settings, with Application to the Systematic Social Observation of Neighborhoods , 1999 .

[32]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[33]  M. Csíkszentmihályi,et al.  Experience Sampling Method: Measuring the Quality of Everyday Life , 2006 .

[34]  Sophia Rabe-Hesketh,et al.  Generalized latent variable models: multilevel, longitudinal, and structural equation models , 2004 .

[35]  G. Kaiser,et al.  Von der Lehrerausbildung in den Beruf – Fachbezogenes Wissen als Voraussetzung für Wahrnehmung, Interpretation und Handeln im Unterricht , 2014 .

[36]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[37]  J. Gustafsson.  Measurement From a Hierarchical Point of View , 2002 .

[38]  Sigrid Blömeke,et al.  Modeling and Measuring Competencies in Higher Education , 2013 .

[39]  Howard Wainer,et al.  Estimating Ability With the Wrong Model , 1987 .

[40]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[41]  Carla Carnaghan,et al.  Competency‐Based Education and Assessment for the Accounting Profession: A Critical Review* , 2003 .

[42]  R. Shavelson,et al.  Research news and Comment: Performance Assessments , 1992 .

[43]  Tim Davey,et al.  Computer-Adaptive Testing for Students with Disabilities: A Review of the Literature. Research Report. ETS RR-11-32. , 2011 .