Evaluating quality of the didactics at university: the opportunities offered by latent class modeling

Purpose Students’ evaluation of teaching quality plays a major role in higher education. Satisfaction is not directly observable, nevertheless it can be measured through multi-item measurement scales. These instruments are extremely useful and their importance requires accurate development and validation procedures. The purpose of this paper is to show how latent class (LC) analysis can improve the procedures for developing and validating a multi-item measurement scale for measuring students’ evaluation of teaching and, at the same time, provide a deeper insight in the phenomenon under investigation. Design/methodology/approach The traditional literature highlights specific protocols along with the statistical instruments to be used for achieving this goal. However, these tools are suited for metric variables but they are adopted even when the nature of the observed variables is different, as it often occurs, since in many cases the items are ordinal. LC analysis takes explicitly into account the ordinal nature of the variables and also the fact that the object of interest is unobservable. Findings The data refer to the questionnaire to evaluate didactics to the students of the University of Padua. Within LC analysis allows an insight of scale properties, such as dimensionality, validity and reliability. Moreover, the results provide a deeper view in the way students use the scale to report satisfaction suggesting to revise the instrument according to the suggestion by the National Agency for University Evaluation. Originality/value The paper gives an original contribution on two sides. On the side of methods, it introduces a more accurate methodology for evaluating scales to measure the students’ satisfaction. On the side of applications, it provides important suggestions to the university management to improve the process of quality of the didactics evaluation.

[1]  L. A. Goodman Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[2]  Gilbert A. Churchill A Paradigm for Developing Better Measures of Marketing Constructs , 1979 .

[3]  Winnie Jenkins,et al.  Adult students' perceptions of good teaching as a function of their conceptions of learning—Part 2. Implications for the evaluation of teaching , 2004 .

[4]  Gary S. Goldstein,et al.  STUDENTS’ AND INSTRUCTORS’ BELIEFS ABOUT EXCELLENT LECTURERS AND DISCUSSION LEADERS , 2006 .

[5]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[6]  Derek Cheung,et al.  Evidence of a Single Second-Order Factor in Student Ratings of Teaching Effectiveness , 2000 .

[7]  Alenoush Saroyan,et al.  Evaluating University Teaching: Time to take stock , 2001 .

[8]  David Kember,et al.  Disciplinary Differences in Student Ratings of Teaching Quality , 2011 .

[9]  Charles A. Burdsal,et al.  The Relative Merits of Different Types of Overall Evaluations of Teaching Effectiveness , 2004 .

[10]  Héctor Grad,et al.  The dimensionality of student ratings of teaching: integration of uni‐ and multidimensional models , 2005 .

[11]  L. M. Aleamoni Student Rating Myths Versus Research Facts from 1924 to 1998 , 1999 .

[12]  Francesca Bassi,et al.  Latent Class Analysis for Marketing Scale Development , 2011 .

[13]  Klaas Sijtsma,et al.  Invited discussion of Cronbach (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334 (26,889 citations in Google Scholar as of 1/1/2016) , 2016 .

[14]  H. Marsh,et al.  Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. , 1997 .

[15]  H. Marsh Students’ Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases and Usefulness , 1984 .

[16]  L. A. Goodman The Analysis of Systems of Qualitative Variables When Some of the Variables Are Unobservable. Part I-A Modified Latent Structure Approach , 1974, American Journal of Sociology.

[17]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[18]  Francisco Zabaleta,et al.  The use and misuse of student evaluations of teaching , 2007 .

[19]  Carmen Pozo-Muñoz,et al.  The 'Ideal Teacher'. Implications for student evaluation of teacher effectiveness , 2000 .

[20]  D. Watson,et al.  Constructing validity: Basic issues in objective scale development , 1995 .

[21]  Kathleen M. T. Collins,et al.  Students’ Perceptions of Characteristics of Effective College Teachers: A Validity Study of a Teaching Evaluation Form Using a Mixed-Methods Analysis , 2007 .

[22]  Lee Harvey,et al.  Assessing Quality in Higher Education: a transbinary research project , 1993 .

[23]  Charles A. Burdsal,et al.  Further evidence supporting the validity of both a multidimensional profile and an overall evaluation of teaching effectiveness , 2008 .

[24]  Vernon D. Miller,et al.  The Influence of Student Sex and Instructor Sex on Student Ratings of Instructors: Results from a College of Communication , 2007 .

[25]  Herbert W. Marsh,et al.  Multiple dimensions of university teacher self-concept , 2000 .

[26]  F. Bassi Experiential Goods and Customer Satisfaction: An Application to Films , 2010 .

[27]  P. Spooren,et al.  On the Validity of Student Evaluation of Teaching , 2013 .

[28]  P. Spooren,et al.  A revalidation of the SET37 questionnaire for student evaluations of teaching , 2009 .

[29]  H. Rindermann,et al.  Generalizability of Multidimensional Student Ratings of University Instruction Across Courses and Teachers , 2001 .

[30]  Pieter Spooren,et al.  On the credibility of the judge: A cross-classified multilevel analysis on students’ evaluation of teaching , 2010 .

[31]  Paul F. Lazarsfeld,et al.  Latent Structure Analysis. , 1969 .

[32]  Tanya N. Beran,et al.  Ratings of university teacher instruction: how much do student and course characteristics really matter? , 2005 .

[33]  Dennis E. Clayson,et al.  Student Evaluations of Teaching: Are They Related to What Students Learn? , 2009 .

[34]  Jay Magidson,et al.  Latent Class Factor and Cluster Models, Bi-Plots, and Related Graphical Displays , 2001 .

[35]  F. Betoret,et al.  Evaluation of the University Teaching/Learning Process for the Improvement of Quality in Higher Education , 2003 .