Evidence of a Single Second-Order Factor in Student Ratings of Teaching Effectiveness

In student-evaluation research, there has been considerable debate on the number of higher order factors in student ratings of university teaching. Using a distance learning program as the context for research, the study described in this article applied structural equation modeling to investigate the 2nd-order factor structure in student ratings of teaching effectiveness. An a priori model was hypothesized, which consisted of four 1st-order factors and one 2nd-order factor. Data were collected from 2,021 distance students using a multifactor questionnaire. In contrast to all past studies, the presence of a single 2nd-order factor in student ratings was empirically demonstrated through hierarchical confirmatory factor analysis. Implications for future research on the higher order factor structure in student ratings of conventional on-campus courses are also discussed.

[1]  R. Turner.  Good Teaching and Its Contexts. , 1970 .

[2]  B. J. Winer.  Statistical Principles in Experimental Design , 1992 .

[3]  Philip C. Abrami,et al.  Multidimensional students' evaluations of teaching effectiveness: Generalizability of "N = 1" research: Comment on Marsh (1991). , 1991 .

[4]  B. Willis.  Distance Education: A Practical Guide , 1994 .

[5]  Margaret Landstrom.  Evaluating Open and Distance Learning , 1989 .

[6]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[7]  Herbert W. Marsh,et al.  Weighting for the right criteria in the Instructional Development and Effectiveness Assessment (IDEA) system: Global and specific ratings of teaching effectiveness and their relation to course objectives. , 1994 .

[8]  Herbert W. Marsh,et al.  The Hierarchical Structure of Self-Concept and the Application of Hierarchical Confirmatory Factor Analysis , 1987 .

[9]  Liu Zheng-de.  Evaluation of Teachers in Higher Education , 2001 .

[10]  Herbert W. Marsh,et al.  Multidimensional students' evaluations of teaching effectiveness : a test of alternative higher-order structures , 1991 .

[11]  H. H. Remmers,et al.  A factor analysis of the Purdue Rating Scale for Instructors. , 1943 .

[12]  Barbara M. Byrne,et al.  A Primer of LISREL , 1989 .

[13]  Margaret Gorts Morabito.  Foundations of Distance Education. , 1997 .

[14]  P. Frey.  A two-dimensional analysis of student ratings of instruction , 1978 .

[15]  B. Byrne.  A Primer of Lisrel: Basic Applications and Programming for Confirmatory Factor Analytic Models , 1989 .

[16]  Karl L. Zinn,et al.  Dimensions of student evaluations of teaching. , 1964 .

[17]  James A. Kulik,et al.  7: The Evaluation of Teachers in Higher Education , 1975 .

[18]  K. Feldman.  The superior college teacher from the students' view , 1976 .

[19]  Raymond L. Debus,et al.  Effects of internally focused feedback and attributional feedback on enhancement of academic self-concept , 1991 .

[20]  P. Abrami.  Dimensions of Effective College Instruction , 2017 .

[21]  William E. Cashin,et al.  Using global student rating items for summative evaluation. , 1992 .

[22]  David Roberts,et al.  Feedback on assignments , 1996 .

[23]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[24]  P. A. Cohen.  Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies , 1981 .

[25]  Joseph M. Ryan,et al.  The relationship between individual instructional characteristics and the overall assessment of teaching effectiveness across different instructional contexts , 1995 .

[26]  Carl T. Finkbeiner,et al.  Course and Instructor Evaluation: Some Dimensions of a Questionnaire. , 1973 .

[27]  Peter W. Frey,et al.  Student Ratings of Instruction: Validation Research1 , 1975 .

[28]  Edmond Marks,et al.  Student Perceptions of Engineering Instructor Behaviors and Their Relationships to the Evaluation of Instructors and Courses , 1970 .

[29]  Kenneth A. Feldman,et al.  The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies , 1989 .

[30]  Karl G. Jöreskog,et al.  Lisrel 8: Structural Equation Modeling With the Simplis Command Language , 1993 .

[31]  H. Marsh.  A Multidimensional Perspective on Students' Evaluations of Teaching Effectiveness: Reply to Abrami and d'Apollonia (1991). , 1991 .

[32]  Derek Cheung,et al.  Developing a student evaluation instrument for distance teaching , 1998 .

[33]  Jannette Kirkwood,et al.  Students' Anxiety in Learning through Distance Education. , 1994 .

[34]  H. Marsh,et al.  The Factorial Invariance of Student Evaluations of College Teaching , 1984 .

[35]  Keith Stevenson,et al.  Student Perceptions of the Tutor's Role in Distance Learning , 1996 .