Are Student Evaluations of Teaching Effectiveness Valid for Measuring Student Learning Outcomes in Business Related Classes? A Neural Network and Bayesian Analyses

In this study we investigate the underlying relational structure between student evaluations of teaching effectiveness (SETEs) and achievement of student learning outcomes in 116 business related courses. Utilizing traditional statistical techniques, a neural network analysis and a Bayesian data reduction and classification algorithm, we find little or no support for the validity of SETEs as a general indicator of teaching effectiveness or student learning. In fact, the underlying structure appears to be non-linear and possibly negatively bimodal where the most effective instructors are within the middle percentiles of student course ratings, while instructors receiving ratings in the top quintile or the bottom quintile are associated with significantly lower levels of student achievement.

[1]  James A. Neal,et al.  A Selective Review of the Validity of Student Ratings of Teaching , 1982 .

[2]  P. Spooren,et al.  Teacher professionalism and student evaluation of teaching: will better teachers receive higher ratings and will better students give higher ratings? , 2006 .

[3]  Iryna Y. Johnson.  Class Size and Student Performance at a Public Research University: A Cross-Classified Model , 2010 .

[4]  Kenneth A. Feldman,et al.  Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers , 1989 .

[5]  Charles A. Burdsal,et al.  Further evidence supporting the validity of both a multidimensional profile and an overall evaluation of teaching effectiveness , 2008 .

[6]  Gabriel Smith,et al.  Students’ Preconceptions of Professors: Benefits and Barriers According to Ethnicity and Gender , 2005 .

[7]  T. Joiner,et al.  Factors influencing teaching evaluations in higher education. , 2006 .

[8]  Jane S. Lopus,et al.  TEACHING TOOLS: SHOULD WE TEACH MICROECONOMIC PRINCIPLES BEFORE MACROECONOMIC PRINCIPLES? , 1995 .

[9]  Kenneth A. Feldman,et al.  The association between student ratings of specific instructional dimensions and student achievement: Refining and extending the synthesis of data from multisection validity studies , 1989 .

[10]  Philip C. Abrami,et al.  The Dimensionality of Ratings and Their Use in Personnel Decisions. , 1990 .

[11]  James Stuart Pounder,et al.  Is student evaluation of teaching worthwhile? An analytical framework for answering the question , 2007 .

[12]  Rubik Atamian,et al.  Teacher Popularity and Teaching Effectiveness: Viewpoint of Accounting Students , 1993 .

[13]  Carol Johnston,et al.  Systematic Influences on Teaching Evaluations: The Case for Caution , 2006 .

[14]  Ernest T. Pascarella,et al.  How College Affects Students: A Third Decade of Research. Volume 2. , 2005 .

[15]  John A. Centra,et al.  Will Teachers Receive Higher Student Evaluations by Giving Higher Grades and Less Course Work? , 2003 .

[16]  L. McCallum.  A meta-analysis of course evaluation data and its use in the tenure decision , 1984 .

[17]  P. A. Cohen.  Validity of Student Ratings in Psychology Courses: A Research Synthesis , 1982 .

[18]  K. Hart,et al.  Criterion validity of the financial skills subscale of the direct assessment of functional status scale , 2009, Psychiatry Research.

[19]  Wilbert J. McKeachie,et al.  Student Ratings of Faculty. , 1969 .

[20]  K. Feldman.  The superior college teacher from the students' view , 1976 .

[21]  Benjamin D. Jee,et al.  Fudging the Numbers: Distributing Chocolate Influences Student Evaluations of an Undergraduate Course , 2007 .

[22]  John Hattie,et al.  The relationship between research and teaching , 2010 .

[23]  John A. Centra,et al.  Research productivity and teaching effectiveness , 1981 .

[24]  F. Costin.  Do Student Ratings of College Teachers Predict Student Achievement? , 1978 .

[25]  John Taylor,et al.  The teaching–research nexus and the importance of context: a comparative study of England and Sweden , 2008 .

[26]  Raymond P. Perry,et al.  The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective , 2007 .

[27]  Charles R. Emery,et al.  Return to academic standards: a critique of student evaluations of teaching effectiveness , 2003 .

[28]  Guy A. Boysen,et al.  Revenge and Student Evaluations of Teaching , 2008 .

[29]  Michael Seid,et al.  PedsQL™ 4.0: Reliability and Validity of the Pediatric Quality of Life Inventory™ Version 4.0 Generic Core Scales in Healthy and Patient Populations , 2001, Medical care.

[30]  Rebecca Bryson.  Teacher Evaluations and Student Learning: A Reexamination. , 1974 .

[31]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[32]  B. P. Smith.  Student Ratings of Teaching Effectiveness: An Analysis of End-of-Course Faculty Evaluations , 2007 .

[33]  Kenneth A. Feldman,et al.  Effective college teaching from the students' and faculty's view: Matched or mismatched priorities? , 1988 .

[34]  Craig S. Galbraith,et al.  Performance analysis of the Bayesian data reduction algorithm , 2009, Int. J. Data Min. Model. Manag..

[35]  R. Buchbinder,et al.  Parsimonious and efficient assessment of health-related quality of life in osteoarthritis research: validation of the Assessment of Quality of Life (AQoL) instrument , 2006, Health and quality of life outcomes.

[36]  K. Feldman.  Identifying Exemplary Teachers and Teaching: Evidence from Student Ratings 1 , 2007 .

[37]  K. C. Johnson,et al.  Hot or Not: Do Professors Perceived as Physically Attractive Receive Higher Student Evaluations? , 2006, The Journal of general psychology.

[38]  Howard K Wachtel,et al.  Student Evaluation of College Teaching Effectiveness: a brief review , 1998 .

[39]  Philip C. Abrami,et al.  The Dimensionality of Student Ratings of Instruction: What We Know and What We Do Not* , 2007 .

[40]  J E Ware,et al.  The Doctor Fox Lecture: a paradigm of educational seduction. , 1973, Journal of medical education.

[41]  John B. Mitchell,et al.  Web-based student evaluations of professors: the relations between perceived quality, easiness and sexiness , 2004 .

[42]  Drew Tiene,et al.  The Impact of Physical Attractiveness, Gender, and Teaching Philosophy on Teacher Evaluations , 1989 .

[43]  Herbert W. Marsh,et al.  Validity of Students' Evaluations of Teaching Effectiveness: Cognitive and Affective Criteria. , 1980 .

[44]  Angela R. Penny.  Changing the Agenda for Research into Students' Views about University Teaching: Four shortcomings of SRT research , 2003 .

[45]  Michael Theall,et al.  Student ratings of instruction : issues for improving practice , 1990 .

[46]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[47]  J. A. Neal,et al.  The Validity and Accuracy of Student Ratings of Instruction: A Reply to Peter A. Cohen , 1983 .

[48]  Burton Rodin,et al.  Student evaluations of teachers. , 1972 .

[49]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[50]  Joachim Zietz,et al.  Containing Cost without Sacrificing Achievement: Some Evidence from College-Level Economics Classes. , 1997 .

[51]  Nathan A. Bowling,et al.  Does the relationship between student ratings of course easiness and course quality vary across schools? The role of school academic rankings , 2008 .

[52]  K. E. Hassan,et al.  Investigating substantive and consequential validity of student ratings of instruction , 2009 .

[53]  John Foster,et al.  Teaching Economics in a Changing Environment: The Case of Introductory Postgraduate Economic Statistics , 2007 .

[54]  N. Ambady,et al.  Half a minute: Predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. , 1993 .

[55]  Peter Willett,et al.  Bayesian classification and feature reduction using uniform Dirichlet priors , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[56]  John Hattie,et al.  The Relationship Between Research and Teaching: A Meta-Analysis , 1996 .

[57]  Gerald R. Adams,et al.  Book Review: Primary Prevention Practices.Martin Bloom. (1996). Thousand Oaks, CA: Sage Publications, Inc. , 1998, Journal of Primary Prevention.

[58]  H. Marsh.  Students’ Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases and Usefulness , 1984 .

[59]  W. Bozeman,et al.  The Value of Student Ratings: Perceptions of Students, Teachers, and Administrators , 2007 .

[60]  John Hattie,et al.  The Relation between Research Productivity and Teaching Effectiveness , 2002 .

[61]  L. Langbein.  Management by Results: Student Evaluation of Faculty Teaching and the Mis-Measurement of Performance. , 2008 .

[62]  P. A. Cohen.  Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies , 1981 .

[63]  M. G. Woodbury,et al.  A prospective, multicenter study to validate use of the PUSH in patients with diabetic, venous, and pressure ulcers. , 2010, Ostomy/wound management.

[64]  William D. Bosshardt,et al.  Comparing Student and Instructor Evaluations of Teaching , 2001 .

[65]  Peter Willett,et al.  Performance considerations for a combined information classification test using Dirichlet priors , 1999, IEEE Trans. Signal Process..

[66]  David M. Shannon,et al.  Student ratings of college teaching: a comparison of faculty and their students , 2010 .

[67]  Brenda Reeb,et al.  Association to Advance Collegiate Schools of Business (AACSB) , 2005 .

[68]  Peter A. Cohen,et al.  Comment on a Selective Review of the Validity of Student Ratings of Teaching , 1983 .

[69]  Wilbert J. McKeachie,et al.  Student Ratings of Faculty: A Reprise. , 1979 .