Face-to-Face versus Online Course Evaluations: A "Consumer's Guide" to Seven Strategies

The research on student rating scales and other measures of teaching effectiveness in face-to-face (F2F) courses has been accumulating for 90 years. With the burgeoning international development of online and blended/hybrid courses over the past decade, the question of what measures to use has challenged directors of distance education programs. Can the traditional F2F scales already in operation be applied to online courses or do all new scales have to be designed? Despite the increasing number of online courses, attention to their evaluation lags far behind that of F2F courses in terms of available measures, quality of measures, and delivery systems. The salient characteristics of F2F and online courses are compared to determine whether they are really different enough to justify separate scales and evaluation systems. Based on a review of the research and current practices, seven concrete measurement options were generated. They are proffered and critiqued as a state-of-the-art "consumer's guide" to the evaluation of online and blended courses and the faculty who teach them.

[1]  Alvin Y. Wang,et al.  Characteristics of students who enroll and succeed in psychology Web-based classes. , 2000 .

[2]  Arthur W. Bangert,et al.  The Development and Validation of the Student Evaluation of Online Teaching Effectiveness , 2008 .

[3]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[4]  Renato Schibeci,et al.  A framework and checklists for evaluating online learning in higher education , 2005 .

[5]  M. Drouin.  What’s the story on evaluations of online teaching? , 2012 .

[6]  A. Chickering,et al.  Seven Principles for Good Practice in Undergraduate Education , 1987, CORE.

[7]  John A. Schibrowsky,et al.  The Interdependence of the Factors Influencing the Perceived Quality of the Online Learning Experience: A Causal Model , 2007 .

[8]  Paul Chi Hong Lip.  Evaluating Teaching and Learning from Students' Perspectives in Their Classroom through Easy-to-Use Online Surveys , 2014 .

[9]  Ronald A. Berk,et al.  Survey of 12 Strategies to Measure Teaching Effectiveness , 2005 .

[10]  Peter Seldin,et al.  Evaluating faculty performance : a practical guide to assessing teaching, research, and service , 2006 .

[11]  A. Lenhart,et al.  The digital revolution and higher education , 2011 .

[12]  Christina Chang Wei,et al.  2007-08 National Postsecondary Student Aid Study (NPSAS:08): Student Financial Aid Estimates for 2007-08. First Look. NCES 2009-166. , 2009 .

[13]  Arthur Bangert,et al.  Identifying factors underlying the quality of online teaching effectiveness: An exploratory study , 2006, J. Comput. High. Educ..

[14]  M. Freyd.  A Graphic Rating Scale for Teachers , 1923 .

[15]  Neal Strudler,et al.  Development and Validation of an Instrument for Student Evaluation of the Quality of Web-Based Instruction , 2004 .

[16]  C. Hunt.  National strategy for higher education to 2030 , 2011 .

[17]  John Beattie,et al.  Evaluating Instruction in Distance Learning Classes , 2002 .

[18]  Richard Oliver,et al.  Using Course Management Systems to Enhance the Value of Student Evaluations of Teaching , 2005 .

[19]  R. Arreola.  Developing a Comprehensive Faculty Evaluation System: A Handbook for College Faculty and Administrators on Designing and Operating a Comprehensive Faculty Evaluation System , 1995 .

[20]  W. R. Dillon,et al.  Students’ Evaluation of Teaching: Concerns of Item Diagnosticity , 2010 .

[21]  Karen A. Loveland.  Student Evaluation of Teaching (SET) in Web-based Classes: Preliminary Findings and a Call for Further Research , 2007 .

[22]  Charles Harrington,et al.  Online Student Evaluation of Teaching for Distance Education: A Perfect Match? , 2005 .

[23]  Marie Bienkowski,et al.  Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief , 2012 .

[24]  Stephen L. Benton,et al.  An Analysis of IDEA Student Ratings of Instruction Using Paper versus Online Survey Methods 2002-2008 Data , 2010 .

[25]  Kevin B. Wright,et al.  Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services , 2006, J. Comput. Mediat. Commun..

[26]  Mary K. Tallent-Runnels,et al.  Teaching Courses Online: A Review of the Research , 2006 .

[27]  Lesley G. Hathorn,et al.  Evaluation of Online Course Websites: Is Teaching Online a Tug-of-War? , 2010 .

[28]  Lyn M. Shulha,et al.  The Program Evaluation Standards: A Guide for Evaluators and Evaluation Users , 2010 .

[29]  D. Stufflebeam.  The Personnel Evaluation Standards: How to Assess Systems for Evaluating Educators , 1988 .

[30]  The Secret to the "Best" Ratings from Any Evaluation Scale. , 2010 .

[31]  Debbie E. McGhee,et al.  Psychometric Properties of Student Ratings of Instruction in Online and On‐Campus Courses , 2003 .

[32]  John P. Campbell,et al.  Analytics in Higher Education: Establishing a Common Language , 2012 .

[33]  Michael F. Beaudoin.  Online learning as a strategic asset , 2009 .

[34]  M. Kite.  Effective Evaluation of Teaching: A Guide for Faculty and Administrators. , 2012 .

[35]  Daniel Compora.  Current Trends in Distance Education: An Administrative Model. , 2003 .

[36]  D. Mitchell,et al.  Criteria for Assessing Student Satisfaction with Online Courses , 2011 .

[37]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[38]  William E. Cashin,et al.  Student Ratings of Teaching : A Summary of Research and Literature , 2012 .

[39]  I. E. Allen,et al.  Going the Distance: Online Education in the United States, 2011. , 2011 .