Multilevel analysis of the effects of student and course characteristics on satisfaction in undergraduate liberal arts courses

The purpose of this study is to analyze the effects of student and course characteristics on student satisfaction in courses by using a two-level hierarchical linear model (HLM). Based on literature reviews, 13 research variables (1 student gender, 2 student academic year, 3 student major, 4 student reason for taking the course, 5 student level of course participation, 6 student expected grade, 7 student achieved grade, 8 faculty gender, 9 faculty age, 10 faculty status, 11 academic field of the course, 12 class size, and 13 course type) were selected and specified as fixed effects in the analysis model. Data were 57,216 ratings of 1,481 undergraduate liberal arts courses at Seoul National University in 2006. The result of unconditional model analysis revealed that student characteristics’ effects on student satisfaction in courses (within-course effects, 88.1%) were much larger than course characteristics’ effects (between-courses effects, 11.9%). The result of conditional model analysis specifying student and course level predictors revealed that those 12 research variables, with the exception of student gender, had statistically significant effects on student satisfaction in courses. The explained variance was 22.0% in student level, 65.8% in course level, and 27.2% of the total variance.

[1]  Tacksoo Shin.  comparison of three growth modeling techniques in the multilevel analysis of longitudinal academic achievement scores: Latent growth modeling, hierarchical linear modeling, and longitudinal profile analysis via multidimensional scaling , 2007 .

[2]  Paul D. Umbach,et al.  How Do Academic Departments Impact Student Satisfaction? Understanding the Contextual Effects of Departments , 2001 .

[3]  J. C. Smart,et al.  Higher Education: Handbook of Theory and Research , 2001, Higher Education: Handbook of Theory and Research.

[4]  W. McKeachie.  Student ratings: The validity of use. , 1997 .

[5]  Myounghwa Kim.  Validity and Reliability of Lecture Evaluation , 2005 .

[6]  김혜정,et al.  A Comprehensive Study of Korean Students' Evaluations of University Teaching , 2005 .

[7]  Herbert W. Marsh,et al.  Effects of Grading Leniency and Low Workload on Students' Evaluations of Teaching: Popular Myth, Bias, Validity, or Innocent Bystanders? , 2000 .

[8]  J. Kulik.  Student Ratings: Validity, Utility, and Controversy , 2001 .

[9]  Fadia Nasser,et al.  MULTILEVEL ANALYSIS OF THE EFFECTS OF STUDENT AND INSTRUCTOR/COURSE CHARACTERISTICS ON STUDENT RATINGS , 2006 .

[10]  John A. Centra,et al.  Reflective Faculty Evaluation: Enhancing Teaching and Determining Faculty Effectiveness. The Jossey-Bass Higher and Adult Education Series. , 1993 .

[11]  Kwok-fai Ting,et al.  A Multilevel Perspective on Student Ratings of Instruction: Lessons from the Chinese Experience , 2000 .

[12]  Trav D. Johnson.  Online Student Ratings: Will Students Respond?. , 2002 .

[13]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[14]  P. A. Cohen.  Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies , 1981 .

[15]  Kenneth A. Feldman,et al.  Reflections on the Study of Effective College Teaching and Student Ratings: One Continuing Quest and Two Unresolved Issues1 , 1998 .

[16]  Joseph Meng-Chun Chin,et al.  Meta-analysis of transformational school leadership effects on school outcomes in Taiwan and the USA , 2007 .

[17]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[18]  Eric L. Dey.  Working with Low Survey Response Rates: The Efficacy of Weighting Adjustments , 1995 .

[19]  Janet T. Civian,et al.  Student and Course Factors Predicting Satisfaction in Undergraduate Courses at Harvard University. , 1996 .

[20]  H. Marsh,et al.  Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility. , 1997 .

[21]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[22]  Herbert W. Marsh,et al.  Multidimensional students' evaluations of teaching effectiveness : a test of alternative higher-order structures , 1991 .