HLM Behind the Curtain: Unveiling Decisions Behind the Use and Interpretation of HLM in Higher Education Research

Hierarchical linear modeling (HLM) has become increasingly popular in the higher education literature, but there is significant variability in the current approaches to the conducting and reporting of HLM. The field currently lacks a general consensus around important issues such as the number of levels of analysis that are important to include and how much variance should be accounted for at each level in order for the HLM analysis to have practical significance (Dedrick et al., Rev Educ Res 79:69–102, 2009). The purpose of this research is to explore the use of a 3-level HLM model, appropriate contextualizing of results of HLM, and the interpretation of HLM results that resonates with practice. We used an example of a 3-level model from the National Study of Living Learning Programs to highlight the practical issues that arise in the interpretation of HLM within a higher education context.

[1]  Carrie B. Myers.  Union Status and Faculty Job Satisfaction: Contemporary Evidence from the 2004 National Study of Postsecondary Faculty , 2011 .

[2]  The Organizational Context , 1998 .

[3]  Thomas E. Drabek,et al.  The Organizational Context , 1990 .

[4]  John James,et al.  College Peer Groups: Problems and Prospects for Research. , 1967 .

[5]  K. Inkelas,et al.  National Study of Living-Learning Programs: 2007 Report of Findings , 2008 .

[6]  T. Newcomb,et al.  College peer groups : problems and prospects for research , 1967 .

[7]  Megan M. Palmer,et al.  Intercollegiate Athletes and Effective Educational Practices: Winning Combination or Losing Effort? , 2006 .

[8]  Alexander W. Astin,et al.  Multi-Campus Studies of College Impact: Which Statistical Method is Appropriate? , 2009 .

[9]  C. Conrad,et al.  THE IMPACT OF HISTORICALLY BLACK COLLEGES AND UNIVERSITIES ON THE ACADEMIC SUCCESS OF AFRICAN-AMERICAN STUDENTS , 2006 .

[10]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[11]  R. Haveman,et al.  Access to Higher Education: Exploring the Variation in Pell Grant Prevalence among U.S. Colleges and Universities , 2008 .

[12]  Megan M. Palmer,et al.  Women Students at Coeducational and Women's Colleges: How Do Their Experiences Compare? , 2007 .

[13]  Jens-Christian Smeby,et al.  Departmental Contexts and Faculty Research Activity in Norway , 2005 .

[14]  Bradley E. Cox,et al.  Pedagogical Signals of Faculty Approachability: Factors Shaping Faculty–Student Interaction Outside the Classroom , 2010 .

[15]  L. Hedges.  Effect Sizes in Cluster-Randomized Designs , 2007 .

[16]  Cecilia Rios-Aguilar,et al.  Multilevel Analysis in Higher Education Research: A Multidisciplinary Approach , 2011 .

[17]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[18]  Corbin M. Campbell,et al.  Mentors and College Student Leadership Outcomes: The Importance of Position and Process , 2012 .

[19]  Roel Bosker,et al.  Modeled Variance in Two-Level Models , 1994 .

[20]  Douglas D. Ready,et al.  Racial/Ethnic Disparities in Collegiate Cognitive Gains: A Multilevel Analysis of Institutional Influences on Learning and its Equitable Distribution , 2011 .

[21]  Marc P. Johnston.  Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education (2nd ed.) by Alexander W. Astin and Anthony Lising Antonio (review) , 2014 .

[22]  Markus Dresel,et al.  Counseling University Instructors Based on Student Evaluations of Their Teaching Effectiveness: A Multilevel Test of its Effectiveness Under Consideration of Bias and Unfairness Variables , 2011 .

[23]  Reginald S. Lee,et al.  Multilevel Modeling: A Review of Methodological Issues and Applications , 2009 .

[24]  Amaury Nora,et al.  The construct validity of Institutional Commitment: A confirmatory factor analysis , 1993 .

[25]  J. Hox,et al.  Sufficient Sample Sizes for Multilevel Modeling , 2005 .

[26]  Terry T. Ishitani.  The Determinants of Out-Migration Among In-State College Students in the United States , 2011 .

[27]  Rudolf H. Moos,et al.  Evaluating Educational Environments , 1979 .

[28]  George D. Kuh,et al.  Maximizing What Students Get Out of College: Testing a Learning Productivity Model , 2003 .

[29]  John C. Smart,et al.  If and When Money Matters: The Relationships Among Educational Expenditures, Student Engagement and Students’ Learning Outcomes , 2011 .

[30]  Robert K. Toutkoushian,et al.  Changes in the Demand for Public Land-Grant Universities: The Use of Panel Datasets , 2001 .

[31]  A. Astin,et al.  Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. American Council on Education/Macmillan Series on Higher Education. , 1991 .

[32]  Stephen R. Porter.  INSTITUTIONAL STRUCTURES AND STUDENT ENGAGEMENT , 2006 .

[33]  Ann A. O'Connell,et al.  Multilevel modeling of educational data , 2008 .

[34]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[35]  George D. Kuh.  High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter , 2008 .

[36]  Ulrich Frank,et al.  Multilevel Modeling , 2014, Business & Information Systems Engineering.

[37]  John C. Hayek,et al.  Institutional Selectivity and Good Practices in Undergraduate Education: How Strong is the Link? , 2006 .

[38]  J. Schofield,et al.  Sense of Belonging as a Predictor of Intentions to Persist Among African American and White First-Year College Students , 2007 .

[39]  A. Astin.  The changing American college student: implications for educational policy and practice , 1991 .

[40]  N. Denson,et al.  Attitudes and Advocacy: Understanding Faculty Views on Racial/Ethnic Diversity , 2009 .

[41]  Mirjam Moerbeek,et al.  The Consequence of Ignoring a Level of Nesting in Multilevel Analysis , 2004, Multivariate behavioral research.

[42]  Praminda Caleb-Solly.  Person-environment interaction , 2016 .

[43]  Jeannie Brown Leonard,et al.  Examining Sense of Belonging Among First-Year Undergraduates From Different Racial/Ethnic Groups , 2007 .