Understanding and Addressing the Challenges of Assessing College Student Growth in Student Affairs.

Asking college students how much they have learned or grown is a common assessment practice in student affairs and elsewhere. Unfortunately, recent research suggests that these self-reported gains do a very poor job of measuring actual student learning and growth. This paper provides an overview of the psychological process of how students likely respond to such questions and why their responses can be seriously flawed. It also discusses circumstances in which self-reported gains are somewhat more valid and offers concrete suggestions for student affairs professionals and other higher education constituents who seek to accurately measure student outcomes. AUTHOR Nicholas A. Bowman, Ph.D.

[1]  Ernest T. Pascarella.  Using Student Self-Reported Gains To Estimate College Impact: A Cautionary Tale. , 2001 .

[2]  George R. Goethals,et al.  The Perception of Consistency in Attitudes. , 1973 .

[3]  Nicholas A. Bowman.  Can 1st-Year College Students Accurately Report Their Learning and Development? , 2010 .

[4]  R. Sternberg,et al.  What is Intelligence?: Contemporary Viewpoints on its Nature and Definition , 1986 .

[5]  M. Ross,et al.  The Relation between Current Impressions and Memories of Self and Dating Partners , 1987 .

[6]  Gregory C. Wolniak,et al.  First-Generation College Students , 2004 .

[7]  Self-Reported Learning Gains: A Theory and Test of College Student Survey Response , 2013 .

[8]  A. Astin,et al.  Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. American Council on Education/Macmillan Series on Higher Education. , 1991 .

[9]  P. Hill,et al.  Measuring how college affects students: Social desirability and other potential biases in college student self-reported gains , 2011 .

[10]  J. Friedlander,et al.  The meaning of response categories: How often is “occasionally,” “often,” and “very often”? , 1982 .

[11]  Pu-Shih Daniel Chen,et al.  Finding Quality Responses: The Problem of Low-Quality Survey Responses and Its Impact on Accountability Measures , 2011 .

[12]  Guadalupe Anaya,et al.  COLLEGE IMPACT ON STUDENT LEARNING: Comparing the Use of Self-Reported Gains, Standardized Test Scores, and College Grades , 1999 .

[13]  Nicky Bowman,et al.  Can College Students Accurately Assess What Affects Their Learning and Development? , 2011 .

[14]  Trudy W. Banta,et al.  Designing Effective Assessment: Principles and Profiles of Good Practice , 2009 .

[15]  Hunter Gehlbach,et al.  Using the Theory of Satisficing to Evaluate the Quality of Survey Data , 2011, Research in Higher Education.

[16]  John B. Washbush,et al.  Perceptions of Learning in TE Simulations , 1999 .

[17]  G. B. Markus,et al.  Stability and change in political attitudes: Observed, recalled, and “explained” , 1986 .

[18]  Marc P. Johnston.  Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education (2nd ed.) by Alexander W. Astin and Anthony Lising Antonio (review) , 2014 .

[19]  Jon A. Hess,et al.  Is teacher immediacy actually related to student cognitive learning? , 2001 .

[20]  S. Kitayama,et al.  Handbook of Cultural Psychology , 2007 .

[21]  Daniel Gilbert,et al.  Stumbling on Happiness , 2015 .

[22]  John T. Pohlmann,et al.  A Study of the Validity of Self-Reported Measures of Academic Growth. , 1974 .

[23]  J. Krosnick.  Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[24]  Robert L. Leahy,et al.  The Construction of the Self: A Developmental Perspective , 2001, Journal of Cognitive Psychotherapy.

[25]  M. Ross.  Relation of Implicit Theories to the Construction of Personal Histories , 1989 .

[26]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[27]  Susan A. Ariew,et al.  National Survey of Student Engagement , 2003 .

[28]  H. Triandis.  The Self and Social Behavior in Differing Cultural Contexts , 1989 .

[29]  Ernest T. Pascarella,et al.  First-Generation College Students , 2004 .

[30]  Paul D. Umbach,et al.  Nonresponse and Online Student Evaluations of Teaching: Understanding the Influence of Salience, Fatigue, and Academic Environments , 2012 .

[31]  Nicholas A. Bowman.  Examining Systematic Errors in Predictors of College Student Self-Reported Gains. , 2011 .

[32]  Stephen R. Porter.  Do College Student Surveys Have Any Validity? , 2011 .

[33]  Gary R. Pike,et al.  The relationship between self reports of college experiences and achievement test scores , 1995 .

[34]  J. Kristeva,et al.  Strangers to Ourselves , 1996, Canadian Theatre Review.

[35]  L. Rips,et al.  The Psychology of Survey Response , 2000 .

[36]  M. Larsen,et al.  The Psychology of Survey Response , 2002 .

[37]  Woodrow Wilson,et al.  Walvoord, B. E. (2010). Assessment clear and simple: A practical guide for institutions, departments, and general education (2 nd ed.). San Francisco: Jossey-Bass. , 2010 .

[38]  H. Markus,et al.  Culture and the self: Implications for cognition, emotion, and motivation. , 1991 .

[39]  R. Berdie.  Self-Claimed and Tested Knowledge , 1971 .

[40]  Gary R. Pike,et al.  Limitations of using students' self-reports of academic development as proxies for traditional achievement measures , 1995 .

[41]  Kristina N. Bauer,et al.  Self-Assessment of Knowledge: A Cognitive Learning or Affective Measure? , 2010 .

[42]  L. Zwerling,et al.  First-Generation Students: Confronting the Cultural Issues. New Directions for Community Colleges, Number 80. , 1992 .

[43]  Gary R. Pike,et al.  THE CONSTANT ERROR OF THE HALO IN EDUCATIONAL OUTCOMES RESEARCH , 1998 .

[44]  Nicholas A. Bowman,et al.  Validity and limitations of college student self-report data , 2011 .

[45]  Jenny J. Lee.  Religion and College Attendance: Change among Students , 2002 .

[46]  R. Nisbett.  The geography of thought : how Asians and Westerners think differently--and why , 2003 .

[47]  Gary R. Pike,et al.  The relationship between perceived learning and satisfaction with college: An alternative view , 1993 .

[48]  B. Walvoord,et al.  Assessment clear and simple: a practical guide for institutions, departments, and general education / Barbara E. Walvoord , 2010 .

[49]  Effects of Item Order and Response Options in College Student Surveys , 2014 .

[50]  Michael Conway,et al.  Getting what you want by revising what you had. , 1984 .

[51]  Nicholas A. Bowman.  Assessing learning and development among diverse college students , 2010 .

[52]  Timothy D. Wilson,et al.  The halo effect: Evidence for unconscious alteration of judgments. , 1977 .

[53]  The tie that binds: The role of self-reported high school gains in self-reported college gains , 2011 .