Rankings and Reactivity: How Public Measures Recreate Social Worlds1

Recently, there has been a proliferation of measures responding to demands for accountability and transparency. Using the example of media rankings of law schools, this article argues that the methodological concept of reactivity—the idea that people change their behavior in reaction to being evaluated, observed, or measured—offers a useful lens for disclosing how these measures effect change. A framework is proposed for investigating the consequences, both intended and unintended, of public measures. The article first identifies two mechanisms, self‐fulfilling prophecy and commensuration, that induce reactivity and then distinguishes patterns of effects produced by reactivity. This approach demonstrates how these increasingly fateful public measures change expectations and permeate institutions, suggesting why it is important for scholars to investigate the impact of these measures more systematically.

[1]  G. Hildreth.  The child in America: Behavior problems and programs. , 1930 .

[2]  R. Merton.  The Self-Fulfilling Prophecy , 1948 .

[3]  A Problem in Soviet Business Administration , 1956 .

[4]  D. Campbell.  Factors relevant to the validity of experiments in social settings. , 1957, Psychological bulletin.

[5]  M. Orne.  On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. , 1962 .

[6]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[7]  R. Rosenthal,et al.  Pygmalion in the Classroom: Teacher Expectation and Pupils' Intellectual Development , 1968 .

[8]  A. Schutz.  On Phenomenology And Social Relations , 1970 .

[9]  G. Simmel.  On individuality and social forms , 1971 .

[10]  G. Simmel,et al.  On individuality and social forms : selected writings , 1971 .

[11]  De Vera,et al.  U. S. News and World Report シ ニ ミラレル ニホン ノ イメージ , 1976 .

[12]  R. D'amico.  Discipline and Punish: The Birth of the Prison , 1978, Telos.

[13]  P. Magrab.  For the Sake of the Children , 1978 .

[14]  M. Foucault,et al.  Discipline and Punish: The Birth of the Prison. , 1978 .

[15]  G. Simmel.  The Philosophy of Money , 1979 .

[16]  J. Tedeschi.  Impression Management Theory and Social Psychological Research , 1981 .

[17]  D. Campbell,et al.  Nonreactive Measures in the Social Sciences , 1981 .

[18]  Alan E. Kazdin,et al.  Observer effects: Reactivity of direct observation. , 1982 .

[19]  The logic of social science research , 1984 .

[20]  George J. McCall,et al.  Systematic Field Observation , 1984 .

[21]  L. Thevenot,et al.  Rules and implements: investment in forms , 1984 .

[22]  Ronald Edward Zupko,et al.  Measures and Men , 1986 .

[23]  T. Porter.  The Rise of Statistical Thinking, 1820-1900 , 2020 .

[24]  Reactive Risk and Rational Action: Managing Moral Hazard in Insurance Contracts. , 1987 .

[25]  William Alonso,et al.  The Politics of Numbers. , 1990 .

[26]  P. Guttorp,et al.  The Taming of Chance. , 1992 .

[27]  L. Suchman,et al.  Interactional Troubles in Face-to-Face Survey Interviews , 1990 .

[28]  B. Latour.  We Have Never Been Modern , 1991 .

[29]  Mark Gould,et al.  Nuts and bolts for the social sciences , 1991 .

[30]  Organizations , 1992, Restoration & Management Notes.

[31]  Rankings of Undergraduate Education in U.S. News & World Report and Money: Are they any Good? , 1992 .

[32]  J. Urla.  cultural politics in an age of statistics: numbers, nations, and the making of Basque identity , 1993 .

[33]  Michael B. Usher,et al.  Science in action , 1993, Nature.

[34]  S. Corrigan.  For the Sake of the Children , 1994 .

[35]  M. Power.  The Audit Explosion , 1994 .

[36]  Roderick M. Kramer,et al.  Members' Responses to Organizational Identity Threats: Encountering and Countering the Business Week Rankings , 1996 .

[37]  M. Strathern.  ‘Improving ratings’: audit in the British University system , 1997, European Review.

[38]  Daniel Breslau.  Contract Shop Epistemology: Credibility and Problem Construction in Applied Social Science , 1997 .

[39]  M. Stevens,et al.  COMMENSURATION AS A SOCIAL PROCESS , 1998 .

[40]  M. Burawoy.  The Extended Case Method* , 1998 .

[41]  The Durability of Law School Reputation , 1998 .

[42]  W. Espeland.  The Struggle for Water: Politics, Rationality, and Identity in the American Southwest , 1998 .

[43]  J. Fitzmaurice.  Economy and Society , 1998 .

[44]  Ronald G. Ehrenberg,et al.  U.S. News & World Report's College Rankings: Why They Do Matter , 1999 .

[45]  M. Power.  The audit society : rituals of verification , 1999 .

[46]  Jonathon E. Mote,et al.  The laws of the markets , 2000 .

[47]  Marilyn Strathern,et al.  Audit Cultures: Anthropological Studies in Accountability, Ethics and the Academy , 2000 .

[48]  Jane D. Bock.  DOING THE RIGHT THING? , 2000 .

[49]  Linda M. Mcneil,et al.  The Harmful Impact of the TAAS System of Testing in Texas: Beneath the Accountability Rhetoric. , 2000 .

[50]  A. Desrosières.  How real are statistics? Four possible attitudes , 2001 .

[51]  Bonnie Brown,et al.  In Schools We Trust: Creating Communities of Learning in an Era of Testing and Standardization , 2002, J. Educ. Technol. Soc..

[52]  Marshall W. Meyer.  Rethinking Performance Measurement: Beyond the Balanced Scorecard , 2002 .

[53]  M. Lamont,et al.  The Study of Boundaries in the Social Sciences , 2002 .

[54]  Jonathan Rodden,et al.  Strength in Numbers? , 2002 .

[55]  Michèle Lamont,et al.  The Study of Boundaries Across the Social Sciences , 2002 .

[56]  D. MacKenzie,et al.  Constructing a Market, Performing Theory: The Historical Sociology of a Financial Derivatives Exchange1 , 2003, American Journal of Sociology.

[57]  Fabrizio Ferraro,et al.  Economics Language and Assumptions: How Theories Can Become Self-Fulfilling , 2003 .

[58]  P. L. Caron,et al.  What Law Schools Can Learn from Billy Beane and the Oakland Athletics , 2004 .

[59]  Michèle Lamont,et al.  What is Originality in the Humanities and the Social Sciences? , 2004 .

[60]  G. Steinmetz.  Odious Comparisons: Incommensurability, the Case Study, and “Small N's” in Sociology* , 2004 .

[61]  W. Zareba,et al.  The influence of public reporting of outcome data on medical decision making by physicians. , 2005, Archives of internal medicine.

[62]  J.J.H. Bartholomée,et al.  Campell, Donald T. , 2005 .

[63]  Jeffrey Evans Stake.  The Interplay between Law School Rankings, Reputations, and Resource Allocation: Ways Rankings Mislead , 2005 .

[64]  Wendy Nelson Espeland,et al.  Strength in Numbers? The Advantages of Multiple Rankings , 2005 .

[65]  Tony Bush,et al.  Accountability in Education , 2005 .

[66]  Michael Sauder,et al.  Do Rankings Matter? The Effects of U.S. News & World Report Rankings on the Admissions Process of Law Schools , 2006 .

[67]  John Carson,et al.  The Measure of Merit , 2006 .

[68]  Stoyan V. Sgourev,et al.  Peer Capitalism: Parallel Relationships in the U.S. Economy1 , 2006, American Journal of Sociology.

[69]  Michael Sauder.  Third parties and status position: How the characteristics of status systems matter , 2006 .

[70]  Maureen T. Hallinan,et al.  Handbook of the Sociology of Education , 2006 .

[71]  M. Stevens.  Creating a Class: College Admissions and the Education of Elites , 2007 .

[72]  B. Carruthers,et al.  The Recursivity of Law: Global Norm Making and National Lawmaking in the Globalization of Corporate Insolvency Regimes1 , 2007, American Journal of Sociology.

[73]  Managing Elites: Professional Socialization in Law and Business Schools , 2007 .

[74]  Elias L. Khalil,et al.  Donald MacKenzie ’ s An engine , not a camera : how financial models shape markets , 2009 .

[75]  Managing Elites: Professional Socialization in Law and Business Schools By Debra J. Schleef Rowman and Littlefield Publishers. 241 pages. $26.95 (cloth) , 2007 .

[76]  James E. Salzman,et al.  Who’s Number One? , 2009 .

[77]  II ON SOCIOLOGICAL THEORIES OF THE MIDDLE RANGE , 2022 .