Discrimination in the Age of Algorithms

The law forbids discrimination. But the ambiguity of human decision-making often makes it extraordinarily hard for the legal system to know whether anyone has actually discriminated. To understand how algorithms affect discrimination, we must therefore also understand how they affect the problem of detecting discrimination. By one measure, algorithms are fundamentally opaque, not just cognitively but even mathematically. Yet for the task of proving discrimination, processes involving algorithms can provide crucial forms of transparency that are otherwise unavailable. These benefits do not happen automatically. But with appropriate requirements in place, the use of algorithms will make it possible to more easily examine and interrogate the entire decision process, thereby making it far easier to know whether discrimination has occurred. By forcing a new level of specificity, the use of algorithms also highlights, and makes transparent, central tradeoffs among competing values. Algorithms are not only a threat to be regulated; with the right safeguards in place, they have the potential to be a positive force for equity.

[1]  C. Jencks,et al.  The Black-White Test Score Gap. , 1998 .

[2]  Mimi Engel,et al.  New Evidence on Teacher Labor Supply , 2011 .

[3]  Ravi Shroff,et al.  The accuracy, equity, and jurisprudence of criminal risk assessment , 2021, Research Handbook on Big Data Law.

[4]  N. McGlynn.  Thinking fast and slow. , 2014, Australian veterinary journal.

[5]  Susan D. Carle.  A Social Movement History of Title VII Disparate Impact Analysis , 2010 .

[6]  O. J. Harvey,et al.  Intergroup Conflict And Cooperation: The Robbers Cave Experiment , 2013 .

[7]  K. Arrow.  The Theory of Discrimination , 1971 .

[8]  Sebastian Ehrlichmann,et al.  The Economics of Discrimination , 2009 .

[9]  Roland G. Fryer.  Reconciling Results on Racial Differences in Police Shootings , 2018 .

[10]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[11]  Michael L. Rich.  Machine Learning, Automated Suspicion Algorithms, and the Fourth Amendment , 2015 .

[12]  Andrew D. Selbst,et al.  Big Data's Disparate Impact , 2016 .

[13]  Michael Evan Gold.  Griggs' Folly: An Essay on the Theory, Problems, and Origin of the Adverse Impact Definition of Employment Discrimination and a Recommendation for Reform , 1985 .

[14]  Gary S. Becker,et al.  The Economics of Discrimination. , 1972 .

[15]  T. C. Green,et al.  Gender and Job Performance: Evidence from Wall Street , 2007 .

[16]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[17]  John M. Olin.  FREE AT LAST ? JUDICIAL DISCRETION AND RACIAL DISPARITIES IN FEDERAL SENTENCING , 2012 .

[18]  L. Cosmides,et al.  Perceptions of race , 2003, Trends in Cognitive Sciences.

[19]  Yehuda Koren,et al.  Lessons from the Netflix prize challenge , 2007, SKDD.

[20]  Tristin K. Green.  Making Sense of the McDonnell Douglas Framework: Circumstantial Evidence and Proof of Disparate Treatment under Title VII , 1999 .

[21]  A. Greenwald,et al.  Measuring individual differences in implicit cognition: the implicit association test. , 1998, Journal of personality and social psychology.

[22]  Elizabeth Bartholet.  Application of Title VII to Jobs in High Places , 1982 .

[23]  M. Selmi.  Was the Disparate Impact Theory a Mistake , 2006 .

[24]  Yu He,et al.  Statistical Significance of the Netflix Challenge , 2012, 1207.5649.

[25]  Pamela L. Perry.  Two Faces of Disparate Impact Discrimination , 1991 .

[26]  I. Ayres,et al.  The Q-Word as Red Herring: Why Disparate Impact Liability Does Not Induce Hiring Quotas , 1996 .

[27]  C. Lawrence,et al.  Two Views of the River: A Critique of the Liberal Defense of Affirmative Action , 2001 .

[28]  J. Galloway,et al.  The state of the union. , 1970, Lancet.

[29]  Michael Sipser,et al.  Introduction to the Theory of Computation , 1996, SIGA.

[30]  R. Stott,et al.  The World Bank , 2008, Annals of tropical medicine and parasitology.

[31]  Narasimhan Jegadeesh,et al.  Gender and Job Performance: Evidence from Wall Street , 2009 .

[32]  Jennifer L. Doleac,et al.  Does “Ban the Box” Help or Hurt Low-Skilled Workers? Statistical Discrimination and Employment Outcomes When Criminal Histories are Hidden , 2016 .

[33]  Joseph Price,et al.  County-level racial prejudice and the black-white gap in infant health outcomes. , 2017, Social science & medicine.

[34]  D. Greiner.  Causal Inference in Civil Rights Litigation , 2008 .

[35]  Sandy L. Zabell,et al.  Statistical proof of discrimination , 1989 .

[36]  Steven J. Ingels,et al.  National Education Longitudinal Study of 1988. Field Test Report. , 1987 .

[37]  D. Fudenberg,et al.  A Dual Self Model of Impulse Control , 2004, The American economic review.

[38]  Victor Lavy,et al.  On the Origins of Gender Human Capital Gaps: Short and Long Term Consequences of Teachers’ Stereotypical Biases , 2015 .

[39]  Sonja B. Starr,et al.  Ban the Box, Criminal Records, and Statistical Discrimination: A Field Experiment , 2016 .

[40]  Linda Lye.  Title VII's Tangled Tale: The Erosion and Confusion of Disparate Impact and the Business Necessity Defense , 1998 .

[41]  Joshua D. Angrist,et al.  Mostly Harmless Econometrics: An Empiricist's Companion , 2008 .

[42]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[43]  D. Kahneman,et al.  Attention and Effort , 1973 .

[44]  David A. Strauss,et al.  Discriminatory Intent and the Taming of Brown , 1989 .

[45]  T. Sauvage,et al.  An exploratory model for outsourcing-purchasing activities based on a comparative study , 2017 .

[46]  B. Norris.  Multiple Regression Analysis in Title VII Cases: A Structural Approach to Attacks of Missing Factors and “Pre-Act Discrimination” , 1986 .

[47]  E. George,et al.  A Tribute to Charles Stein , 2012, 1203.4664.

[48]  Jessica Kay Flake,et al.  Disproportionate Use of Lethal Force in Policing Is Associated With Regional Racial Biases of Residents , 2018 .

[49]  G. Rutherglen.  Disparate Impact, Discrimination, and the Essentially Contested Concept of Equality , 2006 .

[50]  C. Sunstein,et al.  The Law of Implicit Bias , 2006 .

[51]  D. Pager.  The Mark of a Criminal Record1 , 2003, American Journal of Sociology.

[52]  David A. Strauss,et al.  Racial Discrimination in Employment: The Case for Numerical Standards , 1991 .

[53]  Roland G. Fryer.  Reconciling Results on Racial Differences in Police Shootings , 2018 .

[54]  D. G. Robinson,et al.  Danger Ahead: Risk Assessment and the Future of Bail Reform , 2018 .

[55]  J. Kleinberg,et al.  Prediction Policy Problems. , 2015, The American economic review.

[56]  Richard A. Primus.  Equal Protection and Disparate Impact: Round Three , 2003 .

[57]  Jake Elijah Struebing.  Reconsidering Disparate Impact Under Title VII: Business Necessity as Risk Management , 2016 .

[58]  Michael O. Finkelstein.  The Judicial Reception of Multiple Regression Studies in Race and Sex Discrimination Cases , 1980 .

[59]  A. Siddiqi,et al.  An Empirical Study Analyzing Job Productivity in Toxic Workplace Environments , 2018, International journal of environmental research and public health.

[60]  E. Phelps.  The Statistical Theory of Racism and Sexism , 1972 .

[61]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[62]  J. McCrary,et al.  Controlling Crime: Strategies and Tradeoffs , 2011 .

[63]  Christine Jolls,et al.  Antidiscrimination and Accommodation , 2001 .

[64]  Keeping an eye on recruiter behavior , .

[65]  K A Schulman,et al.  The effect of race and sex on physicians' recommendations for cardiac catheterization. , 1999, The New England journal of medicine.

[66]  David R. Williams,et al.  Discrimination and racial disparities in health: evidence and needed research , 2009, Journal of Behavioral Medicine.

[67]  Crystal S. Yang,et al.  Judicial Politics and Sentencing Decisions , 2018, American Economic Journal: Economic Policy.

[68]  Beth A. Glenn,et al.  Understanding Racial and Ethnic Disparities in Arrest: The Role of Individual, Home, School, and Community Characteristics , 2016, Race and Social Problems.

[69]  J. Ely.  The Constitutionality of Reverse Racial Discrimination , 1974 .

[70]  Amanda Pallais,et al.  Discrimination as a Self-Fulfilling Prophecy: Evidence from French Grocery Stores , 2016 .

[71]  Jonathan Guryan,et al.  Taste�?Based or Statistical Discrimination: The Economics of Discrimination Returns to its Roots , 2013 .

[72]  T. Dee.  Teachers, Race, and Student Achievement in a Randomized Experiment , 2001, Review of Economics and Statistics.

[73]  Cass R. Sunstein,et al.  Why Markets Don't Stop Discrimination , 1991, Social Philosophy and Policy.

[74]  S. Raphael,et al.  Car ownership, employment, and earnings , 2002 .

[75]  O. Ashenfelter,et al.  Discrimination in Labor Markets , 1974 .

[76]  J. Kristeva,et al.  Strangers to Ourselves , 1996, Canadian Theatre Review.

[77]  S. Raphael,et al.  Improving Employment Prospects for Former Prison Inmates: Challenges and Policy , 2010 .

[78]  Deven R. Desai,et al.  Trust But Verify: A Guide to Algorithms and the Law , 2017 .

[79]  Anthony G. Greenwald,et al.  Blindspot: Hidden Biases of Good People , 2013 .

[80]  Henrik Kleven,et al.  Children and Gender Inequality: Evidence from Denmark , 2018, American Economic Journal: Applied Economics.

[81]  E. Langer,et al.  The Mindlessness of Ostensibly Thoughtful Action: The Role of "Placebic" Information in Interpersonal Interaction , 1978 .

[82]  Sandra G. Mayson.  Bias In, Bias Out , 2018 .

[83]  Yavar Bathaee,et al.  The Artificial Intelligence Black Box and the Failure of Intent and Causation , 2018 .

[84]  Jure Leskovec,et al.  Human Decisions and Machine Predictions , 2017, The quarterly journal of economics.

[85]  Philip E. Tetlock,et al.  Implicit Bias and Accountability Systems: What Must Organizations Do to Prevent Discrimination? , 2009 .

[86]  Linda Darling-Hammond,et al.  Unequal Opportunity: Race and Education , 1998 .

[87]  Cary Franklin.  Inventing the 'Traditional Concept' of Sex Discrimination , 2012 .

[88]  T. Schneider,et al.  Using implicit bias training to improve attitudes toward women in STEM , 2014, Social Psychology of Education.

[89]  R. Colker.  ANTI-SUBORDINATION ABOVE ALL : SEX , RACE , AND EQUAL PROTECTION , 2022 .

[90]  Philip E. Tetlock,et al.  TARGET ARTICLE: Attributions of Implicit Prejudice, or "Would Jesse Jackson 'Fail' the Implicit Association Test?" , 2004 .

[91]  Ose,et al.  THE EFFECT OF RACE AND SEX ON PHYSICIANS’ RECOMMENDATIONS FOR CARDIAC CATHETERIZATION , 2000 .

[92]  Miguel A. Mendez,et al.  Presumptions of Discriminatory Motive in Title VII Disparate Treatment Cases , 1980 .

[93]  S. Bagenstos.  The Structural Turn and the Limits of Antidiscrimination Law , 2006 .

[94]  Susan D. Carle.  A New Look at the History of Title VII Disparate Impact Doctrine , 2011 .

[95]  Michael Luca,et al.  Supplemental Appendix for : Productivity and Selection of Human Capital with Machine Learning , 2016 .

[96]  Amanda Pallais,et al.  Discrimination as a Self-Fulfilling Prophecy: Evidence from French Grocery Stores , 2017 .

[97]  Jonathan Guryan,et al.  Prejudice and Wages: An Empirical Assessment of Becker’s The Economics of Discrimination , 2008, Journal of Political Economy.

[98]  P. Brest,et al.  The Supreme Court, 1975 Term , 1976 .

[99]  Mark P. Zanna,et al.  The Nonverbal Mediation of Self-Fulfilling Prophecies in Interracial Interaction. , 1974 .

[100]  Dmitry Taubinsky,et al.  Biases and Implicit Knowledge ∗ , 2012 .

[101]  C. Lawrence,et al.  The Id, the Ego, and Equal Protection: Reckoning with Unconscious Racism , 1987 .

[102]  John J. Donohue,et al.  The Law and Economics of Antidiscrimination Law , 2005 .

[103]  E. Wolff.  The Decline of African-American and Hispanic Wealth Since the Great Recession , 2018 .

[104]  Cass R. Sunstein,et al.  The Anticaste Principle , 1994 .

[105]  Audrey J. Lee.  Unconscious Bias Theory in Employment Discrimination Litigation , 2005 .

[106]  Jann Spiess,et al.  Big Data and Discrimination , 2018 .

[107]  Camille Z. Charles,et al.  The Real Record on Racial Attitudes , 2012 .

[108]  Geoffrey L. Cohen,et al.  Party over policy: The dominating impact of group influence on political beliefs. , 2003, Journal of personality and social psychology.

[109]  A. Wax.  Disparate Impact Realism , 2011 .