Machine Learning and Human Capital Complementarities: Experimental Evidence on Bias Mitigation

The use of machine learning (ML) for productivity in the knowledge economy requires considerations of important biases that may arise from ML predictions. We define a new source of bias related to incompleteness in real time inputs, which may result from strategic behavior by agents. We theorize that domain expertise of users can complement ML by mitigating this bias. Our observational and experimental analyses in the patent examination context support this conjecture. In the face of “input incompleteness,” we find ML is biased towards finding prior art textually similar to focal claims and domain expertise is needed to find the most relevant prior art. We also document the importance of vintage-specific skills, and discuss the implications for artificial intelligence and strategic management of human capital.

[1]  Iyad Rahwan,et al.  Toward understanding the impact of artificial intelligence on labor , 2019, Proceedings of the National Academy of Sciences.

[2]  Ritu Agarwal,et al.  The Role of Individual and Organizational Expertise in the Adoption of New Practices , 2019, Organ. Sci..

[3]  Catherine Tucker,et al.  Economics, Fairness and Algorithmic Bias , 2019, SSRN Electronic Journal.

[4]  Rui Zhang,et al.  Detecting Poisoning Attacks on Machine Learning in IoT Environments , 2018, 2018 IEEE International Congress on Internet of Things (ICIOT).

[5]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[6]  Jure Leskovec,et al.  Human Decisions and Machine Predictions , 2017, The quarterly journal of economics.

[7]  Florenta Teodoridis Understanding Team Knowledge Production: The Interrelated Roles of Technology and Expertise , 2016, Manag. Sci..

[8]  Jure Leskovec,et al.  The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables , 2017, KDD.

[9]  Michael Chui,et al.  Artificial intelligence: the next digital frontier? , 2017 .

[10]  T. Khanna,et al.  The Future of Patent Examination at the USPTO , 2017 .

[11]  William Welser,et al.  An Intelligence in Our Image: The Risks of Bias and Errors in Artificial Intelligence , 2017 .

[12]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[13]  Elkin Olaguer Pérez Sánchez,et al.  Measurement of potential absorption capacity in Colombia's innovative companies , 2017 .

[14]  James E. Bessen,et al.  How Computer Automation Affects Occupations: Technology, Jobs, and Skills , 2016 .

[15]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[16]  Joseph Wolf,et al.  User Interface for Customizing Patents Search: An Exploratory Study , 2016, HCI.

[17]  Ron Adner,et al.  Innovation ecosystems and the pace of substitution: Re‐examining technology S‐curves , 2016 .

[18]  C. Winick The Diffusion of an Innovation Among Physicians , 2016 .

[19]  David H. Autor,et al.  Why Are There Still So Many Jobs? The History and Future of Workplace , 2015 .

[20]  J. Sachs,et al.  Robots are Us: Some Economics of Human Replacement , 2015 .

[21]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[22]  Alfonso Gambardella,et al.  Strategic incentives to human capital , 2015 .

[23]  Xiaojing Dong,et al.  The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies , 2014 .

[24]  David Autor,et al.  Polanyi&Apos;S Paradox and the Shape of Employment Growth , 2014 .

[25]  Michael Marien,et al.  Book Review: The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies , 2014 .

[26]  GuptaAlok,et al.  Human Capital Investments and Employee Performance , 2013 .

[27]  Ryan Lampe Strategic Citation , 2010, Review of Economics and Statistics.

[28]  Mark A. Lemley,et al.  Examiner Characteristics and Patent Office Outcomes , 2009, Review of Economics and Statistics.

[29]  J. V. Reenen,et al.  Americans Do IT Better: US Multinationals and the Productivity Miracle Please share how this access benefits you. Your story matters , 2012 .

[30]  Jan van Dalen,et al.  The role of deliberate practice in the acquisition of clinical skills , 2011, BMC medical education.

[31]  Christopher M. Snyder,et al.  Resume Padding Among Economists , 2011 .

[32]  Dietmar Harhoff,et al.  Harmonizing and Combining Large Datasets - an Application to Firm-Level Patent and Accounting Data , 2010 .

[33]  Suzan Verberne,et al.  Quantifying the Challenges in Parsing Patent Claims , 2010, PaIR 2010.

[34]  Eva D'hondt,et al.  Lexical issues of a syntactic approach to interactive patent retrieval , 2009 .

[35]  Daniel C. Snow,et al.  'Old' Technology Responses to 'New' Technology Threats: Demand Heterogeneity and Graceful Technology Retreats , 2009 .

[36]  Boyan Jovanovic The Technology Cycle and Inequality , 2009 .

[37]  Constance E. Helfat,et al.  Strategic Renewal of Organizations , 2009, Organ. Sci..

[38]  John Van Reenen,et al.  Americans Do I.T. Better: US Multinationals and the Productivity Miracle , 2007 .

[39]  M. Gittelman,et al.  Patent Citations as a Measure of Knowledge Flows: The Influence of Examiner Citations , 2006, The Review of Economics and Statistics.

[40]  Scott A. Turk,et al.  The Proper Method for Using Dictionaries to Construe Patent Claims , 2006 .

[41]  Jeffrey H. Dyer,et al.  Human capital and learning as a source of sustainable competitive advantage , 2004 .

[42]  Bianca Zadrozny,et al.  Learning and evaluating classifiers under sample selection bias , 2004, ICML.

[43]  E. Lazear Balanced Skills and Entrepreneurship , 2004 .

[44]  Michael Waldman,et al.  Task-Specific Human Capital , 2004 .

[45]  Gary P. Pisano,et al.  Learning How and Learning What: Effects of Tacit and Codified Knowledge on Performance Improvement Following Technology Adoption , 2003, Decis. Sci..

[46]  P. Thompson,et al.  Learning from Experience and Learning from Others: An Exploration of Learning and Spillovers in Wartime Shipbuilding , 2001 .

[47]  Sandra E. Black,et al.  How to Compete: The Impact of Workplace Practices and Information Technology on Productivity , 1997, Review of Economics and Statistics.

[48]  Mark A. Lemley,et al.  Who's Patenting What? An Empirical Exploration of Patent Prosecution , 2000 .

[49]  Ritu Agarwal,et al.  Are Individual Differences Germane to the Acceptance of New Information Technologies , 1999 .

[50]  Stephen Machin,et al.  Implications of Skill-Biased Technological Change , 1998 .

[51]  Constance E. Helfat,et al.  Specificity of CEO human capital and compensation , 1997 .

[52]  J. Bound,et al.  IMPLICATIONS OF SKILL-BIASED TECHNOLOGICAL CHANGE: INTERNATIONAL EVIDENCE* ELI BERMAN , 1998 .

[53]  Clayton M. Christensen Patterns in the evolution of product competition , 1997 .

[54]  Raymond J. Mooney,et al.  Comparative Experiments on Disambiguating Word Senses: An Illustration of the Role of Bias in Machine Learning , 1996, EMNLP.

[55]  Thomas G. Dietterich,et al.  Error-Correcting Output Coding Corrects Bias and Variance , 1995, ICML.

[56]  P. David,et al.  Toward a new economics of science , 1994 .

[57]  Robert J. Crutcher,et al.  The role of deliberate practice in the acquisition of expert performance. , 1993 .

[58]  Varadarajan V. Chari,et al.  Vintage Human Capital, Growth, and the Diffusion of New Technology , 1991, Journal of Political Economy.

[59]  Ann P. Bartel,et al.  Productivity Gains from the Implementation of Employee Training Programs , 1991 .

[60]  Constance E. Helfat,et al.  Managerial Resources and Rents , 1991 .

[61]  H. Simon,et al.  Bounded Rationality and Organizational Learning , 1991 .

[62]  M. Tushman,et al.  Technological Discontinuities and Dominant Designs: A Cyclical Model of Technological Change , 1990 .

[63]  Herbert A. Simon,et al.  THE MIND'S EYE IN CHESS , 1988 .

[64]  David A. Hounshell,et al.  From the American System to Mass Production 1800–1932: The Development of Manufacturing Technology in the United States by David A. Hounshell (review) , 2023 .

[65]  Sherwin Rosen,et al.  Specialization and Human Capital , 1983, Journal of Labor Economics.

[66]  George A. Akerlof Labor Contracts as Partial Gift Exchange , 1982 .

[67]  H. Simon,et al.  Perception in chess , 1973 .

[68]  G. Becker,et al.  Investment in Human Capital: A Theoretical Analysis , 1962, Journal of Political Economy.