Care and Coordination in Algorithmic Systems: An Economies of Worth Approach

Algorithmic decision-making has permeated health and care domains (e.g., automated diagnoses, fall detection, caregiver staffing). Researchers have raised concerns about how these algorithms are built and how they shape fair and ethical care practices. To investigate algorithm development and understand its impact on people who provide and coordinate care, we conducted a case study of a U.S.-based senior care network and platform. We interviewed 14 technologists, 9 paid caregivers, and 7 care coordinators to explore their interactions with the platform’s algorithms. We find that technologists draw on a multitude of moral frameworks to navigate complex and contradictory demands and expectations. Despite technologists’ espoused commitments to fairness, accountability, and transparency, the platform reassembles problematic aspects of care labor. By analyzing how technologists justify their work, the problems that they claim to solve, the solutions they present, and caregivers’ and coordinators’ experiences, we advance fairness research that focuses on agency and power asymmetries in algorithmic platforms. We (1) make an empirical contribution, revealing tensions when developing and implementing algorithms and (2) provide insight into the social processes that reproduce power asymmetries in algorithmic decision-making.

[1]  Mona Sloane,et al.  German AI Start-Ups and “AI Ethics”: Using A Social Practice Lens for Assessing and Implementing Socio-Technical Innovation , 2022, FAccT.

[2]  Tina J. Park,et al.  Automating Care: Online Food Delivery Work During the CoVID-19 Crisis in India , 2022, FAccT.

[3]  Nithya Sambasivan,et al.  Whose AI Dream? In search of the aspiration in data annotation. , 2022, CHI.

[4]  A. Cooper,et al.  Making the Unaccountable Internet: The Changing Meaning of Accounting in the Early ARPANET , 2022, FAccT.

[5]  Hanna M. Wallach,et al.  Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support , 2021, Proc. ACM Hum. Comput. Interact..

[6]  L. Dencik,et al.  Regimes of justification in the datafied workplace: The case of hiring , 2021, New Media & Society.

[7]  Anne Marie Piper,et al.  The Future of Care Work: Towards a Radical Politics of Care in CSCW Research and Practice , 2021, CSCW Companion.

[8]  D. Goodridge,et al.  “The system is well intentioned, but complicated and fallible” interviews with caregivers and decision makers about palliative care in Canada , 2021, BMC Palliative Care.

[9]  J. Burrell,et al.  The Society of Algorithms , 2021, Annual Review of Sociology.

[10]  Roi Livne Toward a sociology of finitude: life, death, and the question of limits , 2021, Theory and Society.

[11]  A. Hanna,et al.  Documenting Computer Vision Datasets: An Invitation to Reflexive Data Practices , 2021, FAccT.

[12]  D. Carr Values at the End of Life: The Logic of Palliative Care , 2020, Contemporary Sociology: A Journal of Reviews.

[13]  M. C. Elish,et al.  Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts , 2020, FAccT.

[14]  Milagros Miceli,et al.  Between Subjectivity and Imposition , 2020, Proc. ACM Hum. Comput. Interact..

[15]  Aditya Bhaskara,et al.  Fair Clustering via Equitable Group Representations , 2020, FAccT.

[16]  Caitlin Lustig,et al.  How We've Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis , 2020, Proc. ACM Hum. Comput. Interact..

[17]  Guillermina Altomonte Exploiting Ambiguity: A Moral Polysemy Approach to Variation in Economic Practices , 2020 .

[18]  Cameron MacDonald 13. Ethnic Logics Race and Ethnicity in Nanny Employment , 2019, Caring on the Clock.

[19]  Hanna M. Wallach,et al.  Measurement and Fairness , 2019, FAccT.

[20]  V. Dubal,et al.  An Uber Ambivalence: Employee Status, Worker Perspectives, & Regulation in the Gig Economy , 2019, SSRN Electronic Journal.

[21]  Emanuel Moss,et al.  Owning Ethics: Corporate Logics, Silicon Valley, and the Institutionalization of Ethics , 2019 .

[22]  Malte Ziewitz,et al.  Rethinking gaming: The ethical work of optimization in web search engines , 2019, Social studies of science.

[23]  Gerardo Patriotta,et al.  Working for an Algorithm: Power Asymmetries and Agency in Online Work Settings , 2019, Administrative Science Quarterly.

[24]  Eric Gilbert,et al.  User Attitudes towards Algorithmic Opacity and Transparency in Online Reviewing Platforms , 2019, CHI.

[25]  Hannah Lebovits Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor , 2018, Public Integrity.

[26]  Shreeharsh Kelkar,et al.  Engineering a platform: The construction of interfaces, users, organizational roles, and the division of labor , 2018, New Media Soc..

[27]  Julia Ticona,et al.  Trusted strangers: Carework platforms’ cultural entrepreneurship in the on-demand economy , 2018, New Media Soc..

[28]  J. Jensen Justice in reality: overcoming moral relativism in Luc Boltanski’s pragmatic sociology of critique , 2018 .

[29]  James McMaster Matters of care: speculative ethics in more than human worlds , 2017 .

[30]  Benjamin Shestakofsky,et al.  Working Algorithms: Software Automation and the Future of Work , 2017 .

[31]  P. Doty Private Pay Home Care: New Models of Access and Service Delivery , 2017 .

[32]  J. Yolton,et al.  The Moral Background , 2017 .

[33]  Daniel Cockayne,et al.  Sharing and neoliberal discourse: The economic function of sharing in the digital on-demand economy , 2016 .

[34]  Ana Viseu,et al.  The politics of care in technoscience , 2015, Social studies of science.

[35]  Michelle Murphy,et al.  Unsettling care: Troubling transnational itineraries of care in feminist health practices , 2015, Social studies of science.

[36]  Lilly Irani,et al.  The cultural work of microwork , 2015, New Media Soc..

[37]  Laura A. Dabbish,et al.  Working with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers , 2015, CHI.

[38]  Frank A. Pasquale The Black Box Society: The Secret Algorithms That Control Money and Information , 2015 .

[39]  K. K. Cetina,et al.  Unscrewing the big Leviathan: how actors macro- structure reality and how sociologists help them to do so , 2014 .

[40]  Cameron Tonkinwise,et al.  To Save Everything, Click Here: The Folly of Technological Solutionism , 2014 .

[41]  Annemarie Mol,et al.  What Is a Good Tomato? A Case of Valuing in Practice , 2013 .

[42]  Stephen Vaisey,et al.  The New Sociology of Morality , 2013 .

[43]  Allison J. Pugh What good are interviews for thinking about culture? Demystifying interpretive analysis , 2013 .

[44]  Meika Loe,et al.  Aging Our Way: Lessons for Living from 85 and Beyond , 2011 .

[45]  Jens Beckert,et al.  The Worth of Goods: Valuation and Pricing in the Economy , 2011 .

[46]  Mauro Basaure An interview with Luc Boltanski: Criticism and the expansion of knowledge , 2011 .

[47]  María Puig de la Bellacasa Matters of care in technoscience: Assembling neglected things , 2011, Social studies of science.

[48]  E. N. Glenn Forced to Care: Coercion and Caregiving in America , 2010 .

[49]  K. England Home, Work and the Shifting Geographies of Care , 2010 .

[50]  Sarah L. Quinn The Transformation of Morals in Markets: Death, Benefits, and the Exchange of Life Insurance Policies1 , 2008, American Journal of Sociology.

[51]  Kieran Healy,et al.  Moral Views of Market Society , 2007 .

[52]  P. England,et al.  Wages of Virtue: The Relative Pay of Care Work , 2002 .

[53]  Luc Boltanski,et al.  The Sociology of Critical Capacity , 1999 .

[54]  N. Milio Circles of Care: Work and Identity in Women's Lives , 1991 .

[55]  Luc Boltanski,et al.  On Justification , 2021 .

[56]  C. Estes The Aging Enterprise: In Whose Interests? , 1986, International journal of health services : planning, administration, evaluation.

[57]  W. Powell,et al.  The iron cage revisited institutional isomorphism and collective rationality in organizational fields , 1983 .

[58]  M. Lawton,et al.  Assessment of older people: self-maintaining and instrumental activities of daily living. , 1969, The Gerontologist.

[59]  S. Katz,et al.  Progress in development of the index of ADL. , 1970, The Gerontologist.

[60]  Julia Stoyanovich,et al.  Causal Intersectionality and Fair Ranking , 2021, FORC.

[61]  Francis Lee Enacting the Pandemic: Analyzing Agency, Opacity, and Power in Algorithmic Assemblages , 2020 .

[62]  G. J. Mellenbergh Nonprobability Sampling , 2019, SAGE Research Methods Foundations.

[63]  David Stark,et al.  The Sense of Dissonance: Accounts of Worth in Economic Life , 2009 .

[64]  Luc Boltanski,et al.  The New Spirit of Capitalism , 2005 .

[65]  L. Thevenot,et al.  Rethinking comparative cultural sociology: Forms of valuing nature: arguments and modes of justification in French and American environmental disputes , 2000 .

[66]  E. N. Glenn Creating a Caring Society , 2000 .

[67]  C. Estes,et al.  Privatization, system membership, and access to home health care for the elderly. , 1994, The Milbank quarterly.