A multilevel review of artificial intelligence in organizations: Implications for organizational behavior research and practice

The rising use of artificially intelligent (AI) technologies, including generative AI tools, in organizations is undeniable. As these systems become increasingly integrated into organizational practices and processes, understanding their impact on workers' experiences and job designs is critical. However, the ongoing discourse surrounding AI use in the workplace remains divided. Proponents of the technology extol its benefits for enhancing efficiency and productivity, while others voice concerns about the potential harm to human workers. To provide greater clarity on this pressing issue, this article presents a systematic review of empirical research that sheds light on the implications of AI use at work. Organized under five inductively generated themes within a multilevel framework, we uncover individual, group, and organizational factors that shape the interplay between humans and AI. Specifically, the themes are: (1) human–AI collaboration; (2) perceptions of algorithmic and human capabilities; (3) worker attitudes towards AI; (4) AI as a control mechanism in algorithmic management of platform‐based work; and (5) labor market implications of AI use. Our review offers insights into these themes and identifies five pathways for future research. Finally, we provide practical recommendations for organizational leaders seeking to implement AI technologies while prioritizing their employees' well‐being.

[1]  Pauline Schilpzand,et al.  The Vulnerable Workforce: A Call for Research , 2023, Journal of Management.

[2]  E. Brynjolfsson,et al.  Generative AI at Work , 2023, SSRN Electronic Journal.

[3]  Fabiola H. Gerpott,et al.  Employees Adhere More to Unethical Instructions from Human Than AI Supervisors: Complementing Experimental Evidence with Machine Learning , 2023, Journal of Business Ethics.

[4]  Brent B. Clark,et al.  Seeming Ethical Makes You Attractive: Unraveling How Ethical Perceptions of AI in Hiring Impacts Organizational Innovativeness and Attractiveness , 2023, Journal of Business Ethics.

[5]  Paul Formosa,et al.  The Ethical Implications of Artificial Intelligence (AI) For Meaningful Work , 2023, Journal of Business Ethics.

[6]  Amelia Stillwell,et al.  Organizational Justice: Typology, Antecedents and Consequences , 2022, Encyclopedia.

[7]  S. Restubog,et al.  Home and away: How career adaptability and cultural intelligence facilitate international migrant workers' adjustment , 2022, Journal of Vocational Behavior.

[8]  Chih-hai Yang,et al.  How Artificial Intelligence Technology Affects Productivity and Employment: Firm-level Evidence from Taiwan , 2022, Research Policy.

[9]  Brent B. Clark,et al.  Ethical Perceptions of AI in Hiring and Organizational Trust: The Role of Performance Expectancy and Social Influence , 2022, Journal of Business Ethics.

[10]  Mindy K. Shoss,et al.  Artificial Intelligence and the Future of Work: A Functional-Identity Perspective , 2022, Current Directions in Psychological Science.

[11]  P. Dey,et al.  AI-employee collaboration and business performance: Integrating knowledge-based view, socio-technical systems and organisational socialisation framework , 2022, Journal of Business Research.

[12]  Ruth V. Aguilera,et al.  An Eye for Artificial Intelligence: Insights Into the Governance of Artificial Intelligence and Vision for Future Research , 2022, Business & Society.

[13]  Cindy Candrian,et al.  Rise of the machines: Delegating decisions to autonomous AI , 2022, Comput. Hum. Behav..

[14]  D. Acemoglu,et al.  Artificial Intelligence and Jobs: Evidence from Online Vacancies , 2022, Journal of Labor Economics.

[15]  Stefania Innocenti,et al.  Human capital investment and perceived automation risks: Evidence from 16 countries , 2022, Journal of Economic Behavior & Organization.

[16]  E. Anicich Flexing and floundering in the on-demand economy: Narrative identity construction under algorithmic management , 2022, Organizational Behavior and Human Decision Processes.

[17]  G. Morgan,et al.  “Alexa, let’s talk about my productivity”: The impact of digital assistants on work productivity , 2022, Journal of Business Research.

[18]  Florian Pethig,et al.  Biased Humans, (Un)Biased Algorithms? , 2022, Journal of Business Ethics.

[19]  Paul Formosa,et al.  AI Decision Making with Dignity? Contrasting Workers’ Justice Perceptions of Human and AI Decision Making in a Human Resource Management Context , 2022, Information Systems Frontiers.

[20]  Frank M. Fossen,et al.  New digital technologies and heterogeneous wage and employment dynamics in the United States: Evidence from individual-level data , 2022, Technological Forecasting and Social Change.

[21]  Surabhi Verma,et al.  Impact of artificial intelligence-enabled job characteristics and perceived substitution crisis on innovative work behavior of employees from high-tech firms , 2022, Comput. Hum. Behav..

[22]  J. Arias-Pérez,et al.  Ignoring the three-way interaction of digital orientation, Not-invented-here syndrome and employee's artificial intelligence awareness in digital innovation performance: A recipe for failure , 2022, Technological Forecasting and Social Change.

[23]  Manuel F. Gonzalez,et al.  Allying with AI? Reactions toward human-based, AI/ML-based, and augmented hiring processes , 2022, Comput. Hum. Behav..

[24]  P. Leonard,et al.  Engineering the revolution? Imagining the role of new digital technologies in infrastructure work futures , 2021, New Technology, Work and Employment.

[25]  P. Choudhury,et al.  Algorithm-Augmented Work and Domain Experience: The Countervailing Forces of Ability and Aversion , 2021, Organization Science.

[26]  Patrick Mikalef,et al.  How Artificial Intelligence affords digital innovation: A cross-case analysis of Scandinavian companies , 2021 .

[27]  P. Maes,et al.  AI-generated characters for supporting personalized learning and well-being , 2021, Nature Machine Intelligence.

[28]  Luísa Nazareno,et al.  The impact of automation and artificial intelligence on worker well-being , 2021, Technology in Society.

[29]  Emma L. Slade,et al.  SMEs and artificial intelligence (AI): Antecedents and consequences of AI-based B2B practices , 2021, Industrial Marketing Management.

[30]  Philip Meissner,et al.  Managerial overreliance on AI-augmented decision-making processes: How the use of AI-based advisory systems shapes choice behavior in R&D investment decisions , 2021 .

[31]  D. Kwak,et al.  Home-Tutoring Services Assisted with Technology: Investigating the Role of Artificial Intelligence Using a Randomized Field Experiment , 2021, Journal of Marketing Research.

[32]  Nripendra P. Rana,et al.  Understanding AI adoption in manufacturing and production firms using an integrated TAM-TOE model , 2021 .

[33]  U. Rani,et al.  From Flexible Labour to ‘Sticky Labour’: A Tracking Study of Workers in the Food-Delivery Platform Economy of China , 2021, Work, Employment and Society.

[34]  Yogesh K. Dwivedi,et al.  Understanding managers’ attitudes and behavioral intentions towards using artificial intelligence for organizational decision-making , 2021, Technovation.

[35]  James Duggan,et al.  Boundaryless careers and algorithmic constraints in the gig economy , 2021, The International Journal of Human Resource Management.

[36]  S. Schaupp Algorithmic Integration and Precarious (Dis)Obedience: On the Co-Constitution of Migration Regime and Workplace Regime in Digitalised Manufacturing and Logistics , 2021, Work, Employment and Society.

[37]  A. Malik,et al.  A Two‐Wave Cross‐Lagged Study on AI Service Quality: The Moderating Effects of the Job Level and Job Role , 2021, British Journal of Management.

[38]  Marek Hudík,et al.  Beliefs, anxiety and change readiness for artificial intelligence adoption among human resource managers: the moderating role of high-performance work systems , 2021, The International Journal of Human Resource Management.

[39]  Xueming Luo,et al.  The Janus face of artificial intelligence feedback: Deployment versus disclosure effects on employee performance , 2021, Strategic Management Journal.

[40]  Emmanuelle Walkowiak Neurodiversity of the workforce and digital transformation: The case of inclusion of autistic workers at the workplace , 2021, Technological Forecasting and Social Change.

[41]  Soumya K. Ghosh,et al.  Adoption of artificial intelligence-integrated CRM systems in agile organizations in India , 2021, Technological Forecasting and Social Change.

[42]  J. Holm,et al.  The Impact of Artificial Intelligence on Skills at Work in Denmark , 2021, SSRN Electronic Journal.

[43]  A. Bierman,et al.  Über-Alienated: Powerless and Alone in the Gig Economy , 2021, Work and Occupations.

[44]  Heiner Heiland Neither timeless, nor placeless: Control of food delivery gig work via place-based working time regimes , 2021, Human Relations.

[45]  M. Wessels,et al.  Algorithmization of Bureaucratic Organizations: Using a Practice Lens to Study How Context Shapes Predictive Policing Systems , 2021, Public Administration Review.

[46]  C. Deen,et al.  From vocational scholars to social justice advocates: Challenges and opportunities for vocational psychology research on the vulnerable workforce , 2021 .

[47]  A. Varma,et al.  Rebooting employees: upskilling for artificial intelligence in multinational corporations , 2021, The International Journal of Human Resource Management.

[48]  N. Kougiannou,et al.  Breaking the Managerial Silencing of Worker Voice in Platform Capitalism: The Rise of a Food Courier Network , 2021, British Journal of Management.

[49]  Aleksandra Katarzyna Przegalinska,et al.  Cobots in knowledge work , 2021 .

[50]  Yogesh Kumar Dwivedi,et al.  Role of cognitive absorption in building user trust and experience , 2021, Psychology & Marketing.

[51]  Abigail Marks,et al.  Emotional Labour and the Autonomy of Dependent Self-Employed Workers: The Limitations of Digital Managerial Control in the Home Credit Sector , 2021, Work, Employment and Society.

[52]  A. Malik,et al.  May the bots be with you! Delivering HR cost-effectiveness and individualised employee experiences in an MNE , 2020, The International Journal of Human Resource Management.

[53]  Sophia Galière When food‐delivery platform workers consent to algorithmic management: a Foucauldian perspective , 2020 .

[54]  A. Alexiou,et al.  The contingent effect of job automating technology awareness on perceived job insecurity: Exploring the moderating role of organizational culture , 2020 .

[55]  J. Haar,et al.  Technological disruption and employment: The influence on job insecurity and turnover intentions: A multi-country study , 2020 .

[56]  Catherine R. Curtis,et al.  Examining the impact of artificial intelligence on hotel employees through job insecurity perspectives , 2020, International Journal of Hospitality Management.

[57]  Eliane Bucher,et al.  Pacifying the algorithm – Anticipatory compliance in the face of algorithmic management in the gig economy , 2020, Organization.

[58]  Dawid Pieper,et al.  Development, testing and use of data extraction forms in systematic reviews: a review of methodological guidance , 2020, BMC Medical Research Methodology.

[59]  Nathanael J. Fast,et al.  When eliminating bias isn’t fair: Algorithmic reductionism and procedural justice in human resource decisions , 2020 .

[60]  Marco Shaojun Qin,et al.  Artificial Intelligence Coaches for Sales Agents: Caveats and Solutions , 2020, Journal of Marketing.

[61]  Erin E. Makarius,et al.  Rising With the Machines: A Sociotechnical Framework for Bringing Artificial Intelligence Into the Organization , 2020, Journal of Business Research.

[62]  Nicolas Soler,et al.  Automation in Latin America: Are Women at Higher Risk of Losing Their Jobs? , 2020, Technological Forecasting and Social Change.

[63]  Christoph Fuchs,et al.  Preference for Human (vs. Robotic) Labor is Stronger in Symbolic Consumption Contexts , 2020, Journal of Consumer Psychology.

[64]  S. Parker,et al.  How Does the Use of Information Communication Technology Affect Individuals? A Work Design Perspective , 2020, Academy of Management Annals.

[65]  Kohei Kawaguchi,et al.  When Will Workers Follow an Algorithm? A Field Experiment with a Retail Business , 2020, Manag. Sci..

[66]  Giovanna Magnani,et al.  A dynamic perspective on the resilience of firms: A systematic literature review and a framework for future research , 2020 .

[67]  F. Pemer Enacting Professional Service Work in Times of Digitalization and Potential Disruption , 2020, Journal of Service Research.

[68]  G. Perceval,et al.  High Cognitive Flexibility Learners Perform Better in Probabilistic Rule Learning , 2020, Frontiers in Psychology.

[69]  Sangmi Kim,et al.  Designing fair AI for managing employees in organizations: a review, critique, and design agenda , 2020, Hum. Comput. Interact..

[70]  E. MacEachen,et al.  ‘Don't take a poo!': Worker misbehaviour in on‐demand ride‐hail carpooling , 2020 .

[71]  Gudela Grote,et al.  Automation, Algorithms, and Beyond: Why Work Design Matters More Than Ever in a Digital World , 2020 .

[72]  Angèle Christin,et al.  Algorithms at Work: The New Contested Terrain of Control , 2020, Academy of Management Annals.

[73]  Caroline Lloyd,et al.  Rethinking Country Effects: Robotics, AI and Work Futures in Norway and the UK , 2019, New Technology, Work and Employment.

[74]  Jürgen Kai-Uwe Brock,et al.  Demystifying AI: What Digital Transformation Leaders Can Teach You about Realistic Artificial Intelligence , 2019, California Management Review.

[75]  Verena Bader,et al.  Algorithmic decision-making? The user interface and its role for human involvement in decisions supported by artificial intelligence , 2019, Organization.

[76]  Pervaiz Akhtar,et al.  Big Data‐Savvy Teams’ Skills, Big Data‐Driven Actions and Business Performance , 2019, British Journal of Management.

[77]  A. Veen,et al.  Platform-Capital’s ‘App-etite’ for Control: A Labour Process Analysis of Food-Delivery Work in Australia , 2019, Work, Employment and Society.

[78]  Ching-Hua Chuan,et al.  Framing Artificial Intelligence in American Newspapers , 2019, AIES.

[79]  C. Galimberti,et al.  Preventing Technostress Through Positive Technology , 2018, Front. Psychol..

[80]  G. Krogh Artificial Intelligence in Organizations: New Opportunities for Phenomenon-Based Theorizing , 2018, Academy of Management Discoveries.

[81]  Mark Graham,et al.  Good Gig, Bad Gig: Autonomy and Algorithmic Control in the Global Gig Economy , 2018, Work, employment & society : a journal of the British Sociological Association.

[82]  Mohammad Hossein Jarrahi,et al.  Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making , 2018, Business Horizons.

[83]  Dirk Basten,et al.  Approaches for Organizational Learning: A Literature Review , 2018 .

[84]  John Tasioulas,et al.  First Steps Towards an Ethics of Robots and Artificial Intelligence , 2018 .

[85]  Don A. Moore,et al.  Organizational Behavior and Human Decision Processes , 2019 .

[86]  David A Spencer,et al.  Fear and Hope in an Age of Mass Automation: Debating the Future of Work , 2018 .

[87]  J. Halbesleben,et al.  Conservation of Resources in the Organizational Context: The Reality of Resources and Their Consequences , 2018 .

[88]  Min Kyung Lee Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management , 2018, Big Data Soc..

[89]  Rónán Kennedy,et al.  Algorithmic governance: Developing a research agenda through the power of collective intelligence , 2017, Big Data Soc..

[90]  Paul E. Spector,et al.  Best practices in developing, conducting, and evaluating inductive research , 2017 .

[91]  Jarrod M. Haar,et al.  Smart Technology, Artificial Intelligence, Robotics, and Algorithms (STARA): Employees’ perceptions of our future workplace , 2017, Journal of Management & Organization.

[92]  James Scott,et al.  Rise of the machines , 2016, British Medical Journal.

[93]  Travis J. Wiltshire,et al.  Technology as Teammate: Examining the Role of External Cognition in Support of Team Cognitive Processes , 2016, Front. Psychol..

[94]  P. Desmet,et al.  Editorial for “Positive Computing: A New Partnership Between Psychology, Social Sciences and Technologists” , 2016, Psychology of well-being.

[95]  Shoshi Chen,et al.  The commerce and crossover of resources: resource conservation in the service of resilience. , 2015, Stress and health : journal of the International Society for the Investigation of Stress.

[96]  Berkeley J. Dietvorst,et al.  Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err , 2014, Journal of experimental psychology. General.

[97]  Andrew J. Nelson,et al.  'Defining What We Do – All Over Again': Occupational Identity, Technological Change, and the Librarian/Internet-Search Relationship , 2013 .

[98]  J. Mathieu,et al.  The Etiology of the Multilevel Paradigm in Management Research , 2011 .

[99]  Youngho Rhee,et al.  The Role of Organizational and Individual Characteristics in Technology Acceptance , 2009, Int. J. Hum. Comput. Interact..

[100]  David J. Ketchen,et al.  Research on Organizational Configurations: Past Accomplishments and Future Challenges , 2008 .

[101]  J. Mathieu,et al.  Conceptual Framework and Statistical Procedures for Delineating and Testing Multilevel Theories of Homology , 2005 .

[102]  C. D. De Dreu,et al.  Work group diversity and group performance: an integrative model and research agenda. , 2004, The Journal of applied psychology.

[103]  A. Bakker,et al.  The job demands-resources model of burnout. , 2001, The Journal of applied psychology.

[104]  J. Colquitt On the dimensionality of organizational justice: a construct validation of a measure. , 2001, The Journal of applied psychology.

[105]  M. Westman Stress and Strain Crossover , 2001 .

[106]  Wanda J. Orlikowski,et al.  Technological frames: making sense of information technology in organizations , 1994, TOIS.

[107]  Daniel A. Levinthal,et al.  The myopia of learning , 1993 .

[108]  Cheri Ostroff The Effects of Climate and Personal Influences on Individual Behavior and Attitudes in Organizations , 1993 .

[109]  J. March Exploration and exploitation in organizational learning , 1991, STUDI ORGANIZZATIVI.

[110]  E. Fumagalli,et al.  OK computer: Worker perceptions of algorithmic recruitment , 2022, Research Policy.

[111]  Min Kyung Lee,et al.  Employment relationships in algorithmic management: A psychological contract perspective , 2022, Comput. Hum. Behav..

[112]  Christopher McComb,et al.  A cautionary tale about the impact of AI on human design teams , 2021 .

[113]  David De Cremer,et al.  Human-machine collaboration in managerial decision making , 2021, Comput. Hum. Behav..

[114]  Even If AI Can Cure Loneliness, Should It? , 2020, How AI Is Transforming the Organization.

[115]  Paul R. Daugherty,et al.  Collaborative Intelligence: Humans and AI Are Joining Forces , 2018 .

[116]  Michael A. Osborne,et al.  The future of employment: How susceptible are jobs to computerisation? , 2017 .

[117]  Elaine Murphy,et al.  Published by , 1932 .