A Picture is Worth a Collaboration: Accumulating Design Knowledge for Computer-Vision-based Hybrid Intelligence Systems

Computer vision (CV) techniques try to mimic human capabilities of visual perception to support laborintensive and time-consuming tasks like the recognition and localization of critical objects. Nowadays, CV increasingly relies on artificial intelligence (AI) to automatically extract useful information from images that can be utilized for decision support and business process automation. However, the focus of extant research is often exclusively on technical aspects when designing AI-based CV systems while neglecting socio-technical facets, such as trust, control, and autonomy. For this purpose, we consider the design of such systems from a hybrid intelligence (HI) perspective and aim to derive prescriptive design knowledge for CV-based HI systems. We apply a reflective, practice-inspired design science approach and accumulate design knowledge from six comprehensive CV projects. As a result, we identify four design-related mechanisms (i.e., automation, signaling, modification, and collaboration) that inform our derived meta-requirements and design principles. This can serve as a basis for further socio-technical research on CV-based HI systems.

[1]  Joseph P. Simmons,et al.  Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them , 2016, Manag. Sci..

[2]  Alexander Benlian,et al.  Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn , 2020, Business & Information Systems Engineering.

[3]  Christoph Kelp,et al.  Trustworthy artificial intelligence , 2023, Asian Journal of Philosophy.

[4]  Sorin Grigorescu,et al.  A Survey of Deep Learning Techniques for Autonomous Driving , 2020, J. Field Robotics.

[5]  Niklas Kühl,et al.  Utilizing Active Machine Learning for Quality Assurance: A Case Study of Virtual Car Renderings in the Automotive Industry , 2022, Proceedings of the Annual Hawaii International Conference on System Sciences.

[6]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[7]  Ankita Pramanik,et al.  Deep Learning Techniques—R-CNN to Mask R-CNN: A Survey , 2019, Computational Intelligence in Pattern Recognition.

[8]  K. Eisenhardt Agency Theory: An Assessment and Review , 1989 .

[9]  D. Lehmann,et al.  Task-Dependent Algorithm Aversion , 2019, Journal of Marketing Research.

[10]  Adrian Hilton,et al.  Computer vision for sports: Current applications and research topics , 2017, Comput. Vis. Image Underst..

[11]  Antske Fokkens,et al.  A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence , 2020, Computer.

[12]  Nanning Zheng,et al.  Hybrid-augmented intelligence: collaboration and cognition , 2017, Frontiers of Information Technology & Electronic Engineering.

[13]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[14]  Matthias Griebel,et al.  A Picture is Worth More than a Thousand Purchases: Designing an Image-based Fashion Curation System , 2019, ECIS.

[15]  Niklas Kühl,et al.  Towards Leveraging End-of-Life Tools as an Asset: Value Co-Creation based on Deep Learning in the Machining Industry , 2020, HICSS.

[16]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[17]  Juhani Iivari,et al.  Distinguishing and contrasting two strategies for design science research , 2015, Eur. J. Inf. Syst..

[18]  Nikhil Ketkar,et al.  Deep Learning with Python , 2017 .

[19]  Shirley Gregor,et al.  Research Perspectives: The Anatomy of a Design Principle , 2020, J. Assoc. Inf. Syst..

[20]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[21]  Parikshit Ram,et al.  Human-AI Collaboration in Data Science , 2019, Proc. ACM Hum. Comput. Interact..

[22]  Alfred Benedikt Brendel,et al.  Smart Infrastructure Monitoring: Development of a Decision Support System for Vision-Based Road Crack Detection , 2018, ICIS.

[23]  Alexander Maedche,et al.  Designing a Requirement Mining System , 2015, J. Assoc. Inf. Syst..

[24]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[25]  Berkeley J. Dietvorst,et al.  Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err , 2014, Journal of experimental psychology. General.

[26]  Dominik Dellermann,et al.  Hybrid Intelligence , 2019, Business & Information Systems Engineering.

[27]  Hichem Snoussi,et al.  A fast and robust convolutional neural network-based defect detection model in product quality control , 2017, The International Journal of Advanced Manufacturing Technology.

[28]  Taeyoung Lee,et al.  Us vs. Them: Understanding Artificial Intelligence Technophobia over the Google DeepMind Challenge Match , 2017, CHI.

[29]  Philip E. T. Lewis,et al.  Research Methods for Business Students , 2006 .

[30]  Adrian Hofmann,et al.  A taxonomy and archetypes of smart services for smart living , 2020, Electronic Markets.

[31]  Subbarao Kambhampati,et al.  Algorithms for the Greater Good ! On Mental Modeling and Acceptable Symbiosis in Human-AI Collaboration Tathagata Chakraborti , 2018 .

[32]  Kai Heinrich,et al.  Everything counts: a Taxonomy of Deep Learning Approaches for Object Counting , 2019, ECIS.

[33]  Patrick Zschech,et al.  Mit Computer Vision zur automatisierten Qualitätssicherung in der industriellen Fertigung: Eine Fallstudie zur Klassifizierung von Fehlern in Solarzellen mittels Elektrolumineszenz-Bildern , 2020, HMD Praxis der Wirtschaftsinformatik.

[34]  David S. Melnick,et al.  International evaluation of an AI system for breast cancer screening , 2020, Nature.

[35]  Mon-Chu Chen,et al.  Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning , 2019, CHI.

[36]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[37]  Boris Otto,et al.  Towards a Method for Design Principle Development in Information Systems , 2020, DESRIST.

[38]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[39]  Graham Pervan,et al.  Exploring the Links Between Technology Acceptance and Use and the Attainment of Individual and Organizational Goals: A Case Study in the Community Health Sector , 2005, AMCIS.

[40]  A SURVEY ON DEEP LEARNING TECHNIQUES , 2020, Strad Research.

[41]  Kai Heinrich,et al.  Objekterkennung im Weinanbau – Eine Fallstudie zur Unterstützung von Winzertätigkeiten mithilfe von Deep Learning , 2019, HMD Praxis der Wirtschaftsinformatik.

[42]  Zhiming Luo,et al.  An Ensemble Deep Learning Method for Vehicle Type Classification on Visual Traffic Surveillance Sensors , 2017, IEEE Access.

[43]  Roberto Cipolla,et al.  Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding , 2015, BMVC.

[44]  Norbert Schmitz,et al.  Hybrid Teams: Flexible Collaboration Between Humans, Robots and Virtual Agents , 2016, MATES.

[45]  Robert O. Briggs,et al.  Machines as teammates: A research agenda on AI in team collaboration , 2020, Inf. Manag..

[46]  Karim Jebari,et al.  Artificial superintelligence and its limits: why AlphaZero cannot become a general agent , 2020, AI & SOCIETY.

[47]  Stuart J. Russell,et al.  Rationality and Intelligence: A Brief Update , 2013, PT-AI.

[48]  Jan Marco Leimeister,et al.  Design principles for a hybrid intelligence decision support system for business model validation , 2018, Electronic Markets.

[49]  J. Cherrie,et al.  Machine Learning and Deep Learning , 2019, International Journal of Innovative Technology and Exploring Engineering.

[50]  Roman Beck,et al.  Theory-generating design science research , 2013, Inf. Syst. Frontiers.

[51]  Loren G. Terveen,et al.  Overview of human-computer collaboration , 1995, Knowl. Based Syst..

[52]  D. Morgan Focus groups for qualitative research. , 1988, Hospital guest relations report.

[53]  Matthias Griebel,et al.  Applied image recognition: guidelines for using deep learning models in practice , 2019, Wirtschaftsinformatik.

[54]  Viswanath Venkatesh,et al.  Consumer Acceptance and Use of Information Technology: Extending the Unified Theory of Acceptance and Use of Technology , 2012, MIS Q..

[55]  Yadong Liu,et al.  Computer vision technology in agricultural automation —A review , 2020 .

[56]  D. Vladeck Machines without Principals: Liability Rules and Artificial Intelligence , 2014 .

[57]  Matthias Söllner,et al.  AI-Based Digital Assistants , 2019, Business & Information Systems Engineering.

[58]  Adrian Hilton,et al.  Computer Vision in Sports , 2014, Advances in Computer Vision and Pattern Recognition.

[59]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[60]  Christian Janiesch,et al.  White, Grey, Black: Effects of XAI Augmentation on the Confidence in AI-based Decision Support Systems , 2020, ICIS.