Requirements Practices and Gaps When Engineering Human-Centered Artificial Intelligence Systems

[Context] Engineering Artificial Intelligence (AI) software is a relatively new area with many challenges, unknowns, and limited proven best practices. Big companies such as Google, Microsoft, and Apple have provided a suite of recent guidelines to assist engineering teams in building human-centered AI systems. [Objective] The practices currently adopted by practitioners for developing such systems, especially during Requirements Engineering (RE), are little studied and reported to date. [Method] This paper presents the results of a survey conducted to understand current industry practices in RE for AI (RE4AI) and to determine which key human-centered AI guidelines should be followed. Our survey is based on mapping existing industrial guidelines, best practices, and efforts in the literature. [Results] We surveyed 29 professionals and found most participants agreed that all the human-centered aspects we mapped should be addressed in RE. Further, we found that most participants were using UML or Microsoft Office to present requirements. [Conclusion] We identify that most of the tools currently used are not equipped to manage AI-based software, and the use of UML and Office may pose issues to the quality of requirements captured for AI. Also, all human-centered practices mapped from the guidelines should be included in RE.

[1]  M. Dainoff,et al.  Transitioning to Human Interaction with AI Systems: New Challenges and Opportunities for HCI Professionals to Enable Human-Centered AI , 2021, Int. J. Hum. Comput. Interact..

[2]  H. Barzamini,et al.  CADE: The Missing Benchmark in Evaluating Dataset Requirements of AI-enabled Software , 2022, 2022 IEEE 30th International Requirements Engineering Conference (RE).

[3]  L. Briand,et al.  Automated Question Answering for Improved Understanding of Compliance Requirements: A Multi-Document Study , 2022, 2022 IEEE 30th International Requirements Engineering Conference (RE).

[4]  Liming Zhu,et al.  Software engineering for Responsible AI: An empirical study and operationalised patterns , 2021, 2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).

[5]  Xavier Franch,et al.  Software Engineering for AI-Based Systems: A Survey , 2021, ACM Trans. Softw. Eng. Methodol..

[6]  D. Berry.  Requirements Engineering for Artificial Intelligence: What Is a Requirements Specification for an Artificial Intelligence? , 2022, REFSQ.

[7]  Sari Kujala,et al.  Transparency and Explainability of AI Systems: Ethical Guidelines in Practice , 2022, REFSQ.

[8]  Rachel Dzombak,et al.  Human-Centered AI , 2021, IEEE Pervasive Comput..

[9]  J. Grundy,et al.  What’s up with Requirements Engineering for Artificial Intelligence Systems? , 2021, 2021 IEEE 29th International Requirements Engineering Conference (RE).

[10]  Marcos Kalinowski,et al.  Requirements Engineering for Machine Learning: A Systematic Mapping Study , 2021, 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA).

[11]  Jennifer Horkoff,et al.  Non-functional Requirements for Machine Learning: Understanding Current Use and Challenges in Industry , 2021, 2021 IEEE 29th International Requirements Engineering Conference (RE).

[12]  Nicolas Guelfi,et al.  An MDE Method for Improving Deep Learning Dataset Requirements Engineering using Alloy and UML , 2021, MODELSWARD.

[13]  Mark A. Neerincx,et al.  Human-centered XAI: Developing design patterns for explanations of clinical decision support systems , 2021, Int. J. Hum. Comput. Stud..

[14]  Giancarlo Guizzardi,et al.  Ontology-Based Modeling and Analysis of Trustworthiness Requirements: Preliminary Results , 2020, ER.

[15]  Albrecht Schmidt,et al.  Interactive Human Centered Artificial Intelligence: A Definition and Research Challenges , 2020, AVI.

[16]  Nan Niu,et al.  Faulty Requirements Made Valuable: On the Role of Data Quality in Deep Learning , 2020, 2020 IEEE Seventh International Workshop on Artificial Intelligence for Requirements Engineering (AIRE).

[17]  Dietmar Nedbal,et al.  Scenario-Based Requirements Elicitation for User-Centric Explainable AI - A Case in Fraud Detection , 2020, CD-MAKE.

[18]  Mikio Aoyama,et al.  Requirements-Driven Method to Determine Quality Characteristics and Measurements for Machine Learning Software and Its Evaluation , 2020, 2020 IEEE 28th International Requirements Engineering Conference (RE).

[19]  Julio Cesar Sampaio do Prado Leite,et al.  Non-Functional Requirements Orienting the Development of Socially Responsible Software , 2020, BPMDS/EMMSAD@CAiSE.

[20]  Mark O. Riedl,et al.  Human-centered Explainable AI: Towards a Reflective Sociotechnical Approach , 2020, HCI.

[21]  Peter A. Flach,et al.  One Explanation Does Not Fit All , 2020, KI - Künstliche Intelligenz.

[22]  Krzysztof Czarnecki,et al.  Requirements for Monitoring Inattention of the Responsible Human in an Autonomous Vehicle: The Recall and Precision Tradeoff , 2020, REFSQ Workshops.

[23]  Temitayo M. Fagbola,et al.  Towards the Development of Artificial Intelligence-based Systems: Human-Centered Functional Requirements and Open Problems , 2019, 2019 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS).

[24]  Dimitri Bohlender,et al.  Explainability as a Non-Functional Requirement , 2019, 2019 IEEE 27th International Requirements Engineering Conference (RE).

[25]  Sahar Kokaly,et al.  Toward Requirements Specification for Machine-Learned Components , 2019, 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW).

[26]  Jennifer Horkoff,et al.  Non-Functional Requirements for Machine Learning: Challenges and New Directions , 2019, 2019 IEEE 27th International Requirements Engineering Conference (RE).

[27]  Andreas Vogelsang,et al.  Requirements Engineering for Machine Learning: Perspectives from Data Scientists , 2019, 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW).

[28]  Hiroshi Kuwajima,et al.  Adapting SQuaRE for Quality Assessment of Artificial Intelligence Systems , 2019, 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW).

[29]  Kurt Sandkuhl,et al.  Putting AI into Context - Method Support for the Introduction of Artificial Intelligence into Organizations , 2019, 2019 IEEE 21st Conference on Business Informatics (CBI).

[30]  Yunfeng Zhang,et al.  AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias , 2019, IBM Journal of Research and Development.

[31]  Jeanna Neefe Matthews,et al.  Managing Bias in AI , 2019, WWW.

[32]  Wonjong Rhee,et al.  Data Requirements for Applying Machine Learning to Energy Disaggregation , 2019, Energies.

[33]  Qian Yang,et al.  Designing Theory-Driven User-Centric Explainable AI , 2019, CHI.

[34]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[35]  Jon Whittle,et al.  Is Your Software Valueless? , 2019, IEEE Software.

[36]  George Dimitrakopoulos,et al.  Α capability-oriented modelling and simulation approach for autonomous vehicle management , 2019, Simul. Model. Pract. Theory.

[37]  Rachel K. E. Bellamy,et al.  Explaining models an empirical study of how explanations impact fairness judgment , 2019 .

[38]  Mark O. Riedl.  Human-Centered Artificial Intelligence and Machine Learning , 2019, Human Behavior and Emerging Technologies.

[39]  Chong Wang,et al.  Understanding what industry wants from requirements engineers: an exploration of RE jobs in Canada , 2018, ESEM.

[40]  Rachel K. E. Bellamy,et al.  AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias , 2018, ArXiv.

[41]  Foutse Khomh,et al.  Software Engineering for Machine-Learning Applications: The Road Ahead , 2018, IEEE Software.

[42]  Julio Cesar Sampaio do Prado Leite,et al.  Software Transparency as a Key Requirement for Self-Driving Cars , 2018, 2018 IEEE 26th International Requirements Engineering Conference (RE).

[43]  Qiang He,et al.  A Survey of Current End-User Data Analytics Tool Support , 2018, 2018 IEEE International Congress on Big Data (BigData Congress).

[44]  Fabiano Dalpiaz,et al.  A Roadmap for Ethics-Aware Software Engineering , 2018, 2018 IEEE/ACM International Workshop on Software Fairness (FairWare).

[45]  Wilfried Sihn,et al.  A Conceptual Model for Developing a Smart Process Control System , 2018 .

[46]  Ivica Crnkovic,et al.  It takes three to tango: Requirement, outcome/data, and AI driven development , 2018, SiBW.

[47]  Paul R. Daugherty,et al.  Collaborative Intelligence: Humans and AI Are Joining Forces , 2018 .

[48]  Robert Roncace,et al.  Goal model analysis of autonomy requirements for Unmanned Aircraft Systems , 2017, Requirements Engineering.

[49]  Tim Miller,et al.  Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences , 2017, ArXiv.

[50]  Chong Wang,et al.  What the Job Market Wants from Requirements Engineers? An Empirical Analysis of Online Job Ads from the Netherlands , 2017, 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[51]  Mohd Izuan Hafez Ninggal,et al.  A requirement engineering model for big data software , 2017, 2017 IEEE Conference on Big Data and Analytics (ICBDA).

[52]  Virginia Dignum,et al.  Responsible Artificial Intelligence: Designing Ai for Human Values , 2017 .

[53]  Haipeng Shen,et al.  Artificial intelligence in healthcare: past, present and future , 2017, Stroke and Vascular Neurology.

[54]  Roxana Geambasu,et al.  FairTest: Discovering Unwarranted Associations in Data-Driven Applications , 2015, 2017 IEEE European Symposium on Security and Privacy (EuroS&P).

[55]  Francesco Bonchi,et al.  Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining , 2016, KDD.

[56]  Kenney Ng,et al.  Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models , 2016, CHI.

[57]  D. Sculley,et al.  Hidden Technical Debt in Machine Learning Systems , 2015, NIPS.

[58]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[59]  Branko Perisic,et al.  Sirius: A rapid development of DSM graphical editor , 2014, IEEE 18th International Conference on Intelligent Engineering Systems INES 2014.

[60]  Shivani Goel,et al.  Expert system and it's requirement engineering process , 2014, International Conference on Recent Advances and Innovations in Engineering (ICRAIE-2014).

[61]  Fulvio Mastrogiovanni,et al.  Functional requirements and design issues for a socially assistive robot for elderly people with mild cognitive impairments , 2013, 2013 IEEE RO-MAN.

[62]  Daniel Kondermann,et al.  Ground truth design principles: an overview , 2013, VIGTA@ICVS.

[63]  Andrea Herrmann,et al.  Requirements Engineering in Practice: There Is No Requirements Engineer Position , 2013, REFSQ.

[64]  M. Bonfe,et al.  Towards automated surgical robotics: A requirements engineering approach , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[65]  Daniel Amyot,et al.  GRL Modeling and Analysis with jUCMNav , 2011, iStar.

[66]  Shari Lawrence Pfleeger,et al.  Principles of survey research part 2: designing a survey , 2002, SOEN.

[67]  Martin C. Maguire,et al.  Methods to support human-centred design , 2001, Int. J. Hum. Comput. Stud..