Using Simulation to Aid the Design and Optimization of Intelligent User Interfaces for Quality Assurance Processes in Machine Learning

Many mission-critical applications of machine learning (ML) in the real-world require a quality assurance (QA) process before the decisions or predictions of an ML model can be deployed. Because QA4ML users have to view a non-trivial amount of data and perform many input actions to correct errors made by the ML model, an optimally-designed user interface (UI) can reduce the cost of interactions significantly. A UI’s effectiveness can be affected by many factors, such as the number of data objects processed concurrently, the types of commands for correcting errors, and the availability of algorithms for assisting users. We propose using simulation to aid the design and optimization of intelligent user interfaces for QA4ML processes. In particular, we focus on simulating the combined effects of human intelligence in selecting appropriate commands and algorithms, and machine intelligence in providing a collection of general-purpose algorithms for reordering data objects to be quality-assured.