ALEEDSA: Augmented Reality for Interactive Machine Learning

In this work, we present ALEEDSA: the first system for performing interactive machine learning with augmented reality. The system is characterized by the following three distinctive features: First, immersion is used for visualizing machine learning models in terms of their outcomes. The outcomes can then be compared against domain knowledge (e.g., via counterfactual explanations) so that users can better understand the behavior of machine learning models. Second, interactivity with augmented reality along the complete machine learning pipeline fosters rapid modeling. Third, collaboration enables a multi-user setting, wherein machine learning engineers and domain experts can jointly discuss the behavior of machine learning models. The effectiveness of our proof-of-concept is demonstrated in an experimental study involving both students and business professionals. Altogether, ALEEDSA provides a more straightforward utilization of machine learning in organizational and educational practice.

[1]  Hirokazu Kato,et al.  Relation between location of information displayed by augmented reality and user's memorization , 2012, AH '12.

[2]  Hirokazu Kato,et al.  Augmented reality as multimedia: the case for situated vocabulary learning , 2016, Res. Pract. Technol. Enhanc. Learn..

[3]  Anna Syberfeldt,et al.  Augmented Reality Smart Glasses in the Smart Factory: Product Evaluation Guidelines and Review of Available Products , 2017, IEEE Access.

[4]  Franco Turini,et al.  A Survey of Methods for Explaining Black Box Models , 2018, ACM Comput. Surv..

[5]  Long Chen,et al.  Recent Developments and Future Challenges in Medical Mixed Reality , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[6]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[7]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[8]  Alexander M. Rush,et al.  Visual Interaction with Deep Learning Models through Collaborative Semantic Inference , 2019, IEEE Transactions on Visualization and Computer Graphics.

[9]  Charles T. Loop,et al.  Holoportation: Virtual 3D Teleportation in Real-time , 2016, UIST.

[10]  Przemyslaw Biecek,et al.  DALEX: explainers for complex predictive models , 2018, J. Mach. Learn. Res..

[11]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[12]  Zachary Chase Lipton The mythos of model interpretability , 2016, ACM Queue.

[13]  Massimo Bergamasco,et al.  Thermal Feedback in Virtual Environments , 1997, Presence: Teleoperators & Virtual Environments.

[14]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[15]  Ming Yin,et al.  Understanding the Effect of Accuracy on Trust in Machine Learning Models , 2019, CHI.

[16]  M. Mühlhäuser,et al.  CheckMate: Exploring a Tangible Augmented Reality Interface for Remote Interaction , 2018 .

[17]  Stefaan Ternier,et al.  Mobile Augmented Reality for Learning: A Case Study , 2011 .