Improving Variational Autoencoder based Out-of-Distribution Detection for Embedded Real-time Applications

Uncertainties in machine learning are a significant roadblock for its application in safety-critical cyber-physical systems (CPS). One source of uncertainty arises from distribution shifts in the input data between training and test scenarios. Detecting such distribution shifts in real-time is an emerging approach to address the challenge. The high dimensional input space in CPS applications involving imaging adds extra difficulty to the task. Generative learning models are widely adopted for the task, namely out-of-distribution (OoD) detection. To improve the state-of-the-art, we studied existing proposals from both machine learning and CPS fields. In the latter, safety monitoring in real-time for autonomous driving agents has been a focus. Exploiting the spatiotemporal correlation of motion in videos, we can robustly detect hazardous motion around autonomous driving agents. Inspired by the latest advances in the Variational Autoencoder (VAE) theory and practice, we tapped into the prior knowledge in data to further boost OoD detection’s robustness. Comparison studies over nuScenes and Synthia data sets show our methods significantly improve detection capabilities of OoD factors unique to driving scenarios, 42% better than state-of-the-art approaches. Our model also generalized near-perfectly, 97% better than the state-of-the-art across the real-world and simulation driving data sets experimented. Finally, we customized one proposed method into a twin-encoder model that can be deployed to resource limited embedded devices for real-time OoD detection. Its execution time was reduced over four times in low-precision 8-bit integer inference, while detection capability is comparable to its corresponding floating-point model.

[1]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[2]  Najmeh Abiri,et al.  Variational auto-encoders with Student's t-prior , 2020, ESANN.

[3]  Roland Siegwart,et al.  Out-of-Distribution Detection for Automotive Perception , 2020, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC).

[4]  See-Kiong Ng,et al.  Anomaly Detection with Generative Adversarial Networks for Multivariate Time Series , 2018, ArXiv.

[5]  Bogdan Raducanu,et al.  Temporal Coherence for Active Learning in Videos , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[6]  Jasper Snoek,et al.  Likelihood Ratios for Out-of-Distribution Detection , 2019, NeurIPS.

[7]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Alexander A. Alemi,et al.  WAIC, but Why? Generative Ensembles for Robust Anomaly Detection , 2018 .

[9]  Meng Wang,et al.  Generative Adversarial Active Learning for Unsupervised Outlier Detection , 2018, IEEE Transactions on Knowledge and Data Engineering.

[10]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[12]  Yee Whye Teh,et al.  Do Deep Generative Models Know What They Don't Know? , 2018, ICLR.

[13]  Arvind Easwaran,et al.  Out-of-Distribution Detection in Multi-Label Datasets using Latent Space of β-VAE , 2020, 2020 IEEE Security and Privacy Workshops (SPW).

[14]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[15]  William Yang Wang,et al.  Dirichlet Variational Autoencoder for Text Modeling , 2018, ArXiv.

[16]  Kibok Lee,et al.  A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks , 2018, NeurIPS.

[17]  Finbarr Murphy,et al.  End-to-End Autonomous Driving Risk Analysis: A Behavioural Anomaly Detection Approach , 2021, IEEE Transactions on Intelligent Transportation Systems.

[18]  Hanno Gottschalk,et al.  Classification Uncertainty of Deep Neural Networks Based on Gradient Information , 2018, ANNPR.

[19]  Atul Prakash,et al.  Robust Physical-World Attacks on Deep Learning Visual Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Jordi Luque,et al.  Input complexity and out-of-distribution detection with likelihood-based generative models , 2020, ICLR.

[21]  José Miguel Hernández-Lobato,et al.  Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection , 2019, ArXiv.

[22]  Xenofon Koutsoukos,et al.  Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systems , 2020, 2020 ACM/IEEE 11th International Conference on Cyber-Physical Systems (ICCPS).

[23]  Bálint Gyires-Tóth,et al.  Robust Reinforcement Learning-based Autonomous Driving Agent for Simulation and Real World , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[24]  Stephen Cass,et al.  Taking AI to the edge: Google's TPU now comes in a maker-friendly package , 2019, IEEE Spectrum.

[25]  Liang Zhang,et al.  HMFlow: Hybrid Matching Optical Flow Network for Small and Fast-Moving Objects , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[26]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[27]  Zhenfu Cao,et al.  Detecting Vehicle Anomaly in the Edge via Sensor Consistency and Frequency Characteristic , 2019, IEEE Transactions on Vehicular Technology.

[28]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Max Welling,et al.  VAE with a VampPrior , 2017, AISTATS.

[30]  Neil A. Thacker,et al.  Tutorial: Computing 2D and 3D Optical Flow. , 2004 .

[31]  Gabriele Costante,et al.  LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation , 2017, IEEE Robotics and Automation Letters.