Self-awareness in intelligent vehicles: Feature based dynamic Bayesian models for abnormality detection

Abstract The evolution of Intelligent Transportation Systems in recent times necessitates the development of self-awareness in agents. Before the intensive use of Machine Learning, the detection of abnormalities was manually programmed by checking every variable and creating huge nested conditions that are very difficult to track. This paper aims to introduce a novel method to develop self-awareness in autonomous vehicles that mainly focuses on detecting abnormal situations around the considered agents. Multi-sensory time-series data from the vehicles are used to develop the data-driven Dynamic Bayesian Network (DBN) models used for future state prediction and the detection of dynamic abnormalities. Moreover, an initial level collective awareness model that can perform joint anomaly detection in co-operative tasks is proposed. The GNG algorithm learns the DBN models’ discrete node variables; probabilistic transition links connect the node variables. A Markov Jump Particle Filter (MJPF) is applied to predict future states and detect when the vehicle is potentially misbehaving using learned DBNs as filter parameters. In this paper, datasets from real experiments of autonomous vehicles performing various tasks used to learn and test a set of switching DBN models.

[1]  John R. Hershey,et al.  Approximating the Kullback Leibler Divergence Between Gaussian Mixture Models , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[2]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[3]  G. A. Tawney.  Feeling and self-awareness. , 1902 .

[4]  Jens B. Asendorpf,et al.  Self-Awareness and Other-Awareness II: Mirror Self-Recognition, Social Contingency Awareness, and Synchronic Imitation , 1996 .

[5]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[6]  Fernando García,et al.  V2X communications architecture for off-road autonomous vehicles , 2017, 2017 IEEE International Conference on Vehicular Electronics and Safety (ICVES).

[7]  A. Agresti.  [A Survey of Exact Inference for Contingency Tables]: Rejoinder , 1992 .

[8]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[9]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[10]  Aníbal Matos,et al.  A Safety Monitoring Model for a Faulty Mobile Robot , 2018, Robotics.

[11]  David Martin,et al.  Anomaly Detection in Video Data Based on Probabilistic Latent Space Models , 2020, 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS).

[12]  Daphne Koller,et al.  Sampling in Factored Dynamic Systems , 2001, Sequential Monte Carlo Methods in Practice.

[13]  Arturo de la Escalera,et al.  Global and Local Path Planning Study in a ROS-Based Research Platform for Autonomous Vehicles , 2018 .

[14]  Bernhard Rinner,et al.  Self-Aware and Self-Expressive Camera Networks , 2015, Computer.

[15]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[16]  Christian Berger,et al.  From a Competition for Self-Driving Miniature Cars to a Standardized Experimental Platform: Concept, Models, Architecture, and Evaluation , 2014, ArXiv.

[17]  Guangming Xiong,et al.  Autonomous driving of intelligent vehicle BIT in 2009 Future Challenge of China , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[18]  H. P. Blom.  An efficient filter for abruptly changing systems , 1984, The 23rd IEEE Conference on Decision and Control.

[19]  Dominik Endres,et al.  A new metric for probability distributions , 2003, IEEE Transactions on Information Theory.

[20]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[21]  Rodolfo Lourenzutti,et al.  The Hellinger distance in Multicriteria Decision Making: An illustration to the TOPSIS and TODIM methods , 2014, Expert Syst. Appl..

[22]  Mario Marchese,et al.  Dynamic Bayesian Approach for decision-making in Ego-Things , 2019, 2019 IEEE 5th World Forum on Internet of Things (WF-IoT).

[23]  Tony Jebara,et al.  Machine Learning: Discriminative and Generative , 2012 .

[24]  Michael I. Jordan,et al.  Nonparametric Bayesian Learning of Switching Linear Dynamical Systems , 2008, NIPS.

[25]  Carlo S. Regazzoni,et al.  Learning Switching Models for Abnormality Detection for Autonomous Driving , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[26]  L. Pardo.  Statistical Inference Based on Divergence Measures , 2005 .

[27]  Siegfried Dewitte,et al.  Me, Myself, and My Choices: The Influence of Private Self-Awareness on Choice , 2009 .

[28]  D. Rubin,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[29]  R. P. Marques,et al.  Discrete-Time Markov Jump Linear Systems , 2004, IEEE Transactions on Automatic Control.

[30]  Alan F. T. Winfield,et al.  Robots with internal models: A route to self-aware and hence safer robots , 2014 .

[31]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[32]  A. Travers.  The Identification of Self , 1995 .

[33]  S. Duval,et al.  A theory of objective self awareness , 1972 .

[34]  Huanshui Zhang,et al.  Optimal Markov jump filter for stochastic systems with Markovian transmission delays , 2011, Proceedings of the 30th Chinese Control Conference.

[35]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[36]  Richard Bishop.  Intelligent Vehicle Applications Worldwide , 2000, IEEE Intell. Syst..

[37]  Hyun Myung,et al.  Robust Interacting Multiple Model With Modeling Uncertainties for Maneuvering Target Tracking , 2019, IEEE Access.

[38]  Mahdyar Ravanbakhsh,et al.  a Multi-Perspective Approach to Anomaly Detection for Self -Aware Embodied Agents , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[40]  Alejandro Betancourt,et al.  Static force field representation of environments based on agents’ nonlinear motions , 2017, EURASIP Journal on Advances in Signal Processing.

[41]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[42]  F. Deane,et al.  Development of the Levels of Emotional Awareness Scale for Children (LEAS-C). , 2005, The British journal of developmental psychology.

[43]  Bernhard Rinner,et al.  Multisensorial Generative and Descriptive Self-Awareness Models for Autonomous Systems , 2020, Proceedings of the IEEE.

[44]  Mahdyar Ravanbakhsh,et al.  Hierarchy of Gans for Learning Embodied Self-Awareness Model , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[45]  Neil Salkind.  Encyclopedia of Measurement and Statistics , 2006 .

[46]  Y. Boers,et al.  Efficient particle filter for jump Markov nonlinear systems , 2005 .

[47]  Carlo S. Regazzoni,et al.  Self-awareness in Intelligent Vehicles: Experience Based Abnormality Detection , 2019, ROBOT.

[48]  Carlo S. Regazzoni,et al.  Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[49]  Jianqiang Wang,et al.  A Driving Behavior Awareness Model based on a Dynamic Bayesian Network and Distributed Genetic Algorithm , 2018, Int. J. Comput. Intell. Syst..

[50]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.