Inertial Sensing Meets Artificial Intelligence: Opportunity or Challenge?

The inertial navigation system (INS) has been widely used to provide self-contained and continuous motion estimation in intelligent transportation systems. Recently, the emergence of chip-level inertial sensors has expanded the relevant applications from positioning, navigation, and mobile mapping to location-based services, unmanned systems, and transportation big data. Meanwhile, benefit from the emergence of big data and the improvement of algorithms and computing power, artificial intelligence (AI) has become a consensus tool that has been successfully applied in various fields. This article reviews the research on using AI technology to enhance inertial sensing from various aspects, including sensor design and selection, calibration and error modeling, navigation and motion-sensing algorithms, multi-sensor information fusion, system evaluation, and practical application. Based on the over 30 representative articles selected from the nearly 300 related publications, this article summarizes the state of the art, advantages, and challenges on each aspect. Finally, it summarizes nine advantages and nine challenges of AI-enhanced inertial sensing and then points out future research directions.

[1]  Giancarlo Fortino,et al.  Clustering-Learning-Based Long-Term Predictive Localization in 5G-Envisioned Internet of Connected Vehicles , 2021, IEEE Transactions on Intelligent Transportation Systems.

[2]  Keemin Sohn,et al.  Image-Based Learning to Measure Traffic Density Using a Deep Convolutional Neural Network , 2018, IEEE Transactions on Intelligent Transportation Systems.

[3]  Chun-Fei Hsu,et al.  Adaptive fuzzy wavelet neural controller design for chaos synchronization , 2011, Expert Syst. Appl..

[4]  Yong K. Cho,et al.  Effective inertial sensor quantity and locations on a body for deep learning-based worker's motion recognition , 2020 .

[5]  Frank Dellaert,et al.  Information fusion in navigation systems via factor graph based incremental smoothing , 2013, Robotics Auton. Syst..

[6]  Naser El-Sheimy,et al.  Toward Robust Crowdsourcing-Based Localization: A Fingerprinting Accuracy Indicator Enhanced Wireless/Magnetic/Inertial Integration Approach , 2019, IEEE Internet of Things Journal.

[7]  Masud Ahmed,et al.  Prediction of Gender and Age from Inertial Sensor-based Gait Dataset , 2019, 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR).

[8]  N. El-Sheimy,et al.  INTELLIGENT TUNING OF A KALMAN FILTER USING LOW-COST MEMS INERTIAL SENSORS , 2007 .

[9]  Jian Wang,et al.  GPS/INS/Odometer Integrated System Using Fuzzy Neural Network for Land Vehicle Navigation Applications , 2014, Journal of Navigation.

[10]  Martin Brossard,et al.  AI-IMU Dead-Reckoning , 2019, IEEE Transactions on Intelligent Vehicles.

[11]  Liu Jun,et al.  Temperature drift modeling of MEMS gyroscope based on genetic-Elman neural network , 2016 .

[12]  Johannes R. Sveinsson,et al.  Multiple classifiers applied to multisource remote sensing data , 2002, IEEE Trans. Geosci. Remote. Sens..

[13]  Sun Xue-cheng.  Temperature compensation of MEMS gyroscope based on grey model and RBF neural network , 2010 .

[14]  Zhaofeng Li,et al.  Adaptive EKF Based on HMM Recognizer for Attitude Estimation Using MEMS MARG Sensors , 2018, IEEE Sensors Journal.

[15]  Xiaoji Niu,et al.  RAPID AND ACCURATE INS ALIGNMENT FOR LAND APPLICATIONS , 2010 .

[16]  Denis F. Wolf,et al.  Monte Carlo Localization on Gaussian Process Occupancy Maps for Urban Environments , 2018, IEEE Transactions on Intelligent Transportation Systems.

[17]  Michael Milford,et al.  Vision-based place recognition: how low can you go? , 2013, Int. J. Robotics Res..

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[20]  Xiaoji Niu,et al.  Estimate the Pitch and Heading Mounting Angles of the IMU for Land Vehicular GNSS/INS Integrated System , 2021, IEEE Transactions on Intelligent Transportation Systems.

[21]  John Weston,et al.  Strapdown Inertial Navigation Technology, Second Edition , 2005 .

[22]  Mingguo Zhao,et al.  Towards artificial general intelligence with hybrid Tianjic chip architecture , 2019, Nature.

[23]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[24]  Laurent Itti,et al.  Inertial-based Motion Capturing and Smart Training System , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Vibhor Agrawal,et al.  Missing Sample Recovery for Wireless Inertial Sensor-Based Human Movement Acquisition , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Yann LeCun,et al.  1.1 Deep Learning Hardware: Past, Present, and Future , 2019, 2019 IEEE International Solid- State Circuits Conference - (ISSCC).

[27]  Jing-Yu Yang,et al.  Neural network assisted ultra-tightly coupled GPS/INS integration for seamless navigation , 2012, 2012 12th International Conference on ITS Telecommunications.

[28]  Naser El-Sheimy,et al.  Wireless Fingerprinting Uncertainty Prediction Based on Machine Learning , 2019, Sensors.

[29]  Andrew Markham,et al.  Deep-Learning-Based Pedestrian Inertial Navigation: Methods, Data Set, and On-Device Inference , 2020, IEEE Internet of Things Journal.

[30]  Hongli Zhang,et al.  Mobile cloud sensing, big data, and 5G networks make an intelligent and smart world , 2015, IEEE Network.

[31]  Albert-László Barabási,et al.  Understanding individual human mobility patterns , 2008, Nature.

[32]  Naser El-Sheimy,et al.  Deep Reinforcement Learning (DRL): Another Perspective for Unsupervised Wireless Localization , 2020, IEEE Internet of Things Journal.

[33]  Chen Xiyuan.  Modeling random gyro drift by time series neural networks and by traditional method , 2003, International Conference on Neural Networks and Signal Processing, 2003. Proceedings of the 2003.

[34]  Paolo Dario,et al.  Daily activity recognition with inertial ring and bracelet: An unsupervised approach , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Mamoun F. Abdel-Hafez,et al.  Enhanced, Delay Dependent, Intelligent Fusion for INS/GPS Navigation System , 2014, IEEE Sensors Journal.

[36]  Anis Koubâa,et al.  RoadSense: Smartphone Application to Estimate Road Conditions Using Accelerometer and Gyroscope , 2017, IEEE Sensors Journal.

[37]  X. Li.  Comparing the Kalman filter with a Monte Carlo-based artificial neural network in the INS/GPS vector gravimetric system , 2009 .

[38]  Tom Schaul,et al.  Rainbow: Combining Improvements in Deep Reinforcement Learning , 2017, AAAI.

[39]  Karl Iagnemma,et al.  Slippage prediction for off-road mobile robots via machine learning regression and proprioceptive sensing , 2018, Robotics Auton. Syst..

[40]  Alain Pagani,et al.  Learning to Fuse: A Deep Learning Approach to Visual-Inertial Camera Pose Estimation , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[41]  I.O. Lertxundi,et al.  Inertial Events Identification Using Neural Networks in Railway Environments , 2008, 2008 International Conference on Advances in Electronics and Micro-electronics.

[42]  Rong Chen,et al.  A Deep Reinforcement Learning-Based Framework for Dynamic Resource Allocation in Multibeam Satellite Systems , 2018, IEEE Communications Letters.

[43]  Sinpyo Hong,et al.  Observability of error States in GPS/INS integration , 2005, IEEE Transactions on Vehicular Technology.

[44]  Isaac Skog,et al.  In-Car Positioning and Navigation Technologies—A Survey , 2009, IEEE Transactions on Intelligent Transportation Systems.

[45]  Mustafa Mousa,et al.  Inertial Measurement Units-Based Probe Vehicles: Automatic Calibration, Trajectory Estimation, and Context Detection , 2018, IEEE Transactions on Intelligent Transportation Systems.

[46]  Paolo Bonato,et al.  Recent machine learning advancements in sensor-based mobility analysis: Deep learning for Parkinson's disease assessment , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[47]  William J. Horrey,et al.  Recognition of Manual Driving Distraction Through Deep-Learning and Wearable Sensing , 2019 .

[48]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[49]  Zha Feng,et al.  Prediction of Gyro Motor's State Based on Grey Model and BP Neural Network , 2009, 2009 Second International Conference on Intelligent Computation Technology and Automation.

[50]  Yan Shi,et al.  Pipeline Bending Strain Measurement and Compensation Technology Based on Wavelet Neural Network , 2016, J. Sensors.

[51]  Christoffer Heckman,et al.  Learning to calibrate: Reinforcement learning for guided calibration of visual–inertial rigs , 2019, Int. J. Robotics Res..

[52]  Hong Wang,et al.  A low-cost INS/GPS integration methodology based on random forest regression , 2013, Expert Syst. Appl..

[53]  John,et al.  Strapdown Inertial Navigation Technology - 2nd Edition , 2005 .

[54]  Keyu Wu,et al.  AbolDeepIO: A Novel Deep Inertial Odometry Network for Autonomous Vehicles , 2020, IEEE Transactions on Intelligent Transportation Systems.

[55]  Jonathan Kelly,et al.  LSTM-Based Zero-Velocity Detection for Robust Inertial Navigation , 2018, 2018 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[56]  Guang-Zhong Yang,et al.  A Deep Learning Approach to on-Node Sensor Data Analytics for Mobile or Wearable Devices , 2017, IEEE Journal of Biomedical and Health Informatics.

[57]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[58]  Mehmet C. Vuran,et al.  Semi-supervised near-miss fall detection for ironworkers with a wearable inertial measurement unit , 2016 .

[59]  Aboelmagd Noureldin,et al.  A Novel Multi-Level Integrated Navigation System for Challenging GNSS Environments , 2021, IEEE Transactions on Intelligent Transportation Systems.

[60]  Yang Yuanxi,et al.  A New Two-Step Adaptive Robust Kalman Filtering in GPS/INS Integrated Navigation System , 2010 .

[61]  You Li,et al.  Integration of MEMS Sensors, WiFi, and Magnetic Features for Indoor Pedestrian Navigation with Consumer Portable Devices , 2016 .

[62]  Gordon Wyeth,et al.  Autonomous helicopter hover using an artificial neural network , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[63]  Peng Jiang,et al.  An IMU Evaluation Method Using a Signal Grafting Scheme , 2016, Sensors.