RAMT: Real-time Attitude and Motion Tracking for Mobile Devices in Moving Vehicle

Recently a class of new in-vehicle technologies based on off-the-shelf mobile devices have been developed to improve driving safety and experience. For instance, wearables like the smartwatches are utilized to monitor the action of the driver and detect possible secondary tasks. Moreover, wearables can allow a driver to use gesture for in-vehicle controls, reducing distractions to driving. The accuracy of these systems can be significantly improved by tracking the real-time attitude of mobile devices. This paper proposes a novel system called Real-time Attitude and Motion Tracking (RAMT) that can enable a mobile device to accurately learn the coordinate system of a moving vehicle, and hence track its attitude and motion in real time. RAMT consists of a series of lightweight algorithms to sense the vehicle's movement and calculate the device's attitude. It provides a solution for trajectory-based gesture recognition. We have implemented RAMT on a smartphone and a smartwatch and evaluated the performance in 10 real driving trips. Our results show that the overall error of the coordinate system alignment is around 5° for the smartphone and 10° for the smartwatch, and over 84% of customized hand gestures can be accurately recognized with the result of RAMT. A video demo of RAMT is available at https://youtu.be/9rZp7HxyRts.

[1]  D. Gebre-Egziabher,et al.  A gyro-free quaternion-based attitude determination system suitable for implementation using low cost sensors , 2000, IEEE 2000. Position Location and Navigation Symposium (Cat. No.00CH37062).

[2]  Jiangwen Deng,et al.  An HMM-based approach for gesture segmentation and recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[3]  T. Dingus,et al.  Distracted driving and risk of road crashes among novice and experienced drivers. , 2014, The New England journal of medicine.

[4]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Lee Boon-Leng,et al.  Mobile-based wearable-type of driver fatigue detection by GSR and EMG , 2015, TENCON 2015 - 2015 IEEE Region 10 Conference.

[6]  Agus Budiyono,et al.  Principles of GNSS, Inertial, and Multi-sensor Integrated Navigation Systems , 2012 .

[7]  Liu Jun,et al.  Temperature drift modeling of MEMS gyroscope based on genetic-Elman neural network , 2016 .

[8]  Wan-Young Chung,et al.  Wearable driver drowsiness detection system based on biomedical and motion sensors , 2015, 2015 IEEE SENSORS.

[9]  Lei Feng,et al.  A driving behavior detection system based on a smartphone's built‐in sensor , 2017, Int. J. Commun. Syst..

[10]  S.V. Marshall,et al.  Vehicle detection using a magnetic field sensor , 1978, IEEE Transactions on Vehicular Technology.

[11]  Keith J. Burnham,et al.  A Research Study of Hand Gesture Recognition Technologies and Applications for Human Vehicle Interaction , 2007 .

[12]  Hae Young Noh,et al.  VVRRM: Vehicular Vibration-Based Heart RR-Interval Monitoring System , 2018, HotMobile.

[13]  Joan Climent,et al.  A Performance Evaluation of HMM and DTW for Gesture Recognition , 2012, CIARP.

[14]  Chong Wang,et al.  Superpixel-Based Hand Gesture Recognition With Kinect Depth Camera , 2015, IEEE Transactions on Multimedia.

[15]  Minglu Li,et al.  SenSpeed: Sensing Driving Conditions to Estimate Vehicle Speed in Urban Environments , 2014, IEEE Transactions on Mobile Computing.

[16]  James W. Jenness,et al.  Voice-Activated Dialing or Eating a Cheeseburger: Which is More Distracting during Simulated Driving? , 2002 .

[17]  D.A. James,et al.  An accelerometer based sensor platform for insitu elite athlete performance analysis , 2004, Proceedings of IEEE Sensors, 2004..

[18]  Myounghoon Jeon,et al.  Eyes-free In-vehicle Gesture Controls: Auditory-only Displays Reduced Visual Distraction and Workload , 2017, AutomotiveUI.

[19]  Mani B. Srivastava,et al.  From Pressure to Path: Barometer-based Vehicle Tracking , 2015, BuildSys@SenSys.

[20]  Andrew Parkes,et al.  HOW DANGEROUS IS DRIVING WITH A MOBILE PHONE? BENCHMARKING THE IMPAIRMENT TO ALCOHOL , 2002 .

[21]  Anh Nguyen,et al.  Demo: Low-power Capacitive Sensing Wristband for Hand Gesture Recognition , 2017, S3@MobiCom.

[22]  Chunming Qiao,et al.  VehSense: Slippery Road Detection Using Smartphones , 2017, 2017 IEEE 85th Vehicular Technology Conference (VTC Spring).

[23]  H. R. Harrison,et al.  Quaternions and Rotation Sequences: a Primer with Applications to Orbits, Aerospace and Virtual Reality , J. B. Kuipers, Princeton University Press, 41 William Street, Princeton, NJ 08540, USA. 1999. 372pp. Illustrated. £35.00. ISBN 0-691-05872-5. , 1999, The Aeronautical Journal (1968).

[24]  Chung-Yen Su,et al.  Kinect-based mid-air handwritten digit recognition using multiple segments and scaled coding , 2013, 2013 International Symposium on Intelligent Signal Processing and Communication Systems.

[25]  Shan Lin,et al.  Toothbrushing Monitoring using Wrist Watch , 2016, SenSys.

[26]  Niels Henrik Abel,et al.  Mémoire sur les equations algébriques, où l'on démontre l'impossibilité de la résolution de l'équation générale du cinquième degré , 2012 .

[27]  Nassir Navab,et al.  3D Pictorial Structures Revisited: Multiple Human Pose Estimation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Mikkel Baun Kjærgaard,et al.  ACTIVITY RECOGNITION ON SMART DEVICES: Dealing with diversity in the wild , 2016, GETMBL.

[29]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[30]  Paul Coulton,et al.  Using “tilt” as an interface to control “no-button” 3-D mobile games , 2008, CIE.

[31]  Jack B. Kuipers,et al.  Quaternions and Rotation Sequences: A Primer with Applications to Orbits, Aerospace and Virtual Reality , 2002 .

[32]  Richard P. Martin,et al.  Toward Detection of Unsafe Driving with Wearables , 2015, WearSys@MobiSys.

[33]  Mohan M. Trivedi,et al.  In-vehicle Hand Gesture Recognition using Hidden Markov models , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[34]  Jun Ma,et al.  Study on the evaluation method of in-vehicle gesture control , 2017, 2017 3rd IEEE International Conference on Control Science and Systems Engineering (ICCSSE).

[35]  Armando Barreto,et al.  Gyroscope drift correction algorithm for inertial measurement unit used in hand motion tracking , 2016, 2016 IEEE SENSORS.

[36]  Kang Chen,et al.  RoadAware: Learning Personalized Road Information on Daily Routes with Smartphones , 2016, 2016 25th International Conference on Computer Communication and Networks (ICCCN).

[37]  He Wang,et al.  I am a Smartwatch and I can Track my User's Arm , 2016, MobiSys.

[38]  Guoliang Xing,et al.  SafeWatch , 2019, ACM Trans. Cyber Phys. Syst..

[39]  Mohan M. Trivedi,et al.  Driving style recognition using a smartphone as a sensor platform , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[40]  Andrew Zisserman,et al.  Efficient discriminative learning of parts-based models , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[41]  Xue Liu,et al.  SafeDrive , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[42]  Jorge Dias,et al.  Vision and Inertial Sensor Cooperation Using Gravity as a Vertical Reference , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  O-larnnithipong Nonnarit,et al.  Gyroscope drift correction algorithm for inertial measurement unit used in hand motion tracking , 2016 .

[44]  Kenneth Gade,et al.  The Seven Ways to Find Heading , 2016 .