MagTrack: Enabling Safe Driving Monitoring with Wearable Magnetics

"Hands on the wheel, eyes on the road" is the central guideline of safe vehicle driving practices. Many advanced driver assistance systems can effectively detect abnormal vehicle motions. However, these systems often leave insufficient time for drivers to respond to complex road situations, especially when the drivers are distracted. To reduce accidents, it is essential to detect whether a driver complies with safe driving guidelines in real time and provide warnings early before any dangerous maneuvers occur. There are vision-based driver distraction monitoring systems which rely on cameras in high-end vehicles, but their performances are heavily constrained by visibility requirements. In this paper, we present MagTrack, a driver monitoring system that is based on tracking magnetic tags worn by the user. With a single smartwatch and two low-cost magnetic accessories: a hand magnetic ring and a head magnetic eyeglasses clip, our system tracks and classifies a driver's bimanual and head movements simultaneously using both analytical and approximation sensing models. Our approach is robust to driver's postures, vehicles, and environmental changes. We demonstrate that a wide range of activities can be detected by our system, including bimanual steering, visual and manual distractions, and lane changes and turns. In extensive road tests with 500+ instances of driving activities and 500+ minutes of road driving with 10 subjects, MagTrack achieves 87% of precision and 90% of recall rate on the detection of unsafe driving activities.

[1]  Mohan M. Trivedi,et al.  Head Pose Estimation and Augmented Reality Tracking: An Integrated System and Evaluation for Monitoring Driver Awareness , 2010, IEEE Transactions on Intelligent Transportation Systems.

[2]  Philip W. Kithil,et al.  DRIVER ALERTNESS DETECTION RESEARCH USING CAPACITIVE SENSOR ARRAY , 2001 .

[3]  Kocsis Zoltán Tamás,et al.  IEEE World Congress on Computational Intelligence , 2019, IEEE Computational Intelligence Magazine.

[4]  M. Haghi,et al.  Wearable Devices in Medical Internet of Things: Scientific Research and Commercially Available Devices , 2017, Healthcare informatics research.

[5]  Nazan Aksan,et al.  Forward Collision Warning: Clues to Optimal Timing of Advisory Warnings. , 2016, SAE international journal of transportation safety.

[6]  Terry Kline Using Efficient Steering Techniques , 2007 .

[7]  Richard P. Martin,et al.  BigRoad: Scaling Road Data Acquisition for Dependable Self-Driving , 2017, MobiSys.

[8]  Jie Wu,et al.  Large-Scale Battery System Development and User-Specific Driving Behavior Analysis for Emerging Electric-Drive Vehicles , 2011 .

[9]  John McDonald Rotation about an arbitrary axis , 2008 .

[10]  M. Bouvard,et al.  The Increased Risk of Road Crashes in Attention Deficit Hyperactivity Disorder (ADHD) Adult Drivers: Driven by Distraction? Results from a Responsibility Case-Control Study , 2014, PloS one.

[11]  Sheng Zhang,et al.  A Novel Model-Based Driving Behavior Recognition System Using Motion Sensors , 2016, Sensors.

[12]  Maysam Ghovanloo,et al.  A comprehensive method for magnetic sensor calibration: A precise system for 3-D tracking of the tongue movements , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[13]  Claudio Cobelli,et al.  The Effect of Walking on Postprandial Glycemic Excursion in Patients With Type 1 Diabetes and Healthy People , 2012, Diabetes Care.

[14]  儿玉勇司,et al.  Collision avoidance system , 2014 .

[15]  Wei Li,et al.  Evaluation of driver fatigue with multi-indicators based on artificial neural network , 2016 .

[16]  Johannes van der Steen,et al.  Visuomotor impairment in early-stage Alzheimer's disease: changes in relative timing of eye and hand movements. , 2012, Journal of Alzheimer's disease : JAD.

[17]  Michael I. Miller,et al.  Conditional-mean estimation via jump-diffusion processes in multiple target tracking/recognition , 1995, IEEE Trans. Signal Process..

[18]  Jean-Christophe Popieul,et al.  Using driver's head movements evolution as a drowsiness indicator , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[19]  Swarun Kumar,et al.  Towards Wearable Everyday Body-Frame Tracking using Passive RFIDs , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[20]  Hiroaki Seki,et al.  Wearable handwriting input device using magnetic field: 2nd report: Influence of misalignment of magnet and writing plane , 2010 .

[21]  Lama Nachman,et al.  Mago: Mode of Transport Inference Using the Hall-Effect Magnetic Sensor and Accelerometer , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[22]  Benton H. Calhoun,et al.  Body Area Sensor Networks: Challenges and Opportunities , 2009, Computer.

[23]  Daniel P. Siewiorek,et al.  Activity recognition and monitoring using multiple sensors on different body positions , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[24]  Guoliang Xing,et al.  SafeWatch: A Wearable Hand Motion Tracking System for Improving Driving Safety , 2017, 2017 IEEE/ACM Second International Conference on Internet-of-Things Design and Implementation (IoTDI).

[25]  Alexander V. Smirnov,et al.  Smartphone-based identification of dangerous driving situations: Algorithms and implementation , 2016, 2016 18th Conference of Open Innovations Association and Seminar on Information Security and Protection of Information Technology (FRUCT-ISPIT).

[26]  In-Ho Choi,et al.  Tracking a Driver’s Face against Extreme Head Poses and Inference of Drowsiness Using a Hidden Markov Model , 2016 .

[27]  Simon Farrell,et al.  Maximum Likelihood Parameter Estimation , 2018 .

[28]  Alexandrina Rogozan,et al.  Driver head pose estimation using efficient descriptor fusion , 2016, EURASIP J. Image Video Process..

[29]  Dana Kulic,et al.  Segmentation of human upper body movement using multiple IMU sensors , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[30]  VALENTIN RADU,et al.  Multimodal Deep Learning for Activity and Context Recognition , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[31]  Vigneshwaran Subbaraju,et al.  Can multimodal sensing detect and localize transient events? , 2018, Defense + Security.

[32]  Kang G. Shin,et al.  Invisible Sensing of Vehicle Steering with Smartphones , 2015, MobiSys.

[33]  Ted Herman,et al.  Hand hygiene duration and technique recognition using wrist-worn sensors , 2015, IPSN.

[34]  David Sandberg,et al.  Particle swarm optimization of feedforward neural networks for the detection of drowsy driving , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[35]  Santokh Singh,et al.  Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey , 2015 .

[36]  V. Schlageter,et al.  Magnetic pill tracking: a novel non‐invasive tool for investigation of human digestive motility , 2005, Neurogastroenterology and motility : the official journal of the European Gastrointestinal Motility Society.

[37]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[38]  Minglu Li,et al.  Fine-Grained Abnormal Driving Behaviors Detection and Identification with Smartphones , 2017, IEEE Transactions on Mobile Computing.

[39]  Daniel V. McGehee,et al.  Collision Warning Timing, Driver Distraction, and Driver Response to Imminent Rear-End Collisions in a High-Fidelity Driving Simulator , 2002, Hum. Factors.

[40]  Kun Li,et al.  Personalized Driving Behavior Monitoring and Analysis for Emerging Hybrid Vehicles , 2012, Pervasive.

[41]  Dong Nguyen,et al.  ParkSense: Automatic Parking Positioning by Leveraging In-Vehicle Magnetic Field Variation , 2017, IEEE Access.

[42]  Maysam Ghovanloo,et al.  Towards a magnetic localization system for 3-D tracking of tongue movements in speech-language therapy , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[43]  Minglu Li,et al.  SenSpeed: Sensing Driving Conditions to Estimate Vehicle Speed in Urban Environments , 2014, IEEE Transactions on Mobile Computing.

[44]  Vigneshwaran Subbaraju,et al.  The case for smartwatch-based diet monitoring , 2015, 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[45]  Ashraf Darwish,et al.  Wearable and Implantable Wireless Sensor Network Solutions for Healthcare Monitoring , 2011, Sensors.

[46]  Gwen Littlewort,et al.  Drowsy Driver Detection Through Facial Movement Analysis , 2007, ICCV-HCI.

[47]  Mohan M. Trivedi,et al.  Driving style recognition using a smartphone as a sensor platform , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[48]  Giovanni Sparacino,et al.  Physical activity measured by physical activity monitoring system correlates with glucose trends reconstructed from continuous glucose monitoring. , 2013, Diabetes technology & therapeutics.

[49]  Sean White,et al.  uTrack: 3D input using two magnetic sensors , 2013, UIST.

[50]  Hans P A Van Dongen,et al.  Efficient driver drowsiness detection at moderate levels of drowsiness. , 2013, Accident; analysis and prevention.

[51]  Inseok Hwang,et al.  SocioPhone: everyday face-to-face interaction monitoring platform using multi-phone sensor fusion , 2013, MobiSys '13.

[52]  Deborah Estrin,et al.  Using mobile phones to determine transportation modes , 2010, TOSN.

[53]  Philp W Kithil Driver Alertness Detection Research Using Capacitive Sensor Array , 2017 .

[54]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[55]  Maria Staubach,et al.  Factors correlated with traffic accidents as a basis for evaluating Advanced Driver Assistance Systems. , 2009, Accident; analysis and prevention.

[56]  Fanglin Chen,et al.  CarSafe app: alerting drowsy and distracted drivers using dual cameras on smartphones , 2013, MobiSys.

[57]  Andrew S. Miner,et al.  Anomaly intrusion detection using one class SVM , 2004, Proceedings from the Fifth Annual IEEE SMC Information Assurance Workshop, 2004..

[58]  Wei Mei,et al.  Simultaneous tracking and classification: a modularized scheme , 2007, IEEE Transactions on Aerospace and Electronic Systems.