Wireless CSI-based head tracking in the driver seat

In recent years, augmented reality (AR) has drawn significant attention in the automotive industry and shown great potential for a variety of driver-assistance applications. Tracking the driver's head is vital to seamlessly merge the AR content in the driver's view but still remains an open problem. Specifically, most existing in-vehicle AR solutions rely on cameras for head tracking, which suffer from low sampling rate, weak performance at night, and even high computation costs. Wearing a dedicated headset, on the other hand, is intrusive and inconvenient for daily driving. To overcome these limitations, we propose ViHOT, a novel wireless CSI-based predictive & device-free head tracking system for in-vehicle use. Given that drivers usually mount phones on the dashboard for navigation, ViHOT leverages the CSI of the phone's WiFi signal to track the driver's head, with a light-weight design suited for real-time driving assistance. Thanks to the high WiFi frame rate, ViHOT achieves more than 10x sampling rate over conventional camera-based approaches and thus eliminates motion blur. Moreover, ViHOT's novel tracking algorithm accurately translates CSI phase readings to head orientations (merely 4° -10° median error) without relying on any head-mounted device.

[1]  M G Lay,et al.  Connected Car , 2019, Handbuch Digitale Wirtschaft.

[2]  Alec Wolman,et al.  Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming , 2015, MobiSys.

[3]  Dan Wu,et al.  Human respiration detection with commodity wifi devices: do user location and body orientation matter? , 2016, UbiComp.

[4]  Zhenlong Li,et al.  Yawning detection for monitoring driver fatigue based on two cameras , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[5]  Mubarak Shah,et al.  Determining driver visual attention with one camera , 2003, IEEE Trans. Intell. Transp. Syst..

[6]  Luc Van Gool,et al.  Real time head pose estimation with random regression forests , 2011, CVPR 2011.

[7]  Neil Martin Robertson,et al.  Deep Head Pose: Gaze-Direction Estimation in Multimodal Video , 2015, IEEE Transactions on Multimedia.

[8]  Rob Miller,et al.  3D Tracking via Body Radio Reflections , 2014, NSDI.

[9]  B. Tatler,et al.  Steering with the head The visual strategy of a racing driver , 2001, Current Biology.

[10]  Kaishun Wu,et al.  We Can Hear You with Wi-Fi! , 2016, IEEE Trans. Mob. Comput..

[11]  Robin R. Murphy,et al.  Using the Kinect for search and rescue robotics , 2012, 2012 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[12]  Mohan M. Trivedi,et al.  Head Pose Estimation and Augmented Reality Tracking: An Integrated System and Evaluation for Monitoring Driver Awareness , 2010, IEEE Transactions on Intelligent Transportation Systems.

[13]  Parth H. Pathak,et al.  Poster Abstract: Human Tracking and Activity Monitoring Using 60 GHz mmWave , 2016, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[14]  Mohan M. Trivedi,et al.  Looking-in and looking-out vision for Urban Intelligent Assistance: Estimation of driver attentive state and dynamic surround for safe merging and braking , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[15]  Shaojie Tang,et al.  Electronic frog eye: Counting crowd using WiFi , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[16]  Shu Wang,et al.  Acoustic Eavesdropping through Wireless Vibrometry , 2015, MobiCom.

[17]  Shaojie Tang,et al.  Wi-Sleep: Contactless Sleep Monitoring via WiFi Signals , 2014, 2014 IEEE Real-Time Systems Symposium.

[18]  Parth H. Pathak,et al.  Your AP knows how you move: fine-grained device motion recognition through WiFi , 2014, HotWireless@MobiCom.

[19]  J. Cole About face , 1998, Nature.

[20]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[21]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[22]  Yunhao Liu,et al.  Kaleido: You Can Watch It But Cannot Record It , 2015, MobiCom.

[23]  Joseph L. Gabbard,et al.  Behind the Glass: Driver Challenges and Opportunities for AR Automotive Applications , 2014, Proceedings of the IEEE.

[24]  David Wetherall,et al.  Tool release: gathering 802.11n traces with channel state information , 2011, CCRV.

[25]  Björn W. Schuller,et al.  Online Driver Distraction Detection Using Long Short-Term Memory , 2011, IEEE Transactions on Intelligent Transportation Systems.

[26]  Mohan M. Trivedi,et al.  On the design and evaluation of robust head pose for visual user interfaces: algorithms, databases, and comparisons , 2012, AutomotiveUI.

[27]  Richard P. Martin,et al.  Detecting driver phone use leveraging car speakers , 2011, MobiCom.

[28]  Weifeng Liu,et al.  Wi-Fi Radar: Recognizing Human Behavior with Commodity Wi-Fi , 2017, IEEE Communications Magazine.

[29]  Parth H. Pathak,et al.  Monitoring vital signs using millimeter wave , 2016, MobiHoc.

[30]  Ivan Poupyrev,et al.  Soli , 2016, ACM Trans. Graph..

[31]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[32]  Kaishun Wu,et al.  WiFall: Device-free fall detection by wireless networks , 2017, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[33]  Shih-Hau Fang,et al.  Device-free human detection using WiFi signals , 2016, 2016 IEEE 5th Global Conference on Consumer Electronics.

[34]  Yunhao Liu,et al.  Inferring Motion Direction using Commodity Wi-Fi for Interactive Exergames , 2017, CHI.

[35]  Jie Yang,et al.  E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures , 2014, MobiCom.

[36]  Parth H. Pathak,et al.  Human tracking and activity monitoring using 60 GHz mmWave: poster abstract , 2016, IPSN 2016.

[37]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[38]  Henry Jasik,et al.  Antenna engineering handbook , 1961 .

[39]  Tom Minka,et al.  You are facing the Mona Lisa: spot localization using PHY layer information , 2012, MobiSys '12.

[40]  In-So Kweon,et al.  Real-Time Head Orientation from a Monocular Camera Using Deep Neural Network , 2014, ACCV.

[41]  Yunhao Liu,et al.  From RSSI to CSI , 2013, ACM Comput. Surv..

[42]  Nicu Sebe,et al.  Combining Head Pose and Eye Location Information for Gaze Estimation , 2012, IEEE Transactions on Image Processing.

[43]  Luc Van Gool,et al.  Random Forests for Real Time 3D Face Analysis , 2012, International Journal of Computer Vision.

[44]  Allen G. Taylor,et al.  What Is the Microsoft HoloLens , 2016 .

[45]  Luc Van Gool,et al.  Real Time Head Pose Estimation from Consumer Depth Cameras , 2011, DAGM-Symposium.

[46]  Dotan Knaan,et al.  Single image face orientation and gaze detection , 2009, Machine Vision and Applications.

[47]  Riender Happee,et al.  Differences between racing and non-racing drivers: A simulator study using eye-tracking , 2017, PloS one.

[48]  David Chu,et al.  FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization , 2016, MobiSys.

[49]  Holger Regenbrecht,et al.  Towards Pervasive Augmented Reality: Context-Awareness in Augmented Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.

[50]  Sachin Katti,et al.  Position Tracking for Virtual Reality Using Commodity WiFi , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Luke Fletcher,et al.  Driver Inattention Detection based on Eye Gaze—Road Event Correlation , 2009, Int. J. Robotics Res..

[52]  Teruhisa Misu,et al.  Driver Behavior and Performance with Augmented Reality Pedestrian Collision Warning: An Outdoor User Study , 2018, IEEE Transactions on Visualization and Computer Graphics.

[53]  Shaojie Tang,et al.  Contactless Respiration Monitoring Via Off-the-Shelf WiFi Devices , 2016, IEEE Transactions on Mobile Computing.

[54]  Lea Fleischer,et al.  Antenna Engineering Handbook , 2016 .

[55]  Kang G. Shin,et al.  Invisible Sensing of Vehicle Steering with Smartphones , 2015, MobiSys.