EchoTrack: Acoustic device-free hand tracking on smart phones

This paper explores the limits of acoustic ranging on smart phone in the scenario of device-free hand tracking. Tracking the hand is challenging since it requires continuously locating the moving hand in the air with fine resolution. Existing work on hand tracking relies on special hardware or requires users hold the mobile device. This paper presents EchoTrack, which continuously locates the hand by leveraging mobile audio hardware advances without special infrastructure supported. EchoTrack measures the distance from the hand to the speaker array embedded in smart phone via the chirp's Time of Flight (TOF). The speaker array and hand yield a unique triangle. The hand can be located with this triangular geometry. The trajectory accuracy can be improved with the method of Doppler shift compensation and trajectory correction (i.e., roughness penalty smoothing method). We implement a prototype on smart phone and the evaluation shows that EchoTrack can achieve tracking accuracy within about three centimeters of 76% and two centimeters of 48%.

[1]  Chen Wang,et al.  Dynamic Doppler Frequency Shift Errors: Measurement, Characterization, and Compensation , 2015, IEEE Transactions on Instrumentation and Measurement.

[2]  Desney S. Tan,et al.  SoundWave: using the doppler effect to sense gestures , 2012, CHI.

[3]  Sunghyun Choi,et al.  Chirp signal-based aerial acoustic communication for smart devices , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[4]  Khaled A. Harras,et al.  WiGest demo: A ubiquitous WiFi-based gesture recognition system , 2015, 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[5]  D. Cox.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[6]  David Chu,et al.  On the feasibility of real-time phone-to-phone 3D localization , 2011, SenSys.

[7]  David Chu,et al.  SwordFight: enabling a new class of phone-to-phone action games on commodity phones , 2012, MobiSys '12.

[8]  Yunhao Liu,et al.  Ambient rendezvous: Energy-efficient neighbor discovery via acoustic sensing , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[9]  Li Sun,et al.  WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices , 2015, MobiCom.

[10]  Qian Zhang,et al.  CondioSense: high-quality context-aware service for audio sensing system via active sonar , 2017, Personal and Ubiquitous Computing.

[11]  Kasper Hornbæk,et al.  Tracko: Ad-hoc Mobile 3D Tracking Using Bluetooth Low Energy and Inaudible Signals for Cross-Device Interaction , 2015, UIST.

[12]  Richard P. Martin,et al.  Detecting driver phone use leveraging car speakers , 2011, MobiCom.

[13]  Khaled A. Harras,et al.  Wigest: A Ubiquitous Wifi-based Gesture Recognition System , 2014 .

[14]  Pei Zhang,et al.  Spartacus: Spatially-Aware Interaction for Mobile Devices Through Energy-Efficient Audio Sensing , 2015, GETMBL.

[15]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[16]  Yuan Zhang,et al.  Poster: doppler effect based device-free moving object localization , 2014, MobiCom.

[17]  Desney S. Tan,et al.  FingerIO: Using Active Sonar for Fine-Grained Finger Tracking , 2016, CHI.

[18]  Yunxin Liu,et al.  AMIL: Localizing neighboring mobile devices through a simple gesture , 2016, INFOCOM Workshops.

[19]  Sangki Yun,et al.  Turning a Mobile Device into a Mouse in the Air , 2015, MobiSys.

[20]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[21]  Yu Wang,et al.  EchoLoc: Accurate Device-Free Hand Localization Using COTS Devices , 2016, 2016 45th International Conference on Parallel Processing (ICPP).

[22]  Gang Zhou,et al.  A Software-Based Sonar Ranging Sensor for Smart Phones , 2015, IEEE Internet of Things Journal.

[23]  Guobin Shen,et al.  BeepBeep: a high accuracy acoustic ranging system using COTS mobile devices , 2007, SenSys '07.