We Can Hear You with Wi-Fi!

Recent literature advances Wi-Fi signals to “see” people's motions and locations. This paper asks the following question: Can Wi-Fi “hear” our talks? We present WiHear, which enables Wi-Fi signals to “hear” our talks without deploying any devices. To achieve this, WiHear needs to detect and analyze fine-grained radio reflections from mouth movements. WiHear solves this micro-movement detection problem by introducing Mouth Motion Profile that leverages partial multipath effects and wavelet packet transformation. Since Wi-Fi signals do not require line-of-sight, WiHear can “hear” people talks within the radio range. Further, WiHear can simultaneously “hear” multiple people's talks leveraging MIMO technology. We implement WiHear on both USRP N210 platform and commercial Wi-Fi infrastructure. Results show that within our pre-defined vocabulary, WiHear can achieve detection accuracy of 91 percent on average for single individual speaking no more than six words and up to 74 percent for no more than three people talking simultaneously. Moreover, the detection accuracy can be further improved by deploying multiple receivers from different angles.

[1]  Ronald R. Coifman,et al.  Local discriminant bases and their applications , 1995, Journal of Mathematical Imaging and Vision.

[2]  Theodore S. Rappaport,et al.  Wireless Communications: Principles and Practice (2nd Edition) by , 2012 .

[3]  Neal Patwari,et al.  Radio Tomographic Imaging with Wireless Networks , 2010, IEEE Transactions on Mobile Computing.

[4]  G. Charvat,et al.  A Through-Dielectric Radar Imaging System , 2010, IEEE Transactions on Antennas and Propagation.

[5]  Lu Wang,et al.  Pilot: Passive Device-Free Indoor Localization Using Channel State Information , 2013, 2013 IEEE 33rd International Conference on Distributed Computing Systems.

[6]  Robert P. W. Duin,et al.  Bagging, Boosting and the Random Subspace Method for Linear Classifiers , 2002, Pattern Analysis & Applications.

[7]  D. O'Shaughnessy,et al.  Linear predictive coding , 1988, IEEE Potentials.

[8]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[9]  Phil D. Green,et al.  Robust automatic speech recognition with missing and unreliable acoustic data , 2001, Speech Commun..

[10]  Torbjørn Svendsen,et al.  On the automatic segmentation of speech signals , 1987, ICASSP '87. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Desney S. Tan,et al.  Humantenna: using the body as an antenna for real-time whole-body interaction , 2012, CHI.

[12]  Lu Wang,et al.  FIMD: Fine-grained Device-free Motion Detection , 2012, 2012 IEEE 18th International Conference on Parallel and Distributed Systems.

[13]  Paul Congdon,et al.  Avoiding multipath to revive inbuilding WiFi localization , 2013, MobiSys '13.

[14]  Frédo Durand,et al.  The visual microphone , 2014, ACM Trans. Graph..

[15]  Amir Masoud Rahmani,et al.  COAST: Context-aware pervasive speech recognition system , 2011, International Symposium on Wireless and Pervasive Computing.

[16]  James R. Williams,et al.  Guidelines for the Use of Multimedia in Instruction , 1998 .

[17]  Minyi Guo,et al.  TASA: Tag-Free Activity Sensing Using RFID Tag Arrays , 2011, IEEE Transactions on Parallel and Distributed Systems.

[18]  Alexander H. Waibel,et al.  Toward movement-invariant automatic lip-reading and speech recognition , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[19]  Deng Cai,et al.  Unsupervised feature selection for multi-cluster data , 2010, KDD.

[20]  L. J. Chu.  Physical Limitations of Omni‐Directional Antennas , 1948 .

[21]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[22]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[23]  Lawrence Wai-Choong Wong,et al.  Indoor localization with channel impulse response based fingerprint and nonparametric regression , 2010, IEEE Transactions on Wireless Communications.

[24]  Tom Minka,et al.  You are facing the Mona Lisa: spot localization using PHY layer information , 2012, MobiSys '12.

[25]  Voon Chin Phua,et al.  Wireless lan medium access control (mac) and physical layer (phy) specifications , 1999 .

[26]  M. Beigl,et al.  Challenges for device-free radio-based activity recognition , 2011 .

[27]  David Taylor.  Hearing by Eye: The Psychology of Lip-Reading , 1988 .

[28]  Shyamnath Gollakota,et al.  Bringing Gesture Recognition to All Devices , 2014, NSDI.

[29]  Desney S. Tan,et al.  Skinput: appropriating the body as an input surface , 2010, CHI.

[30]  Sneha Kumar Kasera,et al.  Advancing wireless link signatures for location distinction , 2008, MobiCom '08.

[31]  Yunhao Liu,et al.  From RSSI to CSI , 2013, ACM Comput. Surv..

[32]  Kate Ching-Ju Lin,et al.  Random access heterogeneous MIMO networks , 2011, SIGCOMM.

[33]  Theodore S. Rappaport,et al.  Wireless communications - principles and practice , 1996 .

[34]  Moustafa Youssef,et al.  CoSDEO 2016 Keynote: A decade later — Challenges: Device-free passive localization for wireless environments , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[35]  Romit Roy Choudhury,et al.  Using mobile phones to write in air , 2011, MobiSys '11.

[36]  Sachin Katti,et al.  Full duplex backscatter , 2013, HotNets.

[37]  Xiang-Yang Li,et al.  You're driving and texting: detecting drivers using personal smart phones by leveraging inertial sensors , 2013, MobiCom.

[38]  Eric C. Larson,et al.  HeatWave: thermal imaging for surface user interaction , 2011, CHI.

[39]  Rob Miller,et al.  3D Tracking via Body Radio Reflections , 2014, NSDI.

[40]  Tsuhan Chen,et al.  Profile View Lip Reading , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[41]  Jiawei Han,et al.  SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis , 2008, IEEE Transactions on Knowledge and Data Engineering.

[42]  Lei Yang,et al.  3D beamforming for wireless data centers , 2011, HotNets-X.

[43]  Jue Wang,et al.  Dude, where's my card?: RFID positioning that works with multipath and non-line of sight , 2013, SIGCOMM.

[44]  Alexander H. Waibel,et al.  See Me, Hear Me: Integrating Automatic Speech Recognition and Lip-reading , 1994 .

[45]  Dong Chao,et al.  Universal Software Radio Peripheral , 2010 .

[46]  David Wetherall,et al.  Predictable 802.11 packet delivery from wireless channel measurements , 2010, SIGCOMM '10.

[47]  Ben Y. Zhao,et al.  Mirror mirror on the ceiling: flexible wireless links for data centers , 2012, CCRV.

[48]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[49]  Gerrit Beldman,et al.  Lan medium access control (mac) and physical layer (phy) specifications , 1997 .