CondioSense: high-quality context-aware service for audio sensing system via active sonar

Audio sensing has been applied in various mobile applications for sensing personal and environmental information to improve user’s life quality. However, the quality of audio sensing is distorted seriously, while the sensing service is working in incorrect context or the ability of the acoustic sensing is limited (i.e., aging effect of the microphone or interference due to hand covering). To address this challenge, we present CondioSense, a CONtext-aware service for auDIO SENSing system, which identifies the current phone context (i.e., pocket, bag, car, indoor and outdoor) and detects the microphone sensing states. The main idea behind context detection is to extract multipath features from actively generated acoustic signal to identify various contexts since the space size and material among various contexts is different. The sound of physical vibration is explored on microphone sensing state detection, by leveraging that the frequency response of recorded vibration sound changes when the signal propagation in the air is blocked with the microphone covered. We prototype CondioSense on smartphones as an application and perform extensive evaluations. It offers the possibility to recognize various phone contexts with an accuracy exceeding $$92\%$$92% and the accuracy of microphone sensing states detection exceeding $$90\%$$90%.

[1]  Desney S. Tan,et al.  FingerIO: Using Active Sonar for Fine-Grained Finger Tracking , 2016, CHI.

[2]  Mahesh K. Marina,et al.  A semi-supervised learning approach for robust indoor-outdoor detection with smartphones , 2014, SenSys.

[3]  Yu Wang,et al.  SmartLoc: sensing landmarks silently for smartphone-based metropolitan localization , 2016, EURASIP J. Wirel. Commun. Netw..

[4]  Jun Li,et al.  Crowd++: unsupervised speaker count with smartphones , 2013, UbiComp.

[5]  Yu Wang,et al.  Enhancing participant selection through caching in mobile crowd sensing , 2016, 2016 IEEE/ACM 24th International Symposium on Quality of Service (IWQoS).

[6]  Kang G. Shin,et al.  Expansion of Human-Phone Interface By Sensing Structure-Borne Sound Propagation , 2016, MobiSys.

[7]  Sangki Yun,et al.  Turning a Mobile Device into a Mouse in the Air , 2015, MobiSys.

[8]  Yunxin Liu,et al.  AMIL: Localizing neighboring mobile devices through a simple gesture , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[9]  Yunxin Liu,et al.  AMIL: Localizing neighboring mobile devices through a simple gesture , 2016, INFOCOM Workshops.

[10]  Igor Bisio,et al.  A simple ultrasonic Indoor/Outdoor detector for mobile devices , 2015, 2015 International Wireless Communications and Mobile Computing Conference (IWCMC).

[11]  Guobin Shen,et al.  BeepBeep: a high accuracy acoustic ranging system using COTS mobile devices , 2007, SenSys '07.

[12]  Yingshu Li,et al.  Using crowdsourced data in location-based social networks to explore influence maximization , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[13]  Anthony Rowe,et al.  Indoor pseudo-ranging of mobile devices using ultrasonic chirps , 2012, SenSys '12.

[14]  Konrad P. Kording,et al.  Journal of Neuroscience Methods , 2013 .

[15]  David Chu,et al.  SwordFight: enabling a new class of phone-to-phone action games on commodity phones , 2012, MobiSys '12.

[16]  Mo Li,et al.  IODetector: a generic service for indoor outdoor detection , 2012, SenSys '12.

[17]  Xinyu Zhang,et al.  Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization , 2014, MobiSys.

[18]  Jun Yang,et al.  Efficient in-pocket detection with mobile phones , 2013, UbiComp.

[19]  Richard P. Martin,et al.  Detecting driver phone use leveraging car speakers , 2011, MobiCom.

[20]  Romit Roy Choudhury,et al.  Ripple II: Faster Communication through Physical Vibration , 2016, NSDI.

[21]  Desney S. Tan,et al.  SoundWave: using the doppler effect to sense gestures , 2012, CHI.

[22]  Romit Roy Choudhury,et al.  Ripple: Communicating through Physical Vibration , 2015, NSDI.

[23]  Yuan Zhang,et al.  Poster: doppler effect based device-free moving object localization , 2014, MobiCom.

[24]  Frank Englert,et al.  Do you hear what I hear? Using acoustic probing to detect smartphone locations , 2014, 2014 IEEE International Conference on Pervasive Computing and Communication Workshops (PERCOM WORKSHOPS).

[25]  Pei Zhang,et al.  Spartacus: spatially-aware interaction for mobile devices through energy-efficient audio sensing , 2013, MobiSys '13.

[26]  Sang Hyuk Son,et al.  Poster: Are you driving?: non-intrusive driver detection using built-in smartphone sensors , 2014, MobiCom.

[27]  Xiang-Yang Li,et al.  Detecting Driver’s Smartphone Usage via Nonintrusively Sensing Driving Dynamics , 2017, IEEE Internet of Things Journal.

[28]  Gerhard Tröster,et al.  AmbientSense: A real-time ambient sound recognition system for smartphones , 2013, 2013 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[29]  Kang G. Shin,et al.  EchoTag: Accurate Infrastructure-Free Indoor Location Tagging with Smartphones , 2015, MobiCom.

[30]  Qian Zhang,et al.  Touch-and-guard: secure pairing through hand resonance , 2016, UbiComp.

[31]  Nitesh Saxena,et al.  Vibreaker: Securing Vibrational Pairing with Deliberate Acoustic Noise , 2016, WISEC.

[32]  Chen Wang,et al.  Fine-grained sleep monitoring: Hearing your breathing with smartphones , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[33]  Emiliano Miluzzo,et al.  Crowdsensing the speaker count in the wild: implications and applications , 2014, IEEE Communications Magazine.

[34]  David Chu,et al.  On the feasibility of real-time phone-to-phone 3D localization , 2011, SenSys.

[35]  Wen Hu,et al.  Ear-phone: an end-to-end participatory urban noise mapping system , 2010, IPSN '10.

[36]  Yu Wang,et al.  EchoLoc: Accurate Device-Free Hand Localization Using COTS Devices , 2016, 2016 45th International Conference on Parallel Processing (ICPP).

[37]  René Mayrhofer,et al.  Towards device-to-user authentication: protecting against phishing hardware by ensuring mobile device authenticity using vibration patterns , 2015, MUM.

[38]  Rajkumar Darbar,et al.  SurfaceSense: Smartphone Can Recognize Where It Is Kept , 2015, IndiaHCI.

[39]  Jie Yang,et al.  Snooping Keystrokes with mm-level Audio Ranging on a Single Phone , 2015, MobiCom.

[40]  Yunhao Liu,et al.  Ambient rendezvous: Energy-efficient neighbor discovery via acoustic sensing , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).