SilentKey: A New Authentication Framework through Ultrasonic-based Lip Reading

This paper presents SilentKey, a new authentication framework to identify mobile device users through ultrasonic-based lip reading. The main idea is to generate ultrasonic signals from a mobile phone and analyze the fine-grained impact of mouth motions on the reflected signal. The new framework is effective since people have unique characteristics when performing mouth motions, which represent not only what people input, but also how they input. SilentKey is robust against attacks since the input cannot be recorded or imitated. We implement a prototype and demonstrate the effectiveness of the system by fifty volunteers. Such a non-intrusive identification mechanism provides a natural user interface which can also be applied by people with speaking or viewing difficulties.

[1]  Javid Taheri,et al.  SparseDTW: A Novel Approach to Speed up Dynamic Time Warping , 2009, AusDM.

[2]  Andreas P. Heiner,et al.  A closer look at recognition-based graphical passwords on mobile devices , 2010, SOUPS.

[3]  Adam J. Aviv,et al.  Smudge Attacks on Smartphone Touch Screens , 2010, WOOT.

[4]  Bhiksha Raj,et al.  Ultrasonic Doppler Sensing in HCI , 2012, IEEE Pervasive Computing.

[5]  Bhiksha Raj,et al.  One-handed gesture recognition using ultrasonic Doppler sonar , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Rachael-Anne Knight.  Phonetics: Putting sounds together , 2012 .

[7]  Philip Chan,et al.  Toward accurate dynamic time warping in linear time and space , 2007, Intell. Data Anal..

[8]  Ton Kalker,et al.  Voice activity detection and speaker localization using audiovisual cues , 2012, Pattern Recognit. Lett..

[9]  Dawn Xiaodong Song,et al.  Touchalytics: On the Applicability of Touchscreen Input as a Behavioral Biometric for Continuous Authentication , 2012, IEEE Transactions on Information Forensics and Security.

[10]  Rachael-Anne Knight.  Phonetics: Sounds and symbols , 2012 .

[11]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[12]  Alex X. Liu,et al.  Secure unlocking of mobile touch screen devices by simple gestures: you can see it but you can not do it , 2013, MobiCom.

[13]  Eamonn J. Keogh,et al.  Experimental comparison of representation methods and distance measures for time series data , 2010, Data Mining and Knowledge Discovery.

[14]  Tsutomu Terada,et al.  Ultrasound-based movement sensing, gesture-, and context-recognition , 2013, ISWC '13.

[15]  SalvadorStan,et al.  Toward accurate dynamic time warping in linear time and space , 2007 .

[16]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[17]  Wei Wang,et al.  Device-free gesture tracking using acoustic signals , 2016, MobiCom.

[18]  Kaishun Wu,et al.  We Can Hear You with Wi-Fi! , 2016, IEEE Trans. Mob. Comput..

[19]  Khaled H. Hamed,et al.  Time-frequency analysis , 2003 .

[20]  Xiang-Yang Li,et al.  SilentSense: silent user identification via touch and movement behavioral biometrics , 2013, MobiCom.

[21]  Jacob O. Wobbrock,et al.  TapSongs: tapping rhythm-based passwords on a single binary sensor , 2009, UIST '09.

[22]  Shyamnath Gollakota,et al.  Contactless Sleep Apnea Detection on Smartphones , 2015, GetMobile Mob. Comput. Commun..

[23]  Kartik Muralidharan,et al.  Putting ‘pressure’ on mobile authentication , 2014, 2014 Seventh International Conference on Mobile Computing and Ubiquitous Networking (ICMU).

[24]  Michael Weber,et al.  Password entry usability and shoulder surfing susceptibility on different smartphone platforms , 2012, MUM.

[25]  Cam-Tu Nguyen,et al.  SilentTalk: Lip reading through ultrasonic sensing on mobile phones , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[26]  Desney S. Tan,et al.  SoundWave: using the doppler effect to sense gestures , 2012, CHI.

[27]  Xiang-Yang Li,et al.  Continuous user identification via touch and movement behavioral biometrics , 2014, 2014 IEEE 33rd International Performance Computing and Communications Conference (IPCCC).

[28]  James R. Williams,et al.  Guidelines for the Use of Multimedia in Instruction , 1998 .

[29]  Davrondzhon Gafurov,et al.  A Survey of Biometric Gait Recognition: Approaches, Security and Challenges , 2007 .

[30]  Gierad Laput,et al.  Thumprint: Socially-Inclusive Local Group Authentication Through Shared Secret Knocks , 2017, CHI.

[31]  Nasir D. Memon,et al.  Biometric-rich gestures: a novel approach to authentication on multi-touch devices , 2012, CHI.

[32]  Shwetak N. Patel,et al.  AirLink: sharing files between multiple devices using in-air gestures , 2014, UbiComp.

[33]  Yilong Yin,et al.  Semi-supervised Gait Recognition Based on Self-Training , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[34]  Xinyu Zhang,et al.  mTrack: High-Precision Passive Tracking Using Millimeter Wave Radios , 2015, MobiCom.

[35]  Gary M. Weiss,et al.  Cell phone-based biometric identification , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[36]  Eric C. Larson,et al.  DopLink: using the doppler effect for multi-device interaction , 2013, UbiComp.

[37]  Gopal K. Gupta,et al.  Identity authentication based on keystroke latencies , 1990, Commun. ACM.

[38]  Wei Wang,et al.  Gait recognition using wifi signals , 2016, UbiComp.

[39]  Marc Langheinrich,et al.  Back-of-device authentication on smartphones , 2013, CHI.

[40]  Shridatt Sugrim,et al.  User-generated free-form gestures for authentication: security and memorability , 2014, MobiSys.

[41]  Thomas Wennekers,et al.  Gait-based person and gender recognition using micro-doppler signatures , 2011, 2011 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[42]  Kirsi Helkala,et al.  Biometric Gait Authentication Using Accelerometer Sensor , 2006, J. Comput..