LightSpy: Optical eavesdropping on displays using light sensors on mobile devices

Light emanations from flat-panel displays are a side channel hinting towards the displayed content. Optical eavesdropping requires sensors in the proximity of such displays, necessitating physical access to the the target's environment. This requirement may be eliminated by exploiting the light sensor on the target's mobile device, though there are significant challenges. Such sensors measure one-dimensional light intensity, provide no chromatic information, and have very low sampling rate (normally up to 10Hz). In this paper, we demonstrate that in spite of these challenges, it is possible — based on intensity measurements from a mobile device's light sensor — to make quality inferences regarding the displayed content. We do so by selecting features of measured light that capture information related to transitions between samples. Such features are resilient to ambient noise. In our experiments, involving over 60 hours of collected data and 140 movie clips, we were able to (i) classify content into categories (game, movie, etc) with approximately 90% and 70% accuracy for two-class and four-class classification, respectively; and (ii) identify specific movies or TV programs being played with > 85% accuracy. These findings suggest that access to raw light-sensor readings, which can currently be done without special access controls, may carry nontrivial security ramifications.

[1]  Hao Chen,et al.  TouchLogger: Inferring Keystrokes on Touch Screen from Smartphone Motion , 2011, HotSec.

[2]  Vitaly Shmatikov,et al.  Robust De-anonymization of Large Sparse Datasets , 2008, 2008 IEEE Symposium on Security and Privacy (sp 2008).

[3]  Carsten Kleiner,et al.  Using Mobile Devices with BYOD , 2013, Int. J. Web Portals.

[4]  Avery Wang,et al.  An Industrial Strength Audio Search Algorithm , 2003, ISMIR.

[5]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[6]  Markus G. Kuhn,et al.  Optical time-domain eavesdropping risks of CRT displays , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[7]  Jan-Michael Frahm,et al.  Seeing double: reconstructing obscured typed input from repeated compromising reflections , 2013, CCS.

[8]  Hidema Tanaka,et al.  Information Leakage Via Electromagnetic Emanations and Evaluation of Tempest Countermeasures , 2007, ICISS.

[9]  Rakesh Agrawal,et al.  Keyboard acoustic emanations , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[10]  Romit Roy Choudhury,et al.  Tapprints: your finger taps have fingerprints , 2012, MobiSys '12.

[11]  Ishwar K. Sethi,et al.  Classification of general audio data for content-based retrieval , 2001, Pattern Recognit. Lett..

[12]  Martin Vuagnoux,et al.  Compromising Electromagnetic Emanations of Wired and Wireless Keyboards , 2009, USENIX Security Symposium.

[13]  Adi Shamir,et al.  RSA Key Extraction via Low-Bandwidth Acoustic Cryptanalysis , 2014, CRYPTO.

[14]  M. G. Kuhn,et al.  Compromising emanations of LCD TV sets , 2011, 2011 IEEE International Symposium on Electromagnetic Compatibility.

[15]  Barbara J. Wilson.  Media and Children's Aggression, Fear, and Altruism , 2008, The Future of children.

[16]  Shwetak N. Patel,et al.  Televisions, video privacy, and powerline electromagnetic interference , 2011, CCS '11.

[17]  G. Turin,et al.  An introduction to matched filters , 1960, IRE Trans. Inf. Theory.

[18]  Sinziana Mazilu,et al.  Low-Power Ambient Sensing in Smartphones for Continuous Semantic Localization , 2013, AmI.

[19]  W. V. Eck.  Electromagnetic Radiation from Video Display Units: An Eavesdropping Risk? , 1996 .

[20]  Peng Liu,et al.  How Your Phone Camera Can Be Used to Stealthily Spy on You: Transplantation Attacks against Android Camera Service , 2015, CODASPY.

[21]  Patrick Traynor,et al.  (sp)iPhone: decoding vibrations from nearby keyboards using mobile phone accelerometers , 2011, CCS '11.

[22]  Gabi Nakibly,et al.  PowerSpy: Location Tracking Using Mobile Device Power Analysis , 2015, USENIX Security Symposium.

[23]  Raphael Spreitzer,et al.  PIN Skimming: Exploiting the Ambient-Light Sensor in Mobile Devices , 2014, SPSM@CCS.

[24]  Gabi Nakibly,et al.  Gyrophone: Recognizing Speech from Gyroscope Signals , 2014, USENIX Security Symposium.

[25]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[26]  Ulrich Greveler,et al.  Multimedia Content Identification Through Smart Meter Power Usage Profiles , 2012 .

[27]  Jun Han,et al.  ACComplice: Location inference using accelerometers on smartphones , 2012, 2012 Fourth International Conference on Communication Systems and Networks (COMSNETS 2012).

[28]  David A. Umphress,et al.  Information leakage from optical emanations , 2002, TSEC.

[29]  Markus G. Kuhn,et al.  Electromagnetic Eavesdropping Risks of Flat-Panel Displays , 2004, Privacy Enhancing Technologies.