Voice Command Fingerprinting with Locality Sensitive Hashes

Smart home speakers are deployed in millions of homes around the world. These speakers enable users to interact with other IoT devices in the household and provide voice assistance such as telling the weather and reminding appointments. Although smart home speakers facilitate many aspects of our life, security and privacy concerns should be analyzed and addressed. In this paper, we show that an attacker sniffing the network traffic of smart speakers can infer voice commands and compromise the privacy of users. Specifically, we propose a method that utilizes the network traffic of the speakers to fingerprint the voice commands of users without a need for extracting traffic features with machine learning algorithms. We evaluated the proposed method on traffic traces of 100 different voice commands on smart home speakers. Our approach correctly infers 42% of voice commands while machine learning models infer 22% to 34%. We also evaluated the effectiveness of the padding method recommended for preventing voice command fingerprinting and observed that the accuracy of proposed fingerprinting method drops down to 15% and accuracy of machine learning methods ranges from 6% to 15% with traffic padding.

[1]  Theophilus A. Benson,et al.  Detecting Volumetric Attacks on loT Devices via SDN-Based Monitoring of MUD Activity , 2019, SOSR.

[2]  Kristen Grauman,et al.  Kernelized locality-sensitive hashing for scalable image search , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[3]  Mehmet Hadi Gunes,et al.  IoT Event Classification Based on Network Traffic , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[4]  Huchuan Lu,et al.  Video anomaly detection based on locality sensitive hashing filters , 2016, Pattern Recognit..

[5]  Shiyan Hu,et al.  Efficient video retrieval by locality sensitive hashing , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[6]  Jeffrey M. Voas,et al.  “Alexa, Can I Trust You?” , 2017, Computer.

[7]  Mehmet Hadi Gunes,et al.  IoT Traffic Flow Identification using Locality Sensitive Hashes , 2020, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[8]  Metin Bilgin,et al.  Sentiment analysis on Twitter data with semi-supervised Doc2Vec , 2017, 2017 International Conference on Computer Science and Engineering (UBMK).

[9]  Xin Rong,et al.  word2vec Parameter Learning Explained , 2014, ArXiv.

[10]  Mehmet Hadi Gunes,et al.  How to Find Hidden Users: A Survey of Attacks on Anonymity Networks , 2015, IEEE Communications Surveys & Tutorials.

[11]  J. Landolin,et al.  Assembling large genomes with single-molecule sequencing and locality-sensitive hashing , 2014, Nature Biotechnology.

[12]  Muhammad N. Marsono,et al.  Packet‐level open‐digest fingerprinting for spam detection on middleboxes , 2012, Int. J. Netw. Manag..

[13]  Ernesto Damiani,et al.  An Open Digest-based Technique for Spam Detection , 2004, PDCS.

[14]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[15]  Thomas Ristenpart,et al.  Peek-a-Boo, I Still See You: Why Efficient Traffic Analysis Countermeasures Fail , 2012, 2012 IEEE Symposium on Security and Privacy.

[16]  Mauro Conti,et al.  Peek-a-boo: i see your smart home activities, even encrypted! , 2018, WISEC.

[17]  Vassil Roussev,et al.  An evaluation of forensic similarity hashes , 2011, Digit. Investig..

[18]  Huy Kang Kim,et al.  Andro-Simnet: Android Malware Family Classification using Social Network Analysis , 2018, 2018 16th Annual Conference on Privacy, Security and Trust (PST).

[19]  Pete Burnap,et al.  A Supervised Intrusion Detection System for Smart Home IoT Devices , 2019, IEEE Internet of Things Journal.

[20]  Mehmet Hadi Gunes,et al.  Automated IoT Device Identification using Network Traffic , 2019, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[21]  Jonathan Oliver,et al.  TLSH -- A Locality Sensitive Hash , 2013, 2013 Fourth Cybercrime and Trustworthy Computing Workshop.

[22]  Haipeng Li,et al.  I Can Hear Your Alexa: Voice Command Fingerprinting on Smart Home Speakers , 2019, 2019 IEEE Conference on Communications and Network Security (CNS).

[23]  Steven Furnell,et al.  Biometrically Linking Document Leakage to the Individuals Responsible , 2018, TrustBus.

[24]  Antônio J. Pinheiro,et al.  Identifying IoT devices and events based on packet length from encrypted traffic , 2019, Comput. Commun..