Revisiting IoT Device Identification

Internet-of-Things (IoT) devices are known to be the source of many security problems, and as such, they would greatly benefit from automated management. This requires robustly identifying devices so that appropriate network security policies can be applied. We address this challenge by exploring how to accurately identify IoT devices based on their network behavior, while leveraging approaches previously proposed by other researchers. We compare the accuracy of four different previously proposed machine learning models (tree-based and neural network-based) for identifying IoT device. We use packet trace data collected over a period of six months from a large IoT test-bed. We show that, while all models achieve high accuracy when evaluated on the same dataset as they were trained on, their accuracy degrades over time, when evaluated on data collected outside the training set. We show that on average the models’ accuracy degrades after a couple of weeks by up to 40 percentage points (on average between 12 and 21 percentage points). We argue that, in order to keep the models’ accuracy at a high level, these need to be continuously updated.

[1]  Andrew W. Moore,et al.  Internet traffic classification using bayesian analysis techniques , 2005, SIGMETRICS '05.

[2]  Grenville J. Armitage,et al.  A survey of techniques for internet traffic classification using machine learning , 2008, IEEE Communications Surveys & Tutorials.

[3]  AnHai Doan,et al.  Chimera: Large-Scale Classification using Machine Learning, Rules, and Crowdsourcing , 2014, Proc. VLDB Endow..

[4]  Gavin Hackeling,et al.  Mastering Machine Learning With scikit-learn , 2014 .

[5]  Ahmad-Reza Sadeghi,et al.  IoT SENTINEL: Automated Device-Type Identification for Security Enforcement in IoT , 2016, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[6]  Yuval Elovici,et al.  ProfilIoT: a machine learning approach for IoT device identification based on network traffic analysis , 2017, SAC.

[7]  Jaime Lloret,et al.  Network Traffic Classifier With Convolutional and Recurrent Neural Networks for Internet of Things , 2017, IEEE Access.

[8]  Nick Feamster,et al.  A Smart Home is No Castle: Privacy Vulnerabilities of Encrypted IoT Traffic , 2017, ArXiv.

[9]  Anthony Brown,et al.  An Analysis of Home IoT Network Traffic and Behaviour , 2018, ArXiv.

[10]  Qiang Li,et al.  Acquisitional Rule-based Engine for Discovering Internet-of-Thing Devices , 2018, USENIX Security Symposium.

[11]  Prateek Jain,et al.  FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network , 2018, NeurIPS.

[12]  Cedric Baudoin,et al.  Towards the Deployment of Machine Learning Solutions in Network Traffic Classification: A Systematic Survey , 2019, IEEE Communications Surveys & Tutorials.

[13]  Samuel Marchal,et al.  DÏoT: A Federated Self-learning Anomaly Detection System for IoT , 2018, 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS).

[14]  Saharon Rosset,et al.  Lossless Compression of Random Forests , 2019, Journal of Computer Science and Technology.

[15]  Zhongwen Guo,et al.  Time Series Data Classification Based on Dual Path CNN-RNN Cascade Network , 2019, IEEE Access.

[16]  Vijay Sivaraman,et al.  Classifying IoT Devices in Smart Environments Using Network Traffic Characteristics , 2019, IEEE Transactions on Mobile Computing.

[17]  Franck Le,et al.  DeviceMien: network device behavior modeling for identifying unknown IoT devices , 2019, IoTDI.

[18]  Qi Li,et al.  Network service dependencies in commodity internet-of-things devices , 2019, IoTDI.

[19]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[20]  Antônio J. Pinheiro,et al.  Identifying IoT devices and events based on packet length from encrypted traffic , 2019, Comput. Commun..

[21]  Behnam Dezfouli,et al.  Image classification on IoT edge devices: profiling and modeling , 2019, Cluster Computing.

[22]  Athina Markopoulou,et al.  PingPong: Packet-Level Signatures for Smart Home Device Events , 2019, ArXiv.

[23]  Omar Alrawi,et al.  SoK: Security Evaluation of Home-Based IoT Deployments , 2019, 2019 IEEE Symposium on Security and Privacy (SP).

[24]  Ralph E. Droms,et al.  Manufacturer Usage Description Specification , 2019, RFC.

[25]  Hamed Haddadi,et al.  Information Exposure From Consumer IoT Devices: A Multidimensional, Network-Informed Measurement Approach , 2019, Internet Measurement Conference.

[26]  Faisal Zaman,et al.  What is TensorFlow Lite , 2020 .

[27]  Vijay Sivaraman,et al.  Progressive Monitoring of IoT Networks Using SDN and Cost-Effective Traffic Signatures , 2020, 2020 Workshop on Emerging Technologies for Security in IoT (ETSecIoT).

[28]  Daniel J. Dubois,et al.  A Haystack Full of Needles: Scalable Detection of IoT Devices in the Wild , 2020, Internet Measurement Conference.

[29]  Paolo Bellavista,et al.  CoLearn: enabling federated learning in MUD-compliant IoT edge networks , 2020, EdgeSys@EuroSys.

[30]  Hamed Haddadi,et al.  The Case for Retraining of ML Models for IoT Device Identification at the Edge , 2020, ArXiv.

[31]  Nor Badrul Anuar,et al.  The rise of traffic classification in IoT networks: A survey , 2020, J. Netw. Comput. Appl..

[32]  David Patterson,et al.  Benchmarking TinyML Systems: Challenges and Direction , 2020, ArXiv.

[33]  Sasu Tarkoma,et al.  IoT-KEEPER: Detecting Malicious IoT Network Activity Using Online Traffic Analysis at the Edge , 2020, IEEE Transactions on Network and Service Management.

[34]  Li Yang,et al.  IoT ETEI: End-to-End IoT Device Identification Method , 2021, 2021 IEEE Conference on Dependable and Secure Computing (DSC).

[35]  Rogier C. van Dalen,et al.  Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications , 2021, ArXiv.