Robust Network-Based Binary-to-Vector Encoding for Scalable IoT Binary File Retrieval

The goal of IoT binary file retrieval is to retrieve homologous binary files from a large IoT binary file database. Binary file retrieval has many applications, such as security analysis, OEM detection and plagiarism detection. However, traditional string-based approaches are hard to retrieve binary file which contains few or obfuscated strings. To solve this problem, we propose a novel neural network-based approach for encoding binary file into numerical vector based on non-string binary features. Moreover, by using this encoding method, the retrieval task can be accelerated by locality-sensitive hashing technique. For network training and testing, we compile 893 open source components into 71,129 labeled binary file pairs by using 16 different compilation configurations. We implement a prototype called B2V and compare it with IHB, a string-based approach, on both original and string obfuscated testing sets. The results show that the AUC of B2V is better than IHB (0.94 vs. 0.81) on the string obfuscated testing set, while still keeps comparable performance with IHB on the original testing set. Moreover, B2V can be easily retrained to adapt to string obfuscated scenarios with 15%–20% performance improvement. In the interest of open science, we also make our dataset publicly available to seed future improvements.