Your clicks reveal your secrets: a novel user-device linking method through network and visual data

Cameras for visual surveillance are extensively deployed to monitor people’s locations and activities. The law enforcement can analyze the surveillance videos (V-data) to track the whereabouts of the criminal suspects. On the other hand, with the popular use of the mobile phones and a wide coverage of wireless networks, people can easily access the Internet. The law enforcement also need to analyze the network traffic (N-data) to track the device so as to monitor the criminal suspects’ online behaviors. In order to match the suspects’ online and offline behaviors, the key problem is to link the device and its user. In this paper, we present a novel method to link the target with his mobile device by analyzing the N-V data. We use a camera and a wireless access point to monitor people operating their mobile devices in public places such as bars, shopping malls, or similar gathering places. Our user-device linking method is based on the premise that when a user is playing an app, his click activities can generate particular network traffic packets in a short time. Based on this premise, our research is carried out as follows. First, we design experiments to detect the particular packets and figure out the time gap distribution between the user’s clicks and these packets. Through statistical work, we find that for 97.4% of all instances, the time gap is less than 0.5 s. Then we choose five popular social networking apps to evaluate our method. We find that the main impact factors on the experimental results are the different user’s habits and the app’s category. Finally, by simulating two real-world scenarios in which people use different apps, we verify the effectiveness of the linking method. Both in scenario 1 and 2, the accuracy rate of experimental results reaches about 94% when the participants include 5 persons and exceeds 84% in experiments including 10 persons, with the fastest linking speed achieved in 20 s.