Sensing multimedia contexts on mobile devices

We use various multimedia applications on smart devices to consume multimedia content, to communicate with our peers, and to broadcast our events live. This paper investigates the utilization of different media input/output devices, e.g., camera, microphone, and speaker, by different types of multimedia applications, and introduces the notion of multimedia context. Our measurements lead to a sensing algorithm called MediaSense, which senses the states of multiple I/O devices and identifies eleven multimedia contexts of a mobile device in real time. The algorithm distinguishes stored content playback from streaming, live broadcasting from local recording, and conversational multimedia sessions from GSM/VoLTE calls on mobile devices.

[1]  Matti Siekkinen,et al.  Mobile multimedia streaming techniques: QoE and energy saving perspective , 2013, Pervasive Mob. Comput..

[2]  T. Karagiannis,et al.  Challenges in Network Application Identification , 2012 .

[3]  Cédric Westphal,et al.  Network Support for AR/VR and Immersive Video Application: A Survey , 2018, ICETE.

[4]  Michael Seufert,et al.  Streaming Characteristics of Spotify Sessions , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[5]  William J. Buchanan,et al.  Forensics study of IMO call and chat app , 2018, Digit. Investig..

[6]  Dario Rossi,et al.  Revealing skype traffic: when randomness plays with you , 2007, SIGCOMM '07.

[7]  Andrew Reed,et al.  Identifying HTTPS-Protected Netflix Videos in Real-Time , 2017, CODASPY.

[8]  Sasu Tarkoma,et al.  In Situ Network and Application Performance Measurement on Android Devices and the Imperfections , 2020, ArXiv.

[9]  Matti Siekkinen,et al.  Using Viewing Statistics to Control Energy and Traffic Overhead in Mobile Video Streaming , 2016, IEEE/ACM Transactions on Networking.

[10]  Srinivasan Seshan,et al.  Developing a predictive model of quality of experience for internet video , 2013, SIGCOMM.

[11]  Aravind Srinivasan,et al.  Cellular traffic offloading through opportunistic communications: a case study , 2010, CHANTS '10.

[12]  Bo Han,et al.  On the Networking Challenges of Mobile Augmented Reality , 2017, VR/AR Network@SIGCOMM.

[13]  Shijie Sun,et al.  Pytheas: Enabling Data-Driven Quality of Experience Optimization Using Group-Based Exploration-Exploitation , 2017, NSDI.

[14]  David A. Wagner,et al.  Android Permissions Remystified: A Field Study on Contextual Integrity , 2015, USENIX Security Symposium.

[15]  Pradipta De,et al.  Aggregating inter-app traffic to optimize cellular radio energy consumption on smartphones , 2015, 2015 7th International Conference on Communication Systems and Networks (COMSNETS).

[16]  Ranveer Chandra,et al.  Optimizing background email sync on smartphones , 2013, MobiSys '13.

[17]  Ibrahim M. Baggili,et al.  WhatsApp network forensics: Decrypting and understanding the WhatsApp call signaling messages , 2015, Digit. Investig..

[18]  Feng Li,et al.  Silhouette: Identifying YouTube Video Flows from Encrypted Traffic , 2018, NOSSDAV.