A Real Time Wireless Interactive Multimedia System

Recent years, various interactive multimedia systems have been applied to relevant fields such as education, entertainment, etc. Researchers exploit sensors, computer vision, ultrasonic, and electromagnetic radiation to achieve human-computer interaction (HCI). This paper proposes an interactive wireless multimedia system which utilizes ubiquitous wireless signals to identify human motions around smart WiFi devices. Compared with related work, our system realizes interactions between human and computer without extra hardware devices. The system identifies human gestures around the smart devices (i.e., a laptop) equipped with the commercial 802.11n NIC, and it maps different gestures into distinguishable computer instructions. We build a proof-of-concept prototype using off-the-shelf laptop and evaluate the system in a laboratory environment with standard WiFi access points. The results show that our system detects human gesture with an accuracy over 95 % and it achieves an average gesture classification accuracy of 89 % for five different users.

[1]  Li Sun,et al.  WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices , 2015, MobiCom.

[2]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[3]  Kaishun Wu,et al.  We Can Hear You with Wi-Fi! , 2014, IEEE Transactions on Mobile Computing.

[4]  Peter A. Dinda,et al.  Sonar-based measurement of user presence and attention , 2009, UbiComp.

[5]  Wei Wang,et al.  Understanding and Modeling of WiFi Signal Based Human Activity Recognition , 2015, MobiCom.

[6]  Khaled A. Harras,et al.  Wigest: A Ubiquitous Wifi-based Gesture Recognition System , 2014 .

[7]  David Wetherall,et al.  Tool release: gathering 802.11n traces with channel state information , 2011, CCRV.

[8]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[9]  Desney S. Tan,et al.  Skinput: appropriating the body as an input surface , 2010, CHI.

[10]  Patrick Olivier,et al.  Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor , 2012, UIST.

[11]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[12]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[13]  Wei Wang,et al.  Keystroke Recognition Using WiFi Signals , 2015, MobiCom.

[14]  M. Beigl,et al.  Challenges for device-free radio-based activity recognition , 2011 .

[15]  Keith J. Burnham,et al.  A Research Study of Hand Gesture Recognition Technologies and Applications for Human Vehicle Interaction , 2007 .

[16]  Paolo Dario,et al.  A Survey of Glove-Based Systems and Their Applications , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[17]  Ivan E. Sutherland,et al.  Sketch pad a man-machine graphical communication system , 1964, DAC.

[18]  Brad A. Myers,et al.  A brief history of human-computer interaction technology , 1998, INTR.

[19]  Michael Beigl,et al.  Device-free and device-bound activity recognition using radio signal strength , 2013, AH.

[20]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[21]  Parameswaran Ramanathan,et al.  Leveraging directional antenna capabilities for fine-grained gesture recognition , 2014, UbiComp.

[22]  Rob Miller,et al.  3D Tracking via Body Radio Reflections , 2014, NSDI.

[23]  Desney S. Tan,et al.  SoundWave: using the doppler effect to sense gestures , 2012, CHI.

[24]  Desney S. Tan,et al.  Humantenna: using the body as an antenna for real-time whole-body interaction , 2012, CHI.