Demo: Turning a Mobile Device into a Mouse in the Air

Motivation: Amouse is widely popular for controlling the graphic user interface due to its ease of use. Its attraction will soon penetrate well beyond just computers. There already have been mice designed for game consoles and smart TVs. A smart TV allows a user to run popular computer programs and smartphone applications. More and more devices in the future, such as Google Glasses, baby monitors, and new generation of home appliances, will all desire mouse functionalities, which allow users to choose from a wide variety of options and easily click on different parts of the view. However, a traditional mouse, which requires a flat and smooth surface to operate, cannot satisfy many new usage scenarios. A user wants to interact with the remote device on the move. For example, a speaker wants to freely move around and click on different objects in his slide; a smart TV user wants to watch TV in any part of a room; a Google Glass user wants to query about objects while touring around. Wouldn’t it be nice if a user can simply turn his smartphone or smart watch into a mouse by moving it in the air?

[1]  Swarun Kumar,et al.  Bringing cross-layer MIMO to today's wireless LANs , 2013, SIGCOMM.

[2]  Hari Balakrishnan,et al.  Tracking moving devices with the cricket location system , 2004, MobiSys '04.

[3]  Guobin Shen,et al.  BeepBeep: a high accuracy acoustic ranging system using COTS mobile devices , 2007, SenSys '07.

[4]  Yunhao Liu,et al.  Shake and walk: Acoustic direction finding and fine-grained indoor localization using smartphones , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[5]  Inseok Hwang,et al.  E-Gesture: a collaborative architecture for energy-efficient gesture recognition with hand-worn sensor and mobile devices , 2011, SenSys.

[6]  Swarun Kumar,et al.  Accurate indoor localization with zero start-up cost , 2014, MobiCom.

[7]  Shwetak N. Patel,et al.  AirLink: sharing files between multiple devices using in-air gestures , 2014, UbiComp.

[8]  Lei Yang,et al.  Tagoram: real-time tracking of mobile RFID tags to high precision using COTS devices , 2014, MobiCom.

[9]  Sangki Yun,et al.  Multi-point to multi-point MIMO in wireless LANs , 2013, 2013 Proceedings IEEE INFOCOM.

[10]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[11]  Pei Zhang,et al.  Spartacus: Spatially-Aware Interaction for Mobile Devices Through Energy-Efficient Audio Sensing , 2015, GETMBL.

[12]  Anshul Rai,et al.  Zee: zero-effort crowdsourcing for indoor localization , 2012, Mobicom '12.

[13]  Eric C. Larson,et al.  DopLink: using the doppler effect for multi-device interaction , 2013, UbiComp.

[14]  E. Jacobsen,et al.  The sliding DFT , 2003, IEEE Signal Process. Mag..

[15]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[16]  Monika Eisenhower,et al.  Encyclopedia Of Physical Science And Technology , 2016 .

[17]  D. Katabi,et al.  MegaMIMO : Scaling Wireless Capacity with User Demands , 2012 .

[18]  Emiko Charbonneau,et al.  The Wiimote and Beyond: Spatially Convenient Devices for 3D User Interfaces , 2010, IEEE Computer Graphics and Applications.

[19]  Desney S. Tan,et al.  SoundWave: using the doppler effect to sense gestures , 2012, CHI.

[20]  Dina Katabi,et al.  Beyond the bits: cooperative packet recovery using physical layer information , 2007, MobiCom '07.

[21]  Bernd Eggers,et al.  Encyclopedia Of Physical Science And Technology , 2016 .

[22]  Wolfram Burgard,et al.  Monte Carlo localization for mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[23]  Kun-Ming Yu,et al.  Automatic Music Genre Classification Based on Modulation Spectral Analysis of Spectral and Cepstral Features , 2009, IEEE Transactions on Multimedia.

[24]  David Chu,et al.  SwordFight: enabling a new class of phone-to-phone action games on commodity phones , 2012, MobiSys '12.

[25]  Feng Zhao,et al.  A reliable and accurate indoor localization method using phone inertial sensors , 2012, UbiComp.

[26]  David Chu,et al.  On the feasibility of real-time phone-to-phone 3D localization , 2011, SenSys.

[27]  Dina Katabi,et al.  RF-IDraw: virtual touch screen in the air using RF signals , 2014, S3 '14.

[28]  Kevin Townsend,et al.  Bluetooth low energyをはじめよう , 2015 .

[29]  Romit Roy Choudhury,et al.  Using mobile phones to write in air , 2011, MobiSys '11.

[30]  Anthony Rowe,et al.  Indoor pseudo-ranging of mobile devices using ultrasonic chirps , 2012, SenSys '12.

[31]  Jie Xiong,et al.  ArrayTrack: A Fine-Grained Indoor Location System , 2011, NSDI.

[32]  Fredrik Gustafsson,et al.  A high-performance tracking system based on camera and IMU , 2013, Proceedings of the 16th International Conference on Information Fusion.

[33]  Jue Wang,et al.  Dude, where's my card?: RFID positioning that works with multipath and non-line of sight , 2013, SIGCOMM.

[34]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .