Smartphone camera based pointer

Large screen displays are omnipresent today as a part of infrastructure for presentations and entertainment. Also powerful smartphones with integrated camera(s) are ubiquitous. However, there are not many ways in which smartphones and screens can interact besides casting the video from a smartphone. In this paper, we present a novel idea that turns a smartphone into a direct virtual pointer on the screen using the phone's camera. The idea and its implementation are simple, robust, efficient and fun to use. Besides the mathematical concepts of the idea we accompany the paper with a small javascript project (this http URL) which demonstrates the possibility of the new interaction technique presented as a massive multiplayer game in the HTML5 framework.

[1]  Adam Herout,et al.  PClines — Line detection using parallel coordinates , 2011, CVPR 2011.

[2]  Andreas Butz,et al.  Touch projector: mobile interaction through video , 2010, CHI.

[3]  Daniel Jackson,et al.  Smart Phone Interaction with Registered Displays , 2009, IEEE Pervasive Computing.

[4]  Olli Silvén,et al.  Interactive multi-frame reconstruction for mobile devices , 2012, Multimedia Tools and Applications.

[5]  Marko Jurmu,et al.  Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays , 2009, OZCHI.

[6]  Michael Rohs,et al.  Sweep and point and shoot: phonecam-based interactions for large public displays , 2005, CHI Extended Abstracts.

[7]  Sebastian Boring,et al.  Making public displays interactive everywhere , 2013, IEEE Computer Graphics and Applications.

[8]  Michael Rohs,et al.  The smart phone: a ubiquitous input device , 2006, IEEE Pervasive Computing.

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Rahul Sukthankar,et al.  AutomaticKeystone Correction for Camera-Assisted Presentation Interfaces , 2000, ICMI.

[11]  Hao Jiang,et al.  Direct pointer: direct manipulation for large-display interaction using handheld cameras , 2006, CHI.

[12]  Daniel Jackson,et al.  Display Registration for Device Interaction - a Proof of Principle Prototype , 2008, VISAPP.

[13]  Daniel C. Alexander.  Advances in daylight statistical colour modelling , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[14]  Andreas Butz,et al.  Multi-user interaction on media facades through live video on mobile devices , 2011, CHI.