Looking at you: fused gyro and face tracking for viewing large imagery on mobile devices

We present a touch-free interface for viewing large imagery on mobile devices. In particular, we focus on viewing paradigms for 360 degree panoramas, parallax image sequences, and long multi-perspective panoramas. We describe a sensor fusion methodology that combines face tracking using a front-facing camera with gyroscope data to produce a robust signal that defines the viewer's 3D position relative to the display. The gyroscopic data provides both low-latency feedback and allows extrapolation of the face position beyond the the field-of-view of the front-facing camera. We also demonstrate a hybrid position and rate control that uses the viewer's 3D position to drive exploration of very large image spaces. We report on the efficacy of the hybrid control vs. position only control through a user study.

[1]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[2]  Eva Eriksson,et al.  Use your head: exploring face tracking for mobile interaction , 2006, CHI Extended Abstracts.

[3]  W. Buxton,et al.  Boom chameleon: simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display , 2002, UIST '02.

[4]  Edward Cutrell,et al.  Quantitative analysis of scrolling techniques , 2002, CHI.

[5]  Jan Stage,et al.  Handbook of Research on User Interface Design and Evaluation for Mobile Technology , 2008 .

[6]  B. Bederson,et al.  Understanding Single-Handed Mobile Device Interaction , 2006 .

[7]  Steven K. Feiner,et al.  Content-aware scrolling , 2006, UIST.

[8]  Takeo Igarashi,et al.  Speed-dependent automatic zooming for browsing large documents , 2000, UIST '00.

[9]  Janne Heikkilä,et al.  Face Tracking for Spatially Aware Mobile User Interfaces , 2008, ICISP.

[10]  William Buxton,et al.  Boom chameleon: simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display , 2003, ACM Trans. Graph..

[11]  Eva Eriksson,et al.  Movement-based interaction in camera spaces: a conceptual framework , 2007, Personal and Ubiquitous Computing.

[12]  Richard Szeliski,et al.  Street slide: browsing street level imagery , 2010, ACM Trans. Graph..

[13]  Terry Winograd,et al.  Gaze-enhanced scrolling techniques , 2007, UIST.

[14]  K. Hinckley.  Input technologies and techniques , 2002 .

[15]  Daniel Vogel,et al.  RubberEdge: reducing clutching by combining position and rate control with elastic feedback , 2007, UIST.

[16]  David Salesin,et al.  Parallax photography: creating 3D cinematic effects from stills , 2009, Graphics Interface.

[17]  Steven K. Feiner,et al.  Rubbing and tapping for precise and rapid selection on touch-screen displays , 2008, CHI.

[18]  Monica M. C. Schraefel,et al.  The radial scroll tool: scrolling support for stylus- or touch-based document navigation , 2004, UIST '04.

[19]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[20]  Gang Hua,et al.  PEYE: Toward a Visual Motion Based Perceptual Interface for Mobile Devices , 2007, ICCV-HCI.

[21]  Jose L. Contreras-Vidal,et al.  Understanding One-Handed Use of Mobile Devices , 2008 .

[22]  Stéphane Huot,et al.  TapTap and MagStick: improving one-handed target acquisition on small touch-screens , 2008, AVI '08.

[23]  Eric Lecolinet,et al.  MicroRolls: expanding touch-screen input vocabulary by distinguishing rolls vs. slides of the thumb , 2009, CHI.