Estimating 3D Finger Angle on Commodity Touchscreens

We describe a novel approach for estimating the pitch and yaw of fingers relative to a touchscreen's surface, offering two additional, analog degrees of freedom for interactive functions. Further, we show that our approach can be achieved on off-the-shelf consumer touchscreen devices: a smartphone and smartwatch. We validate our technique though a user study on both devices and conclude with several demo applications that illustrate the value and immediate feasibility of our approach.

[1]  Geehyuk Lee,et al.  Force gestures: augmenting touch screen gestures with normal and tangential forces , 2011, UIST.

[2]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[3]  Feng Wang,et al.  Empirical evaluation for finger input properties in multi-touch interaction , 2009, CHI.

[4]  Geehyuk Lee,et al.  Force gestures: augmented touch screen gestures using normal and tangential force , 2011, CHI Extended Abstracts.

[5]  Gudrun Klinker,et al.  KinectTouch: accuracy test for a very low-cost 2.5D multitouch tracking system , 2011, ITS '11.

[6]  Ravin Balakrishnan,et al.  Pressure widgets , 2004, CHI.

[7]  Eric Lecolinet,et al.  MicroRolls: expanding touch-screen input vocabulary by distinguishing rolls vs. slides of the thumb , 2009, CHI.

[8]  Chris Harrison,et al.  TapSense: enhancing finger interaction on touch surfaces , 2011, UIST.

[9]  Xiang 'Anthony' Chen,et al.  The fat thumb: using the thumb's contact size for single-handed mobile interaction , 2012, Mobile HCI.

[10]  Maribeth Back,et al.  PointPose: finger pose estimation for touch input on mobile devices using a depth sensor , 2013, ITS.

[11]  Xiang Cao,et al.  ShapeTouch: Leveraging contact shape on interactive surfaces , 2008, 2008 3rd IEEE International Workshop on Horizontal Interactive Human Computer Systems.

[12]  Jun Rekimoto,et al.  Z-touch: an infrastructure for 3d gesture interaction in the proximity of tabletop surfaces , 2010, ITS '10.

[13]  Xiang Cao,et al.  Detecting and leveraging finger orientation for interaction with direct-touch surfaces , 2009, UIST '09.

[14]  Patrick Baudisch,et al.  The generalized perceived input point model and how to double touch accuracy by extracting fingerprints , 2010, CHI.

[15]  Tovi Grossman,et al.  The design and evaluation of multitouch marking menus , 2010, CHI.

[16]  Simon Rogers,et al.  AnglePose: robust, precise capacitive touch tracking via 3d orientation estimation , 2011, CHI.

[17]  Vadim Zaliva.  3D finger posture detection and gesture recognition on touch surfaces , 2012, 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV).