PinchType: Text Entry for Virtual and Augmented Reality Using Comfortable Thumb to Fingertip Pinches

Text entry is an integral component to many use cases in virtual and augmented reality. We present PinchType: A new method of virtual text entry that combines users' existing knowledge of the QWERTY keyboard layout with simple thumb and finger interactions. Users pinch with the thumb and fingertip to select from the same group of letters the finger would press on a QWERTY keyboard; a language model disambiguates. In a preliminary study with 14 participants, we investigated PinchType's speed and accuracy on initial use, as well as its physical comfort relative to a mid-air keyboard. After entering 40 phrases, most people reported that PinchType was more comfortable than the mid-air keyboard. Most participants reached a mean speed of 12.54 WPM, or 20.07 WPM without the time spent correcting errors. This compares favorably to other thumb-to-finger virtual text entry methods.

[1]  Falko Kuester,et al.  Towards keyboard independent touch typing in VR , 2005, VRST '05.

[2]  Yuanchun Shi,et al.  ATK: Enabling Ten-Finger Freehand Typing in Air Based on 3D Hand Tracking Data , 2015, UIST.

[3]  I. Scott MacKenzie,et al.  Phrase sets for evaluating text entry techniques , 2003, CHI Extended Abstracts.

[4]  Doug A. Bowman,et al.  Novel Uses of Pinch Gloves™ for Virtual Environment Interaction Techniques , 2002, Virtual Reality.

[5]  Kasper Hornbæk,et al.  Vulture: a mid-air word-gesture keyboard , 2014, CHI.

[6]  Kenrick Kin,et al.  Online optical marker-based hand tracking with deep labels , 2018, ACM Trans. Graph..

[7]  Anders Markussen,et al.  Selection-Based Mid-Air Text Entry on Large Displays , 2013, INTERACT.

[8]  Feng Chen,et al.  HiFinger: One-Handed Text Entry Technique for Virtual Environments Based on Touches between Fingers , 2019, Sensors.

[9]  Lucy E. Dunne,et al.  Argot: a wearable one-handed keyboard glove , 2014, ISWC '14 Adjunct.

[10]  Jan Krüger,et al.  A reduced QWERTY keyboard for mobile text entry , 2004, CHI EA '04.

[11]  Shwetak N. Patel,et al.  DigiTouch: Reconfigurable Thumb-to-Finger Input and Text Entry on Head-mounted Displays , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[12]  Per Ola Kristensson,et al.  Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[13]  Kening Zhu,et al.  FingerT9: Leveraging Thumb-to-finger Interaction for Same-side-hand Text Entry on Smartwatches , 2018, CHI.