A Protocol to Evaluate Virtual Keyboards for Indian Languages

We were asked to evaluate input mechanisms for touch-screen devices with the objective of standardising one of them for each of 14 major languages of India. For this purpose, we propose a protocol that consists of a 45-minute long training session, a 20-word first-time usability test, and a longitudinal test consisting of about 30 sessions, each of which required the user to type about 10 phrases 4 to 6 words long (a total of 300 phrases). The evaluation should be done with school children from standards 4th to 7th. The course of the evaluation may last 2-4 weeks for each user. To help follow the protocol over this long period and to collate the data, we offer a tool. Currently, we provide test corpora for Assamese, Bengali, Gujarati, Hindi, Kannada, Marathi, Odia, Tamil, Telugu, and Urdu. We have ensured that each corpus represents a mix of informal communication between people, popular phrases from films, songs, poetry and public discourse, and formal texts from school books and literature. We have tagged each phrase according to typing difficulty, phrase length, and memorability and age appropriateness. We evaluated the protocol through pilot tests with 206 users in Marathi, Gujarati, Hindi, Bengali, Odia, Assamese and Tamil. In this paper, we present the original protocol, the detailed findings from the Marathi pilots, and the proposed modifications to the protocol.

[1]  Anirudha Joshi,et al.  Text Entry in Indian Languages on Mobile: User Perspectives , 2014, IHCI.

[2]  Anirudha Joshi,et al.  Corpus of Marathi Word Frequencies from Touch-Screen Devices Using Swarachakra Android Keyboard , 2014, IHCI.

[3]  Gilbert Cockton,et al.  CHI '03 Extended Abstracts on Human Factors in Computing Systems , 2003, CHI 2003.

[4]  I. Scott MacKenzie,et al.  Phrase sets for evaluating text entry techniques , 2003, CHI Extended Abstracts.

[5]  Sriram Subramanian,et al.  GesText: accelerometer-based gestural text-entry systems , 2010, CHI.

[6]  Vittorio Fuccella,et al.  Gestures and widgets: performance in text editing on multi-touch capable mobile devices , 2013, CHI.

[7]  Anirudha Joshi,et al.  A Dynamic Text Input scheme for phonetic scripts like Devanagari , 2002 .

[8]  Nagraj Emmadi,et al.  Testing the efficacy of an Indic script virtual keyboard: Swarachakra , 2014, IHCI.

[9]  Anirudha N. Joshi,et al.  Swarachakra keyboard for indic scripts (tutorial) , 2014, MOBILESoft 2014.

[10]  Khai N. Truong,et al.  Disambiguation of imprecise input with one-dimensional rotational text entry , 2014, TCHI.

[11]  Sugata Mitra.  Minimally invasive education: a progress report on the "hole-in-the-wall" experiments , 2003, Br. J. Educ. Technol..

[12]  Dhaval Joshi,et al.  Solving the great Indian text input puzzle: touch screen-based mobile text input design , 2011, Mobile HCI.

[13]  Khai N. Truong,et al.  Evaluating the Effect of Phrase Set in Hindi Text Entry , 2013, INTERACT.

[14]  Sayan Sarcar,et al.  Eyeboard++: an enhanced eye gaze-based text entry system in Hindi , 2013, APCHI.

[15]  Benjamin B. Bederson,et al.  Multi-modal text entry and selection on a mobile device , 2010, Graphics Interface.

[16]  Kalina Bontcheva,et al.  Corpus Linguistics and South Asian Languages: Corpus Creation and Tool Development , 2004, Lit. Linguistic Comput..

[17]  Kasper Hornbæk,et al.  Vulture: a mid-air word-gesture keyboard , 2014, CHI.

[18]  Anirudha Joshi,et al.  Keylekh: a keyboard for text entry in indic scripts , 2004, CHI EA '04.

[19]  Anirudha Joshi,et al.  Design and evaluation of Devanagari virtual keyboards for touch screen mobile phones , 2011, Mobile HCI.

[20]  Pranav Mistry,et al.  AKSHAR: A mechanism for inputting Indic scripts on digital devices , 2007 .

[21]  Jugal K. Kalita,et al.  Design and Evaluation of Soft Keyboards for Brahmic Scripts , 2013, TALIP.

[22]  Manoj Kumar Sharma,et al.  Word Prediction System for Text Entry in Hindi , 2014, ACM Trans. Asian Lang. Inf. Process..