Evaluation of IVR data collection UIs for untrained rural users

Due to the rapid spread of mobile phones and coverage in the developing world, mobile phones are being increasingly used as a technology platform for developing-world applications including data collection. In order to reach the vast majority of mobile phone users without access to specialized software, applications must make use of interactive voice response (IVR) UIs. However, it is unclear whether rural users in the developing world can use such UIs without prior training or IVR experience; and if so, what UI design choices improve usability for these target populations. This paper presents the results of a real-world deployment of an IVR application for collecting feedback from teachers in rural Uganda. Automated IVR data collection calls were delivered to over 150 teachers over a period of several months. Modifications were made to the IVR interface throughout the study period in response to user interviews and recorded transcripts of survey calls. Significant differences in task success rate were observed for different interface designs (from 0% to over 75% success). Notably, most participants were not able to use a touchtone or touchtone-voice hybrid interface without prior training. A set of design recommendations is proposed based on the performance of several tested interface designs.

[1]  Roger Tourangeau,et al.  Humanizing self-administered surveys: experiments on social presence in web and IVR surveys , 2003, Comput. Hum. Behav..

[2]  Etienne Barnard,et al.  HIV health information access using spoken dialogue systems: Touchtone vs. speech , 2009, 2009 International Conference on Information and Communication Technologies and Development (ICTD).

[3]  Ronald Rosenfeld,et al.  HealthLine: Speech-based access to health information by low-literate users , 2007, 2007 International Conference on Information and Communication Technologies and Development.

[4]  Gaetano Borriello,et al.  Open Source Data Collection in the Developing World , 2009, Computer.

[5]  Emma Brunskill,et al.  Evaluating the accuracy of data collection on mobile phones: A study of forms, SMS, and voice , 2009, 2009 International Conference on Information and Communication Technologies and Development (ICTD).

[6]  Ronald Rosenfeld,et al.  Speech vs. touch-tone: Telephony interfaces for information access by low literate users , 2009, 2009 International Conference on Information and Communication Technologies and Development (ICTD).

[7]  R W Millard,et al.  Cross-sectional comparison of live and interactive voice recognition administration of the SF-12 health status survey. , 1999, The American journal of managed care.

[8]  William Thies,et al.  An Audio Wiki for Publishing User-Generated Content in the Developing World , 2008 .

[9]  Kentaro Toyama,et al.  A comparison of mobile money-transfer UIs for non-literate and semi-literate users , 2009, CHI.

[10]  Matthias Peissner,et al.  Voice User Interface Design , 2004, UP.

[11]  Science and technology for development: the new paradigm of ICT , 2007 .

[12]  Roni Rosenfeld,et al.  The Case for Speech Technology for Developing Regions , 2004 .

[13]  Frank Oberle,et al.  Who, Why and How Often? Key Elements for the Design of a Successful Speech Application Taking Account of the Target Groups , 2008 .

[14]  Marcel Tanner,et al.  The use of personal digital assistants for data entry at the point of collection in a large household survey in southern Tanzania , 2007, Emerging themes in epidemiology.

[15]  Joyojeet Pal,et al.  Speech Recognition for Illiterate Access to Information and Technology , 2006, 2006 International Conference on Information and Communication Technologies and Development.

[16]  Nitendra Rajput,et al.  A comparative study of speech and dialed input voice interfaces in rural India , 2009, CHI.

[17]  Neal Lesh,et al.  OpenROSA, JavaROSA, GloballyMobile - Collaborations around Open Standards for Mobile Applications , 2008 .

[18]  Bernhard Suhm,et al.  Ivr Usability Engineering Using Guidelines And Analyses Of End-to-End Calls , 2008 .