Evaluating the accuracy of data collection on mobile phones: A study of forms, SMS, and voice

While mobile phones have found broad application in reporting health, financial, and environmental data, there has been little study of the possible errors incurred during mobile data collection. This paper provides the first (to our knowledge) quantitative evaluation of data entry accuracy on mobile phones in a resource-poor setting. Via a study of 13 users in Gujarat, India, we evaluated three user interfaces: 1) electronic forms, containing numeric fields and multiple-choice menus, 2) SMS, where users enter delimited text messages according to printed cue cards, and 3) voice, where users call an operator and dictate the data in real-time. Our results indicate error rates (per datum entered) of 4.2% for electronic forms, 4.8% for SMS, and 0.45% for voice. These results caused us to migrate our own initiative (a tuberculosis treatment program in rural India) from electronic forms to voice, in order to avoid errors on critical health data. While our study has some limitations, including varied backgrounds and training of participants, it suggests that some care is needed in deploying electronic interfaces in resource-poor settings. Further, it raises the possibility of using voice as a low-tech, high-accuracy, and cost-effective interface for mobile data collection.

[1]  S. Tata,et al.  Achieving the Health Millennium Development Goals in Asia and the Pacific. Policies and actions within health systems and beyond. , 2007 .

[2]  P. Gupta,et al.  Survey of sociodemographic characteristics of tobacco use among 99,598 individuals in Bombay, India using handheld computers. , 1996, Tobacco control.

[3]  Kentaro Toyama,et al.  Mobile phones and paper documents: evaluating a new approach for capturing microfinance data in rural India , 2006, CHI.

[4]  Nathaniel P. Katz,et al.  Comparative study of electronic vs. paper VAS ratings: a randomized, crossover trial using healthy volunteers , 2002, PAIN.

[5]  Bryant Thomas Karras,et al.  Design and Implementation of Cell-PREVEN: A Real-Time Surveillance System for Adverse Events Using Cell Phones in Peru , 2005, AMIA.

[6]  B. Tiplady,et al.  The Use of Electronic Diaries in Respiratory Studies , 1997 .

[7]  P. Byass,et al.  Evaluation of a computerized field data collection system for health surveys. , 1991, Bulletin of the World Health Organization.

[8]  T. Engebretsen,et al.  Acceptance of information technology by health research projects in low-income countries : intention to use and acceptance of using EpiHandy (IUAUE) , 2005 .

[9]  D. Skinner,et al.  Evaluation of use of cellphones to aid compliance with drug therapy for HIV patients , 2007, AIDS care.

[10]  Tapan S. Parikh.  Using mobile phones for secure, distributed document processing in the developing world , 2005, IEEE Pervasive Computing.

[11]  Dorian G. W. Smith,et al.  Palm computer demonstrates a fast and accurate means of burn data collection. , 2000, The Journal of burn care & rehabilitation.

[12]  J. Bahnson,et al.  Using a hand-held computer to collect data in an orthopedic outpatient clinic: a randomized trial of two survey methods. , 1999, Medical care.

[13]  Sally Grisedale,et al.  Designing a graphical user interface for healthcare workers in rural India , 1997, CHI.

[14]  Shannon J. Lane,et al.  Bmc Medical Informatics and Decision Making a Review of Randomized Controlled Trials Comparing the Effectiveness of Hand Held Computers with Paper Methods for Data Collection , 2006 .

[15]  T. Groves,et al.  SatelLife: getting relevant information to the developing world , 1996, BMJ.

[16]  Hamish S. F. Fraser,et al.  Development, implementation and preliminary study of a PDA-based bacteriology collection system , 2006, AMIA.

[17]  Marcel Tanner,et al.  The use of personal digital assistants for data entry at the point of collection in a large household survey in southern Tanzania , 2007, Emerging themes in epidemiology.

[18]  James P. Hughes,et al.  Bmc Medical Informatics and Decision Making Handheld Computers for Self-administered Sensitive Data Collection: a Comparative Study in Peru , 2007 .

[19]  H Fraser,et al.  Cost and implementation analysis of a personal digital assistant system for laboratory data collection. , 2008, The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease.

[20]  David Wypij,et al.  Short report: Piloting paperless data entry for clinical research in Africa. , 2005, The American journal of tropical medicine and hygiene.

[21]  Gaetano Borriello,et al.  E-imci: improving pediatric health care in low-income countries , 2008, CHI.

[22]  Ha Chan,et al.  Remote HIV/AIDS Patient Monitoring Tool Using 3G/GPRS Packet-Switched Mobile Technology , 2006 .

[23]  P. Quinn,et al.  Assessment of an electronic daily diary in patients with overactive bladder , 2003, BJU international.

[24]  William M. Tierney,et al.  A computer-based medical record system and personal digital assistants to assess and follow patients with respiratory tract infections visiting a rural Kenyan health centre , 2006, BMC Medical Informatics Decis. Mak..

[25]  A. Pentland,et al.  Handheld computers for rural healthcare : Experiences from research concept to global operations , 2002 .

[26]  N Heddle,et al.  Comparing hand‐held computers and paper diaries for haemophilia home therapy: a randomized trial , 2004, Haemophilia : the official journal of the World Federation of Hemophilia.

[27]  Justin Starren,et al.  A Methodological Framework for Evaluating Mobile Health Devices , 2006, AMIA.