Patients' involvement in e-health services quality assessment: A system for the automatic interpretation of SMS-based patients' feedback

PURPOSE Effective communication between patients and health services providers is a key aspect for optimizing and maintaining these services. This work describes a system for the automatic evaluation of users' perception of the quality of SmsCup, a reminder system for outpatient visits based on short message service (SMS). The final purpose is the creation of a closed-loop control system for the outpatient service, where patients' complaints and comments represent a feedback that can be used for a better implementation of the service itself. METHODS SmsCup was adopted since about eight years by an Italian healthcare organization, with very good results in reducing the no-show (missing visits) phenomenon. During these years, a number of citizens, even if not required, sent a message back, with comments about the service. The automatic interpretation of the content of those SMS may be useful for monitoring and improving service performances.Yet, due to the complex nature of SMS language, their interpretation represents an ongoing challenge. The proposed system uses conditional random fields as the information extraction method for classifying messages into several semantic categories. The categories refer to appreciation of the service or complaints of various types. Then, the system analyzes the extracted content and provides feedback to the service providers, making them learning and acting on this basis. RESULTS At each step, the content of the messages reveals the actual state of the service as well as the efficacy of corrective actions previously undertaken. Our evaluations showed that: (i) the SMS classification system has achieved good overall performance with an average F1-measure and an overall accuracy of about 92%; (ii) the notification of the patients' feedbacks to service providers showed a positive impact on service functioning. CONCLUSIONS Our study proposed an interactive patient-centered system for continuous monitoring of the service quality. It has demonstrated the feasibility of a tool for the analysis and notification of the patients' feedback on their service experiences, which would support a more regular access to the service.

[1]  Hee-Seung Kim,et al.  Technological intervention for obese patients with type 2 diabetes. , 2008, Applied nursing research : ANR.

[2]  John G Meara,et al.  Use of SMS text messaging to improve outpatient attendance , 2005, The Medical journal of Australia.

[3]  Giordano Lanzola,et al.  An AI-Based Approach to Support Communication in Health Care Organizations , 2001, AIME.

[4]  Giordano Lanzola,et al.  An Agent-Based Architecture for Home Care Monitoring and Education of Chronic Patients , 2010, 2010 Complexity in Engineering.

[5]  Gunilla C. Nilsson,et al.  Health care consumers' experiences of information communication technology - A summary of literature , 2007, Int. J. Medical Informatics.

[6]  Jaakko Kujala,et al.  The impact of patient-physician web messaging on healthcare service provision , 2008, Int. J. Medical Informatics.

[7]  V. Ostojić,et al.  Improving asthma control through telemedicine: a study of short-message service. , 2005, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[8]  Giuseppe Riva,et al.  Stress management using UMTS cellular phones: a controlled trial. , 2006, Studies in health technology and informatics.

[9]  Paola Russo,et al.  A system for the extraction and representation of summary of product characteristics content , 2013, Artif. Intell. Medicine.

[10]  David W. Bates,et al.  The impact of medical informatics on patient satisfaction: A USA-based literature review , 2013, Int. J. Medical Informatics.

[11]  Robert S Gold,et al.  The promise and challenge of eHealth interventions. , 2002, American journal of health behavior.

[12]  Rob Garrett,et al.  Telecommunications access--matching available technologies to people with physical disabilities. , 2006, Australasian physical & engineering sciences in medicine.

[13]  J Starren,et al.  Architectural requirements for a multipurpose natural language processor in the clinical environment. , 1995, Proceedings. Symposium on Computer Applications in Medical Care.

[14]  C. Guinovart,et al.  The role of mobile phones in improving vaccination rates in travelers. , 2004, Preventive medicine.

[15]  Richard Wootton,et al.  Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review , 2011, Journal of telemedicine and telecare.

[16]  Joshua C. Denny,et al.  Focus on health information technology, electronic health records and their financial impact: PASTE: patient-centered SMS text tagging in a medication management system , 2012, J. Am. Medical Informatics Assoc..

[17]  M. Crawford,et al.  Systematic review of involving patients in the planning and development of health care , 2002, BMJ : British Medical Journal.

[18]  Alan R. Aronson,et al.  Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program , 2001, AMIA.

[19]  M. Stefanelli,et al.  Ontology and Terminology Servers in Agent-based Health-care Information Systems , 1997, Methods of Information in Medicine.

[20]  R. Neal,et al.  Missed appointments in primary care: questionnaire and focus group study of health professionals. , 2004, The British journal of general practice : the journal of the Royal College of General Practitioners.

[21]  Jiali Ye,et al.  E-mail in patient-provider communication: a systematic review. , 2010, Patient education and counseling.

[22]  Lorraine K. Tanabe,et al.  Tagging gene and protein names in biomedical text , 2002, Bioinform..

[23]  E. Balas,et al.  Healthcare via cell phones: a systematic review. , 2009, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[24]  Mary J Wills,et al.  Smoking cessation using mobile phone text messaging is as effective in Maori as non-Maori. , 2005, The New Zealand medical journal.

[25]  Dean F. Sittig.  Bmc Medical Informatics and Decision Making Results of a Content Analysis of Electronic Messages (email) Sent between Patients and Their Physicians , 2022 .

[26]  David T. Stern,et al.  Research Paper: A Content Analysis of E-mail Communication between Patients and Their Providers: Patients Get the Message , 2004, J. Am. Medical Informatics Assoc..

[27]  Silvana Quaglini,et al.  Reducing dropouts in outpatient care through an SMS-based system , 2006, MIE.

[28]  Peter J. Haug,et al.  Natural language processing to extract medical problems from electronic clinical documents: Performance evaluation , 2006, J. Biomed. Informatics.

[29]  Daniel Hanisch,et al.  : identifying , 2022 .

[30]  P. Laippala,et al.  Measurement of outpatients' views of service quality in a Finnish university hospital. , 2002, Journal of advanced nursing.

[31]  Alastair Baker,et al.  Crossing the Quality Chasm: A New Health System for the 21st Century , 2001, BMJ : British Medical Journal.

[32]  Jian Su,et al.  A Phrase-Based Statistical Model for SMS Text Normalization , 2006, ACL.

[33]  Hae-Chang Rim,et al.  Two Phase Model for SMS Text Messages Refinement , 2008, AAAI 2008.

[34]  Giordano Lanzola,et al.  First steps toward an adaptive spoken dialogue system in medical domain , 2001, INTERSPEECH.

[35]  Howard Bauchner,et al.  A Content Analysis of E-mail Communication Between Primary Care Providers and Parents , 2005, Pediatrics.

[36]  Justin C McArthur,et al.  A programmable prompting device improves adherence to highly active antiretroviral therapy in HIV-infected subjects with memory impairment. , 2005, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[37]  J. S. Sodhi,et al.  Using Internet and Mobile Phone Technology to Deliver an Automated Physical Activity Program: Randomized Controlled Trial , 2007, Journal of medical Internet research.

[38]  Johan Hoogstraten,et al.  Failed appointments in an orthodontic clinic. , 2005, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[39]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.