Development and validation of the Italian version of the Mobile Application Rating Scale and its generalisability to apps targeting primary prevention

BackgroundA growing body of literature affirms the usefulness of mobile technologies, including mobile applications (apps), in the primary prevention field. The quality of health apps, which today number in the thousands, is a crucial parameter, as it may affect health-related decision-making and outcomes among app end-users. The mobile application rating scale (MARS) has recently been developed to evaluate the quality of such apps, and has shown good psychometric properties. Since there is no standardised tool for assessing the apps available in Italian app stores, the present study developed and validated an Italian version of MARS in apps targeting primary prevention.MethodsThe original 23-item version of the MARS assesses mobile app quality in four objective quality dimensions (engagement, functionality, aesthetics, information) and one subjective dimension. Validation of this tool involved several steps; the universalist approach to achieving equivalence was adopted. Following two backward translations, a reconciled Italian version of MARS was produced and compared with the original scale. On the basis of sample size estimation, 48 apps from three major app stores were downloaded; the first 5 were used for piloting, while the remaining 43 were used in the main study in order to assess the psychometric properties of the scale. The apps were assessed by two raters, each working independently. The psychometric properties of the final version of the scale was assessed including the inter-rater reliability, internal consistency, convergent, divergent and concurrent validities.ResultsThe intralingual equivalence of the Italian version of the MARS was confirmed by the authors of the original scale. A total of 43 apps targeting primary prevention were tested. The MARS displayed acceptable psychometric properties. The MARS total score showed an excellent level of both inter-rater agreement (intra-class correlation coefficient of .96) and internal consistency (Cronbach’s α of .90 and .91 for the two raters, respectively). Other types of validity, including convergent, divergent, discriminative, known-groups and scalability, were also established.ConclusionsThe Italian version of MARS is a valid and reliable tool for assessing the health-related primary prevention apps available in Italian app stores.

[1]  Maria Pavia,et al.  Physicians' knowledge, attitudes and professional use of RCTs and meta-analyses: a cross-sectional survey. , 2009, European journal of public health.

[2]  Darren George,et al.  SPSS for Windows Step by Step: A Simple Guide and Reference , 1998 .

[3]  Nick Wilson,et al.  Smartphone apps for weight loss and smoking cessation: Quality ranking of 120 apps. , 2015, The New Zealand medical journal.

[4]  Jiang Bian,et al.  Low Quality of Free Coaching Apps With Respect to the American College of Sports Medicine Guidelines: A Review of Current Mobile Apps , 2015, JMIR mHealth and uHealth.

[5]  J. Bernhardt,et al.  Behavioral Functionality of Mobile Apps in Health Interventions: A Systematic Review of the Literature , 2015, JMIR mHealth and uHealth.

[6]  Oksana Zelenko,et al.  Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile Apps , 2015, JMIR mHealth and uHealth.

[7]  Matthew Hankins.  Questionnaire discrimination : ( re )-introducing coefficient delta , 2017 .

[8]  C. Terwee,et al.  Quality criteria were proposed for measurement properties of health status questionnaires. , 2007, Journal of clinical epidemiology.

[9]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[10]  Jaehee Cho,et al.  The impact of post-adoption beliefs on the continued use of health apps , 2016, Int. J. Medical Informatics.

[11]  Alexander Domnich,et al.  An overview of current and potential use of information and communication technologies for immunization promotion among adolescents , 2013, Human vaccines & immunotherapeutics.

[12]  Danit R Shahar,et al.  The Impact of a Web-Based App (eBalance) in Promoting Healthy Lifestyles: Randomized Controlled Trial , 2015, Journal of medical Internet research.

[13]  Barry Lambe,et al.  A qualitative study of lifestyle counselling in general practice in Ireland. , 2010, Family practice.

[14]  Emanuela Carloni,et al.  Comparison of quality of internet pages on human papillomavirus immunization in Italian and in English. , 2010, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[15]  M. Kuehnhausen,et al.  Trusting smartphone Apps? To install or not to install, that is the question , 2013, 2013 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA).

[16]  Wanda Pratt,et al.  Healthcare in the pocket: Mapping the space of mobile-phone health interventions , 2012, J. Biomed. Informatics.

[17]  Peter B. Jones,et al.  Mokken scale analysis of mental health and well-being questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers , 2012, BMC Medical Research Methodology.

[18]  Hitoshi Shimano,et al.  Quality of Internet information related to the Mediterranean diet , 2011, Public Health Nutrition.

[19]  L. Portney,et al.  Foundations of Clinical Research: Applications to Practice , 2015 .

[20]  T. van Achterberg,et al.  Barriers, facilitators and attitudes influencing health promotion activities in general practice: an explorative pilot study , 2013, BMC Family Practice.

[21]  J F Fries,et al.  Aging, health risks, and cumulative disability. , 1998, The New England journal of medicine.

[22]  M. Hankins,et al.  Questionnaire discrimination: (re)-introducing coefficient δ , 2007, BMC medical research methodology.

[23]  H Boeing,et al.  History of the concept of ‘levels of evidence’ and their current status in relation to primary prevention through lifestyle interventions , 2004, Public Health Nutrition.

[24]  Stoyan R. Stoyanov,et al.  Review and Evaluation of Mindfulness-Based iPhone Apps , 2015, JMIR mHealth and uHealth.

[25]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[26]  G Y Zou,et al.  Sample size formulas for estimating intraclass correlation coefficients with precision and assurance , 2012, Statistics in medicine.

[27]  Urs-Vito Albrecht,et al.  mHealth 2.0: Experiences, Possibilities, and Perspectives , 2014, JMIR mHealth and uHealth.

[28]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[29]  Yin Liu,et al.  mHealthApps: A Repository and Database of Mobile Health Apps , 2015, JMIR mHealth and uHealth.

[30]  Mikkel Godsk,et al.  THE PLEASURE OF E-LEARNING – TOWARDS AESTHETIC E-LEARNING PLATFORMS , .

[31]  R. D'Agostino.  Transformation to normality of the null distribution of g1 , 1970 .

[32]  C. Guinovart,et al.  The role of mobile phones in improving vaccination rates in travelers. , 2004, Preventive medicine.

[33]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[34]  Molly E Waring,et al.  Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations , 2014, Translational behavioral medicine.

[35]  X. Badia,et al.  A Model of Equivalence in the Cultural Adaptation of HRQoL Instruments: The Universalist Approach , 1998, Quality of Life Research.

[36]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[37]  R. J. Cline,et al.  Consumer health information seeking on the Internet: the state of the art. , 2001, Health education research.

[38]  Nívea de M. O. Morales,et al.  Psychometric properties of the initial Brazilian version of the CHQ-PF50 applied to the caregivers of children and adolescents with cerebral palsy , 2007, Quality of Life Research.

[39]  Rebecca Schnall,et al.  Review and analysis of existing mobile phone applications for health care-associated infection prevention. , 2015, American journal of infection control.

[40]  Kevin L. Joiner,et al.  A Novel Diabetes Prevention Intervention Using a Mobile App: A Randomized Controlled Trial With Overweight Adults at Risk. , 2015, American journal of preventive medicine.

[41]  V. Sierpina,et al.  mhealth (mobile Health)—using Apps for Health and Wellness , 2011 .

[42]  C. Spearman.  CORRELATION CALCULATED FROM FAULTY DATA , 1910 .

[43]  M. Stockwell,et al.  Text message reminders to promote human papillomavirus vaccination. , 2011, Vaccine.

[44]  Harold L. Somers,et al.  Computers and translation : a translator's guide , 2003 .

[45]  Flora Douglas,et al.  Primary care staff's views and experiences related to routinely advising patients about physical activity. A questionnaire survey , 2006, BMC public health.

[46]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[47]  Ten great public health achievements--worldwide, 2001-2010. , 2011, MMWR. Morbidity and mortality weekly report.

[48]  Leonard S. Feldt,et al.  Testing the Equality of Two Alpha Coefficients , 1996 .

[49]  W. Brown.  SOME EXPERIMENTAL RESULTS IN THE CORRELATION OF MENTAL ABILITIES1 , 1910 .

[50]  David B Buller,et al.  Evaluation of immediate and 12-week effects of a smartphone sun-safety mobile application: a randomized clinical trial. , 2015, JAMA dermatology.