Reliability of Data Collected via Ecological Momentary Assessment on the Example of FeverApp Registry

The FeverApp registry is an ambulant ecological momentary assessment (EMA) model registry focusing on research of fever in children. Verification of EMA reliability is a challenge, due to absence of other source data. To ensure the reliability of EMA data, 973 families were invited to reassess their documentation in a survey. The survey contained questions (a) regarding the number of children, (b) genuineness of entries, (c) completeness of submitted fever episodes, (d) medication, (e) usefulness and further usage of the app. Of those invited, 438 families (45% response rate) participated in the survey. Of these, 363 (83%) families have registered all their children, 208 families have one child. The majority (n = 325, 74.2%) of families stated that they only made genuine entries in the app. Agreement between survey and app for fever episodes is 90% (Cohen’s κ = 0.75 [0.66, 0.82]). Medication shows 73.7% agreement, κ = 0.49 [0.42; 0.54]. The majority (n = 245, 55.9%) consider the app as an additional benefit and 87.3% would like to use it further. Email surveys are a possible approach to evaluate EMA based registry data. The possible observation units (children and fever episodes) show an adequate reliability. With this approach, surveys of further samples and variables could help to improve the quality of EMA based registries.

[1]  E. Jenetzky,et al.  Data Completeness and Concordance in the FeverApp Registry: Comparative Study , 2022, JMIR pediatrics and parenting.

[2]  C. Levy,et al.  Trends in pediatric ambulatory community acquired infections before and during COVID-19 pandemic: A prospective multicentric surveillance study in France , 2022, The Lancet Regional Health - Europe.

[3]  Eric S. Cerino,et al.  Little evidence for consistent initial elevation bias in self-reported momentary affect: A coordinated analysis of ecological momentary assessment studies. , 2022, Psychological assessment.

[4]  C. Soriano-Mas,et al.  Development and Validation of a Smartphone-Based App for the Longitudinal Assessment of Anxiety in Daily Life , 2021, Assessment.

[5]  Jinshil Hyun,et al.  Reliabilities of intra-individual mean and intra-individual variability of self-reported pain derived from ecological momentary assessments: Results from the Einstein Aging Study. , 2021, The journal of pain.

[6]  P. Wattananon,et al.  The inter-rater reliability of clinical observation of prone hip extension and association between aberrant movement and chronic low back pain. , 2021, Musculoskeletal science & practice.

[7]  E. Jenetzky,et al.  [The FeverApp Registry - A Way to Empower Parents through their Own Documentation to a Graduated Decision]. , 2021, Das Gesundheitswesen.

[8]  B. Koletzko,et al.  COVID-19 Associated Contact Restrictions in Germany: Marked Decline in Children’s Outpatient Visits for Infectious Diseases without Increasing Visits for Mental Health Disorders , 2021, Children.

[9]  Nadja Schinkel-Bielefeld,et al.  Is it Possible to Identify Careless Responses with Post-hoc Analysis in EMA Studies? , 2021, UMAP.

[10]  G. Warren,et al.  Concordance Between Electronic Health Record and Tumor Registry Documentation of Smoking Status Among Patients With Cancer. , 2021, JCO clinical cancer informatics.

[11]  A. Büssing,et al.  Sociodemographic Characteristics and Interests of FeverApp Users , 2021, International journal of environmental research and public health.

[12]  V. Sallinen,et al.  Accuracy of Finnish Cancer Registry colorectal cancer data: a comparison between registry data and clinical records , 2021, Scandinavian journal of gastroenterology.

[13]  Jürgen Stausberg,et al.  Facilitating harmonized data quality assessments. A data quality framework for observational health research data collections with software implementations in R , 2020, BMC Medical Research Methodology.

[14]  M. Marino,et al.  Assessing Cancer History Accuracy in Primary Care Electronic Health Records Through Cancer Registry Linkage , 2020, Journal of the National Cancer Institute.

[15]  Ekkehart Jenetzky,et al.  The FeverApp registry – ecological momentary assessment (EMA) of fever management in families regarding conformity to up-to-date recommendations , 2020, BMC Medical Informatics and Decision Making.

[16]  Yu-Hsiang Wu,et al.  Test-Retest Reliability of Ecological Momentary Assessment in Audiology Research , 2020, Journal of the American Academy of Audiology.

[17]  F. M. van der Veen,et al.  No effect of repetitive tDCS on daily smoking behaviour in light smokers: A placebo controlled EMA study , 2020, PloS one.

[18]  B. Samoliński,et al.  The association between paracetamol use and the risk of asthma, rhinitis and eczema in the Polish population. , 2018, Annals of agricultural and environmental medicine : AAEM.

[19]  R. Laxer,et al.  Approach to recurrent fever in childhood. , 2017, Canadian family physician Medecin de famille canadien.

[20]  Aaron Schroeder,et al.  The Evolution of Data Quality: Understanding the Transdisciplinary Origins of Data Quality Concepts and Approaches , 2017 .

[21]  A. Moore,et al.  Feverish illness in children: Assessment and initial management in children younger than 5 years , 2015 .

[22]  Christopher P. Cerasoli,et al.  Intrinsic motivation and extrinsic incentives jointly predict performance: a 40-year meta-analysis. , 2014, Psychological bulletin.

[23]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[24]  T. Davis.  NICE guideline: feverish illness in children—assessment and initial management in children younger than 5 years , 2013, Archives of Disease in Childhood: Education & Practice Edition.

[25]  Niels Wedderkopp,et al.  Comparison between data obtained through real-time data capture by SMS and a retrospective telephone interview , 2010, Chiropractic & osteopathy.

[26]  S. Shiffman,et al.  Ecological momentary assessment. , 2008, Annual review of clinical psychology.

[27]  M. Kidon,et al.  Severe Anaphylactic Reaction to Ibuprofen in a Child With Recurrent Urticaria , 2007, Pediatrics.

[28]  David H. Epstein,et al.  An electronic diary software for ecological momentary assessment (EMA) in clinical trials , 2004, Proceedings. 17th IEEE Symposium on Computer-Based Medical Systems.

[29]  F. Blay,et al.  Acetaminophen (paracetamol)-induced anaphylactic shock. , 2002, Southern medical journal.

[30]  M. Litt,et al.  Ecological momentary assessment (EMA) with treated alcoholics: methodological problems and potential solutions. , 1998, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[31]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[32]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .