Psychometric Evaluation and Workflow Integration Study of a Tablet-Based Tool to Detect Mild Cognitive Impairment in Older Adults: Protocol for a Mixed Methods Study

Background With the rapid aging of the global population, experts anticipate a surge in the prevalence of mild cognitive impairment (MCI) and dementia worldwide. It is argued that developing more sensitive, easy to administer, and valid MCI screening tools for use in primary care settings may initiate timely clinical and personal care planning and treatment, enabling early access to programs and services. Including functional competence measures in screening tests makes them more ecologically valid and may help to identify cognitive deficits at an earlier stage. Objective We aim to conduct a preliminary evaluative study comparing the sensitivity, specificity, and reliability of the BrainFx Screen (referred to as SCREEN hereafter), a novel digital tool designed to assess functional competence and detect early signs of cognitive impairment, with the Quick Mild Cognitive Impairment, a validated and highly sensitive tool that detects MCI in the older adult population. We will also investigate the perceived usefulness and integration of the SCREEN into primary care practice to identify demonstrable impacts on clinical workflow and health care providers’ (HCP) perceptions of its success as a screening tool. Patients’ perceptions of completing the SCREEN and its impact on their quality of life will also be explored. Methods This study has a concurrent, mixed methods, prospective, and quasi-experimental design. Participants will be recruited from 5 primary care family health teams (FHTs; defined by multidisciplinary practice and capitated funding) across southwestern Ontario, Canada. Participants will include HCPs, patients, care partners, and FHT administrative executives. Patients 55 years and older with no history of diagnoses for MCI, dementia, or Alzheimer disease rostered in one of the FHTs participating in the study will be eligible to participate. Their care partners will help triangulate the qualitative data collected from patients. Participating FHTs will identify an occupational therapist from their site to participate in the study; this HCP will both administer the research protocol and participate in semistructured in-depth interviews and questionnaires. Principal component analysis will be conducted on the SCREEN data to understand the test components better. Tests comparing sensitivity, specificity, and test-retest reliability will assess the validity of SCREEN as a screening tool for MCI. Results This paper describes the study protocol and its activities to date. Data collection was halted early because of COVID-19 restrictions on research activity, and data analysis is currently in progress. Conclusions At the end of the project, we anticipate having an initial comparative evaluation of the SCREEN as a tool for early detection of MCI in primary care older adult patient populations. Resource constraints on this research study limit our ability to conduct a randomized controlled trial; however, the results will assist developers of the SCREEN in determining whether rigorous controlled testing is warranted. International Registered Report Identifier (IRRID) DERR1-10.2196/25520

[1]  I. Lang,et al.  Case-finding in clinical practice: An appropriate strategy for dementia identification? , 2018, Alzheimer's & dementia.

[2]  Jennifer S Lin,et al.  Screening for Cognitive Impairment in Older Adults: An Evidence Update for the U.S. Preventive Services Task Force , 2013 .

[3]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[4]  O. Uchendu,et al.  Validation and Reliability of the 12-item Zarit Burden Interview among Informal Caregivers of Elderly Persons in Nigeria. , 2018, Archives of basic and applied medicine.

[5]  Z. Nasreddine,et al.  Validation of Montreal Cognitive Assessment, MoCA, Alternate French Versions , 2016, Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques.

[6]  C. Paiva,et al.  A critical analysis of test-retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review , 2014, BMC Medical Research Methodology.

[7]  A. Hadgu,et al.  Evaluation of Nucleic Acid Amplification Tests in the Absence of a Perfect Gold-Standard Test: A Review of the Statistical and Epidemiologic Issues , 2005, Epidemiology.

[8]  D. Molloy,et al.  Comparing the Diagnostic Accuracy of Two Cognitive Screening Instruments in Different Dementia Subtypes and Clinical Depression , 2019, Diagnostics.

[9]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[10]  A. Mitchell,et al.  A meta-analysis of the accuracy of the Addenbrooke's Cognitive Examination (ACE) and the Addenbrooke's Cognitive Examination-Revised (ACE-R) in the detection of dementia , 2013, International Psychogeriatrics.

[11]  W. Maier,et al.  General Practitioners’ Judgment of their Elderly Patients’ Cognitive Status , 2009, Journal of General Internal Medicine.

[12]  M. Boustani,et al.  Screening for dementia – is it a no brainer? , 2013, International journal of clinical practice.

[13]  J. Cummings,et al.  The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool For Mild Cognitive Impairment , 2005, Journal of the American Geriatrics Society.

[14]  Douglas K Owens,et al.  Screening for Cognitive Impairment in Older Adults: US Preventive Services Task Force Recommendation Statement. , 2020, JAMA.

[15]  Elizabeth Eckstrom,et al.  Screening for Cognitive Impairment in Older Adults: A Systematic Review for the U.S. Preventive Services Task Force , 2013, Annals of Internal Medicine.

[16]  J. Weuve,et al.  2016 Alzheimer's disease facts and figures , 2016 .

[17]  C. Gass,et al.  Test anxiety in relation to measures of cognitive and intellectual functioning. , 2011, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[18]  L. Maxim,et al.  Screening tests: a review with examples , 2014, Inhalation toxicology.

[19]  M. Boustani,et al.  Dementia screening in primary care: is it time? , 2007, JAMA.

[20]  B. Lawlor,et al.  Is the Quick Mild Cognitive Impairment Screen (QMCI) more accurate at detecting mild cognitive impairment than existing short cognitive screening tests? A systematic review of the current literature , 2019, International journal of geriatric psychiatry.

[21]  D. William Molloy,et al.  Screening for Mild Cognitive Impairment: Comparison of “MCI Specific” Screening Instruments , 2016, Journal of Alzheimer's disease : JAD.

[22]  W. Heiss,et al.  Mild Cognitive Impairment (MCI) , 2021, Encyclopedia of Gerontology and Population Aging.

[23]  E. Tangalos,et al.  Mild Cognitive Impairment Clinical Characterization and Outcome , 1999 .

[24]  N. Anderson State of the science on mild cognitive impairment (MCI) , 2019, CNS Spectrums.

[25]  D. William Molloy,et al.  The Quick Mild Cognitive Impairment Screen (Q mci ) , 2017 .

[26]  D L Streiner,et al.  A checklist for evaluating the usefulness of rating scales. , 1993, Canadian journal of psychiatry. Revue canadienne de psychiatrie.

[27]  J. Husbands,et al.  Quantifying the diagnostic pathway for patients with cognitive impairment: real-world data from Australia , 2019, International Psychogeriatrics.

[28]  R. Stern,et al.  Practice Effects on Story Memory and List Learning Tests in the Neuropsychological Assessment of Older Adults , 2016, PloS one.

[29]  Ping-Shun Chen,et al.  Applying Task-Technology Fit Model to the Healthcare Sector: a Case Study of Hospitals’ Computed Tomography Patient-Referral Mechanism , 2015, Journal of Medical Systems.

[30]  J. Aguilar The Canadian Task Force on Preventive Health Care , 2012 .

[31]  Christianna S. Williams,et al.  Cognitive screening for dementia and mild cognitive impairment in assisted living: comparison of 3 tests. , 2008, Journal of the American Medical Directors Association.

[32]  R. Petersen Mild cognitive impairment as a diagnostic entity , 2004, Journal of internal medicine.

[33]  K. Pottie,et al.  Recommendations on screening for cognitive impairment in older adults , 2016, Canadian Medical Association Journal.

[34]  Tor J. Larsen,et al.  The role of task-technology fit as users' motivation to continue information system use , 2009, Comput. Hum. Behav..

[35]  R. Charter,et al.  Psychometric Properties of the Folstein Mini-Mental State Examination , 2005, Assessment.

[36]  Y. Jang,et al.  Psychometric and diagnostic properties of the Taiwan version of the Quick Mild Cognitive Impairment screen , 2018, PloS one.

[37]  Martin Rossor,et al.  Consent recommendations for research and international data sharing involving persons with dementia , 2018, Alzheimer's & Dementia.

[38]  David L. Streinmer A Checklist for Evaluating the Usefulness of Rating Scales , 1993 .

[39]  E. Tangalos,et al.  Mild Cognitive Impairment in Geriatrics. , 2018, Clinics in geriatric medicine.

[40]  M. Simard,et al.  Cognitive intervention programs for individuals with mild cognitive impairment: systematic review of the literature. , 2010, The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry.

[41]  D. Hambrick,et al.  Reliability and stability of executive functioning in older adults. , 2006, Neuropsychology.

[42]  J. Becker,et al.  Ecological assessment of executive functions in mild cognitive impairment and mild Alzheimer’s disease , 2009, Journal of the International Neuropsychological Society.

[43]  W. Kremen,et al.  Extensive memory testing improves prediction of progression to MCI in late middle age , 2020, Alzheimer's & dementia.

[44]  D. Molloy,et al.  The Zarit Burden Interview: a new short version and screening version. , 2001, The Gerontologist.

[45]  M. Balsamo,et al.  Assessment of anxiety in older adults: a review of self-report measures , 2018, Clinical interventions in aging.

[46]  L. Fratiglioni,et al.  Mild cognitive impairment: a concept in evolution , 2014, Journal of internal medicine.

[47]  Daniel Casey,et al.  Cognitive tests for the detection of mild cognitive impairment (MCI), the prodromal stage of dementia: Meta‐analysis of diagnostic accuracy studies , 2018, International journal of geriatric psychiatry.

[48]  P. Belchior,et al.  Identification and assessment of functional performance in mild cognitive impairment: a survey of occupational therapy practices. , 2015, Australian occupational therapy journal.

[49]  A. Compare,et al.  An investigation of the psychometric properties of the Geriatric Anxiety Scale (GAS) in an Italian sample of community-dwelling older adults , 2018, Aging & mental health.

[50]  B. Albensi,et al.  Future Trends and the Economic Burden of Dementia in Manitoba: Comparison with the Rest of Canada and the World , 2018, Neuroepidemiology.

[51]  N. Anderson State of the Science on Mild Cognitive Impairment. , 2020, The journals of gerontology. Series B, Psychological sciences and social sciences.

[52]  Johannes B. Reitsma,et al.  A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. , 2009, Journal of clinical epidemiology.

[53]  Angela J. Deokar,et al.  Self-Reported Increased Confusion or Memory Loss and Associated Functional Difficulties Among Adults Aged ≥60 Years — 21 States, 2011 , 2013, MMWR. Morbidity and mortality weekly report.

[54]  S. Zarit,et al.  Relatives of the impaired elderly: correlates of feelings of burden. , 1980, The Gerontologist.

[55]  Linda Beuscher,et al.  Challenges in conducting qualitative research with individuals with dementia. , 2009, Research in gerontological nursing.

[56]  F. Jessen,et al.  Prognosis of Mild Cognitive Impairment in General Practice: Results of the German AgeCoDe Study , 2014, The Annals of Family Medicine.

[57]  P. Belchior,et al.  Assessing functional impairment in individuals with mild cognitive impairment , 2016 .

[58]  P. Neumann,et al.  The economics of mild cognitive impairment , 2013, Alzheimer's & Dementia.

[59]  Brent Furneaux,et al.  Task-technology fit theory: A survey and synopsis of the literature , 2012 .

[60]  Carrie D. Patnode,et al.  Screening for Cognitive Impairment in Older Adults: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force. , 2020, JAMA.

[61]  S. McLennan,et al.  Validity of the Montreal Cognitive Assessment (MoCA) as a Screening Test for Mild Cognitive Impairment (MCI) in a Cardiovascular Population , 2011, Journal of geriatric psychiatry and neurology.

[62]  Edward C. Jones,et al.  A comparison of two time intervals for test-retest reliability of health status instruments. , 2003, Journal of clinical epidemiology.

[63]  P. Vik,et al.  Neuropsychological Assessment Anxiety: A Systematic Review , 2018, Practice Innovations.

[64]  E. Tangalos,et al.  Long-Term Stability and Intercorrelations of Cognitive Abilities in Older Persons. , 1995 .

[65]  Paul Gallagher,et al.  Comparing Approaches to Optimize Cut-off Scores for Short Cognitive Screening Instruments in Mild Cognitive Impairment and Dementia , 2017, Journal of Alzheimer's disease : JAD.

[66]  D. Segal,et al.  Geriatric Anxiety Scale: item response theory analysis, differential item functioning, and creation of a ten-item short form (GAS-10) , 2014, International Psychogeriatrics.

[67]  Marshall Godwin,et al.  Health measurement scales , 1991 .

[68]  G. Smith,et al.  Everyday Impact of Cognitive Interventions in Mild Cognitive Impairment: a Systematic Review and Meta-Analysis , 2016, Neuropsychology Review.

[69]  E. Scarpini,et al.  The instruments used by the Italian centres for cognitive disorders and dementia to diagnose mild cognitive impairment (MCI) , 2018, Aging Clinical and Experimental Research.

[70]  Ashley A. Miles,et al.  Cognitive performance and functional competence as predictors of community independence in schizophrenia. , 2010, Schizophrenia bulletin.

[71]  M. Pai,et al.  Measuring burden in dementia caregivers: Confirmatory factor analysis for short forms of the Zarit Burden Interview. , 2017, Archives of gerontology and geriatrics.

[72]  C. Cohen Screening for cognitive impairment in the elderly. , 1991, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[73]  Paul Gallagher,et al.  Comparison of the quick mild cognitive impairment (Qmci) screen and the SMMSE in screening for mild cognitive impairment , 2012, Age and ageing.

[74]  C. Ritchie,et al.  Quantifying the Diagnostic Pathway for Patients with Cognitive Impairment: Real-World Data from Seven European and North American Countries. , 2018, Journal of Alzheimer's disease : JAD.

[75]  V. Moyer,et al.  Assessing diagnostic and screening tests: Part 1. Concepts. , 2001, The Western journal of medicine.

[76]  D. William Molloy,et al.  Which part of the Quick mild cognitive impairment screen (Qmci) discriminates between normal cognition, mild cognitive impairment and dementia? , 2013, Age and ageing.

[77]  R. Dishman,et al.  Functional Disability in Mild Cognitive Impairment: A Systematic Review and Meta-Analysis , 2016, Neuropsychology Review.

[78]  E. Howe,et al.  Informed consent, participation in research, and the Alzheimer's patient. , 2012, Innovations in clinical neuroscience.

[79]  A. Terracciano,et al.  Effectiveness of Powerful Tools for Caregivers on Caregiver Burden and on Care Recipient Behavioral and Psychological Symptoms of Dementia: A Randomized Controlled Trial. , 2019, Journal of the American Medical Directors Association.

[80]  S. Barbic,et al.  Diagnostic accuracy of the Ottawa 3DY and Short Blessed Test to detect cognitive dysfunction in geriatric patients presenting to the emergency department , 2018, BMJ Open.

[81]  H. Horst,et al.  Preferences regarding disclosure of a diagnosis of dementia: a systematic review , 2014 .

[82]  A. Astell,et al.  Cognitive Behavior Therapy for Anxious and Depressed Youth: Improving Homework Adherence Through Mobile Technology , 2016, JMIR research protocols.

[83]  R. Mace,et al.  Predicting Functional Dependence in Mild Cognitive Impairment: Differential Contributions of Memory and Executive Functions. , 2018, The Gerontologist.

[84]  H. Arrighi,et al.  Mild cognitive impairment: Disparity of incidence and prevalence estimates , 2012, Alzheimer's & Dementia.

[85]  Khajak J. Ishak,et al.  An economic evaluation of early assessment for Alzheimer’s disease in the United Kingdom , 2012, Alzheimer's & Dementia.

[86]  A. Mitchell,et al.  A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. , 2009, Journal of psychiatric research.

[87]  D. Segal,et al.  Psychometric Properties of the Geriatric Anxiety Scale: Comparison to the Beck Anxiety Inventory and Geriatric Anxiety Inventory , 2010 .

[88]  C. Ferri,et al.  World Alzheimer Report 2011 : The benefits of early diagnosis and intervention , 2018 .

[89]  L. Rajmil Health measurement scales. A practical guide to their development and use, 3rd ed , 2005 .

[90]  John Bond,et al.  The worldwide economic impact of dementia 2010 , 2013, Alzheimer's & Dementia.

[91]  D. Swinglehurst,et al.  ‘Timely’ diagnosis of dementia: what does it mean? A narrative analysis of GPs’ accounts , 2014, BMJ Open.

[92]  D. Morgan,et al.  Factor Analysis of the 12-Item Zarit Burden Interview in Caregivers of Persons Diagnosed With Dementia , 2016, Journal of applied gerontology : the official journal of the Southern Gerontological Society.

[93]  Laurie M. Baker,et al.  Longitudinal Change in Performance on the Montreal Cognitive Assessment in Older Adults , 2015, The Clinical neuropsychologist.

[94]  Y. Braw,et al.  A Call for Establishing a Public Policy for Mitigating the Effects of Dementia on Modern Workplaces , 2018 .

[95]  D. Segal,et al.  Development and initial validation of a self-report assessment tool for anxiety among older adults: the Geriatric Anxiety Scale. , 2010, Journal of anxiety disorders.

[96]  M. Boustani Dementia Screening in Primary Care: Not Too Fast! , 2013, Journal of the American Geriatrics Society.

[97]  B. Ragan,et al.  Test–Retest Reliability of the BrainFx 360 ® Performance Assessment , 2019, Athletic Training & Sports Health Care.

[98]  R. Clarnette,et al.  Comparison of the Quick Mild Cognitive Impairment (Qmci) screen to the Montreal Cognitive Assessment (MoCA) in an Australian geriatrics clinic , 2017, International journal of geriatric psychiatry.

[99]  Y. Ben-Shlomo,et al.  Clinical judgement by primary care physicians for the diagnosis of all-cause dementia or cognitive impairment in symptomatic people. , 2022, The Cochrane database of systematic reviews.

[100]  D. Harvey,et al.  MCI is Associated With Deficits in Everyday Functioning , 2006, Alzheimer disease and associated disorders.

[101]  Tejal Patel,et al.  The “Rising Tide” of dementia in Canada , 2015, Canadian pharmacists journal : CPJ = Revue des pharmaciens du Canada : RPC.

[102]  C. Littlewood,et al.  Does treating Alzheimer's disease early, delay institutionalisation? , 2010, International journal of geriatric psychiatry.

[103]  K. Pottie,et al.  Recommendations on screening for developmental delay , 2016, Canadian Medical Association Journal.

[104]  L. Clare,et al.  Cognitive training for people with mild to moderate dementia. , 2018, The Cochrane database of systematic reviews.

[105]  S. Black,et al.  Screening for mild cognitive impairment: If not now, when? , 2016, Canadian Medical Association Journal.

[106]  Victor R. Preedy,et al.  United Nations Population Division , 2010 .

[107]  A. Larner,et al.  Zarit Burden Interview: pragmatic study in a dedicated cognitive function clinic , 2015 .

[108]  D. Fitzpatrick-Lewis,et al.  Treatment for mild cognitive impairment: a systematic review and meta-analysis. , 2015, CMAJ open.

[109]  Sara Graziadio,et al.  Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard – An update , 2019, PloS one.

[110]  A. M. Clarfield,et al.  The decreasing prevalence of reversible dementias: an updated meta-analysis. , 2003, Archives of internal medicine.

[111]  L. Clare,et al.  Cognitive training for people with mild to moderate dementia. , 2018, The Cochrane database of systematic reviews.

[112]  P. Panegyres,et al.  Early Dementia Screening , 2016, Diagnostics.

[113]  Claude Sicotte,et al.  Assessing Task–Technology Fit in a PACS Upgrade: Do Users’ and Developers’ Appraisals Converge? , 2011, Journal of Digital Imaging.

[114]  D. Yu,et al.  The effectiveness of physical exercise on cognitive and psychological outcomes in individuals with mild cognitive impairment: A systematic review and meta-analysis. , 2018, International journal of nursing studies.

[115]  R. Petersen,et al.  Mild cognitive impairment and mild dementia: a clinical perspective. , 2014, Mayo Clinic proceedings.

[116]  J. Neuhaus,et al.  Dementia=(MC)2: A 4-item Screening Test for Mild Cognitive Impairment and Dementia , 2011, Alzheimer disease and associated disorders.

[117]  H. Chertkow,et al.  Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sample , 2012, Neurology.

[118]  P. Yap,et al.  The optimal short version of the Zarit Burden Interview for dementia caregivers: diagnostic utility and externally validated cutoffs , 2019, Aging & mental health.

[119]  M. Nuno,et al.  The Efficacy of Cognitive Intervention in Mild Cognitive Impairment (MCI): a Meta-Analysis of Outcomes on Neuropsychological Measures , 2017, Neuropsychology Review.

[120]  Simon A Naji,et al.  Health measurement scales: a practical guide to their development and use (5th edition). , 1990, Australian and New Zealand journal of public health.

[121]  Ronald C. Petersen,et al.  Practice guideline update summary: Mild cognitive impairment , 2018, Neurology.

[122]  Andrew J. Saykin,et al.  Relationship between the Montreal Cognitive Assessment and Mini-mental State Examination for assessment of mild cognitive impairment in older adults , 2015, BMC Geriatrics.

[123]  A. Larner Short Montreal Cognitive Assessment , 2017, Journal of geriatric psychiatry and neurology.

[124]  Allan Gustavo Brigola,et al.  Brief version of Zarit Burden Interview (ZBI) for burden assessment in older caregivers , 2019, Dementia & neuropsychologia.

[125]  Laura Johnson,et al.  How Many Interviews Are Enough? , 2006 .