An example of how false conclusions could be made with personalized health tracking and suggestions for avoiding similar situations

Personalizing interventions and treatments is a necessity for optimal medical care. Recent advances in computing, such as personal electronic devices, have made it easier than ever to collect and utilize vast amounts of personal data on individuals. This data could support personalized medicine; however, there are pitfalls that must be avoided. We discuss an example, longitudinal medical tracking, in which traditional methods of evaluating machine learning algorithms fail and present the opportunity for false conclusions. We then pose three suggestions for avoiding such opportunities for misleading results in medical applications, where reliability is essential.

[1]  D. Mohr,et al.  Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. , 2017, Annual review of clinical psychology.

[2]  S. Hollon,et al.  Defining empirically supported therapies. , 1998, Journal of consulting and clinical psychology.

[3]  Gary M. Weiss.  Mining with rarity: a unifying framework , 2004, SKDD.

[4]  Rui Wang,et al.  CrossCheck: toward passive sensing and detection of mental health changes in people with schizophrenia , 2016, UbiComp.

[5]  Oscar Mayora-Ibarra,et al.  Using smart phone mobility traces for the diagnosis of depressive and manic episodes in bipolar patients , 2014, AH.

[6]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[7]  Andrew T. Campbell,et al.  Next-generation psychiatric assessment: Using smartphone sensors to monitor behavior and mental health. , 2015, Psychiatric rehabilitation journal.

[8]  Alex Pentland,et al.  Social fMRI: Investigating and shaping social mechanisms in the real world , 2011, Pervasive Mob. Comput..

[9]  Matthew Hutson,et al.  Self-taught artificial intelligence beats doctors at predicting heart attacks , 2017 .

[10]  David Lazer,et al.  Inferring friendship network structure by using mobile phone data , 2009, Proceedings of the National Academy of Sciences.

[11]  R. Nelson-Gray,et al.  An overview of self-monitoring research in assessment and treatment. , 1999 .

[12]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[13]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[14]  Max A. Little,et al.  Using and understanding cross-validation strategies. Perspectives on Saeb et al. , 2017, GigaScience.

[15]  Konrad P. Kording,et al.  The need to approximate the use-case in clinical machine learning , 2017, GigaScience.

[16]  Zhigang Liu,et al.  The Jigsaw continuous sensing engine for mobile phone applications , 2010, SenSys '10.

[17]  Alan E Kazdin,et al.  Evidence-based treatment and practice: new opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. , 2008, The American psychologist.

[18]  Benjamin Recht,et al.  A step towards quantifying when an algorithm can and cannot predict an individual's wellbeing , 2017, UbiComp/ISWC Adjunct.

[19]  Mirco Musolesi,et al.  Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis , 2015, UbiComp.

[20]  Cem Ersoy,et al.  A Review and Taxonomy of Activity Recognition on Mobile Phones , 2013 .

[21]  Konrad Paul Kording,et al.  Mobile Phone Sensor Correlates of Depressive Symptom Severity in Daily-Life Behavior: An Exploratory Study , 2015, Journal of medical Internet research.

[22]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[23]  L. Piwek,et al.  The Rise of Consumer Health Wearables: Promises and Barriers , 2016, PLoS medicine.

[24]  H. Riper,et al.  Mobile Phone-Based Unobtrusive Ecological Momentary Assessment of Day-to-Day Mood: An Explorative Study , 2016, Journal of medical Internet research.

[25]  Konrad P. Körding,et al.  Meaningless comparisons lead to false optimism in medical machine learning , 2017, PloS one.

[26]  John Torous,et al.  2nd Symposia on Computing and Mental Health , 2017, CHI 2017.

[27]  Fanglin Chen,et al.  Unobtrusive sleep monitoring using smartphones , 2013, 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops.

[28]  Konrad Paul Kording,et al.  Distributed under Creative Commons Cc-by 4.0 the Relationship between Mobile Phone Location Sensor Data and Depressive Symptom Severity , 2022 .