Reading Face, Reading Health: Exploring Face Reading Technologies for Everyday Health

With the recent advancement in computer vision, Artificial Intelligence (AI), and mobile technologies, it has become technically feasible for computerized Face Reading Technologies (FRTs) to learn about one's health in everyday settings. However, how to design FRT-based applications for everyday health practices remains unexplored. This paper presents a design study with a technology probe called Faced, a mobile health checkup application based on the facial diagnosis method from Traditional Chinese Medicine (TCM). A field trial of Faced with 10 participants suggests potential usage modes and highlights a number of critical design issues in the use of FRTs for everyday health, including adaptability, practicality, sensitivity, and trustworthiness. We end by discussing design implications to address the unique challenges of fully integrating FRTs into everyday health practices.

[1]  I. Olkin,et al.  Using pedometers to increase physical activity and improve health: a systematic review. , 2007, JAMA.

[2]  Yvonne Rogers,et al.  Concealing or Revealing Mobile Medical Devices?: Designing for Onstage and Offstage Presentation , 2015, CHI.

[3]  Geraldine Fitzpatrick,et al.  Understanding the Mundane Nature of Self-care: Ethnographic Accounts of People Living with Parkinson's , 2018, CHI.

[4]  Nervo Verdezoto,et al.  Beyond self-monitoring: understanding non-functional aspects of home-based healthcare technology , 2013, UbiComp.

[5]  Jakob E. Bardram,et al.  Designing mobile health technology for bipolar disorder: a field trial of the monarca system , 2013, CHI.

[6]  Fang Li,et al.  Online fortune telling system using biometrics feature , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[7]  Gari D. Clifford,et al.  Shortliffe Edward H, Cimino James J: "Biomedical Informatics; Computer Applications in Health Care and Biomedicine" , 2006 .

[8]  StashNatalia,et al.  The effects of transparency on trust in and acceptance of a content-based art recommender , 2008 .

[9]  J. Cafazzo,et al.  Mobile phone-based remote patient monitoring system for management of hypertension in diabetic patients. , 2007, American journal of hypertension.

[10]  Judy Kay,et al.  Using an intergenerational communications system as a 'light-weight' technology probe , 2006, CHI EA '06.

[11]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[12]  Fufeng Li,et al.  Qualitative and Quantitative Analysis for Facial Complexion in Traditional Chinese Medicine , 2014, BioMed research international.

[13]  Nadia Bianchi-Berthouze,et al.  Supporting Everyday Function in Chronic Pain Using Wearable Technology , 2017, CHI.

[14]  Silvia Lindtner,et al.  Fish'n'Steps: Encouraging Physical Activity with an Interactive Computer Game , 2006, UbiComp.

[15]  Jie Yang,et al.  An Image Analysis System for Tongue Diagnosis in Traditional Chinese Medicine , 2004, CIS.

[16]  Allison Druin,et al.  Technologies for families , 2002, CHI Extended Abstracts.

[17]  Allison Druin,et al.  Technology probes: inspiring design for and with families , 2003, CHI '03.

[18]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Sunny Consolvo,et al.  Lullaby: a capture & access system for understanding the sleep environment , 2012, UbiComp.

[20]  Wang Zh.  Study on Face Shape of Patients with Five-Zang Organs Diseases , 2014 .

[21]  Sergio Escalera,et al.  Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Wenqiang Zhang,et al.  Robust Lip Segmentation Based on Complexion Mixture Model , 2016, PCM.

[23]  J. Anastasi,et al.  Tongue Inspection in TCM: Observations in a Study Sample of Patients Living with HIV. , 2014, Medical acupuncture.

[24]  Deborah Lupton,et al.  Self-tracking, health and medicine , 2017 .

[25]  Kiyoshi Yasuda,et al.  Remote Reminiscence Talking and Scheduling Prompter for Individuals with Dementia Using Video Phone , 2009, HCI.

[26]  Jian-Jun Shu,et al.  Developing classification indices for Chinese pulse diagnosis. , 2007, Complementary therapies in medicine.

[27]  Kerry Reynolds,et al.  Primary Care Providers' Beliefs About Teen and Parent Barriers to Depression Care , 2014, Journal of developmental and behavioral pediatrics : JDBP.

[28]  Wang Qi,et al.  Status and Prospect of Constitutional Theory in Traditional Chinese Medicine , 2002 .

[29]  William W. Gaver,et al.  Design: Cultural probes , 1999, INTR.

[30]  R. Weisberg.  A-N-D , 2011 .

[31]  Lora Aroyo,et al.  The effects of transparency on trust in and acceptance of a content-based art recommender , 2008, User Modeling and User-Adapted Interaction.

[32]  Bogdan J. Matuszewski,et al.  Mirror mirror on the wall… An intelligent multisensory mirror for well-being self-assessment , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[33]  Lena Mamykina,et al.  MAHI: investigation of social scaffolding for reflective thinking in diabetes management , 2008, CHI.

[34]  J. Flegr,et al.  Perceived Intelligence Is Associated with Measured Intelligence in Men but Not Women , 2014, PloS one.

[35]  Sarvapali D. Ramchurn,et al.  The Effect of Displaying System Confidence Information on the Usage of Autonomous Systems for Non-specialist Applications: A Lab Study , 2016, CHI.

[36]  Theodore Antonakopoulos,et al.  Probe-based ultrahigh-density storage technology , 2008, IBM J. Res. Dev..

[37]  John Zimmerman,et al.  Designing for slowness, anticipation and re-visitation: a long term field study of the photobox , 2014, CHI.

[38]  David A. Kindig,et al.  Health literacy : a prescription to end confusion , 2004 .

[39]  Sara Colantonio,et al.  User acceptance of self-monitoring technology to prevent cardio-metabolic diseases: The Wize Mirror , 2017, 2017 IEEE 13th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).

[40]  Michel Owayjan,et al.  Face Recognition Security System , 2015 .

[41]  Jun Rekimoto,et al.  Smiling makes us happier: enhancing positive mood and communication with smile-encouraging digital appliances , 2011, UbiComp '11.

[42]  Sean A. Munson,et al.  Beyond Abandonment to Next Steps: Understanding and Designing for Life after Personal Informatics Tool Use , 2016, CHI.

[43]  Rui Guo,et al.  Analysis and Recognition of Traditional Chinese Medicine Pulse Based on the Hilbert-Huang Transform and Random Forest in Patients with Coronary Heart Disease , 2015, Evidence-based complementary and alternative medicine : eCAM.

[44]  Yvonne Rogers,et al.  Designing for positive health affect: Decoupling negative emotion and health monitoring technologies , 2013, 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops.

[45]  K. C. Nwosu,et al.  MOBILE FACIAL RECOGNITION SYSTEM FOR PATIENT IDENTIFICATION IN MEDICAL EMERGENCIES FOR DEVELOPING ECONOMIES , 2018, Journal for the Advancement of Developing Economies.

[46]  Paul Marshall,et al.  Quantifying the Body and Caring for the Mind: Self-Tracking in Multiple Sclerosis , 2017, CHI.

[47]  F. Cheung.  TCM: Made in China , 2011, Nature.

[48]  Allison Druin,et al.  Designing an Interactive Message Board as a Technology Probe forFamily Communication , 2001 .

[49]  Min Dai,et al.  Therapeutic effect of traditional Chinese medicine on coagulation disorder and accompanying intractable jaundice in hepatitis B virus-related liver cirrhosis patients. , 2008, World journal of gastroenterology.

[50]  Mohan Karunanithi,et al.  A mobile phone-based care model for outpatient cardiac rehabilitation: the care assessment platform (CAP) , 2010, BMC cardiovascular disorders.

[51]  Enrica Papi,et al.  A knee monitoring device and the preferences of patients living with osteoarthritis: a qualitative study , 2015, BMJ Open.

[52]  Gregory D. Abowd,et al.  Barriers and Negative Nudges: Exploring Challenges in Food Journaling , 2015, CHI.

[53]  M. Shamim Hossain,et al.  Cloud-Assisted Speech and Face Recognition Framework for Health Monitoring , 2015, Mobile Networks and Applications.

[54]  Chang Liu,et al.  Computerized color analysis for facial diagnosis in traditional Chinese medicine , 2013, 2013 IEEE International Conference on Bioinformatics and Biomedicine.

[55]  Mohan S. Kankanhalli,et al.  Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda , 2018, CHI.

[56]  Therdpong Tempark,et al.  Chinese fortune-telling based on face and body mole positions: a hidden agenda regarding mole removal. , 2012, Archives of dermatology.

[57]  Yang Tao,et al.  Construction and application of eight principal syndrome differentiation knowledge base of tongue diagnosis based on ANN , 2010 .

[58]  Masataka Tokumaru,et al.  A Support System for Healthy Eating Habits: Optimization of Recipe Retrieval , 2013, HCI.

[59]  Geraldine Fitzpatrick,et al.  Self-Care Technologies in HCI: Trends, Tensions, and Opportunities , 2015, TCHI.

[60]  Phoebe Sengers,et al.  Fit4life: the design of a persuasive technology promoting healthy behavior and ideal weight , 2011, CHI.

[61]  Lena Mamykina,et al.  No longer wearing: investigating the abandonment of personal health-tracking technologies on craigslist , 2015, UbiComp.

[62]  Andrew D. Miller,et al.  StepStream: a school-based pervasive social fitness system for everyday adolescent health , 2014, CHI.