The Intelligent ICU Pilot Study: Using Artificial Intelligence Technology for Autonomous Patient Monitoring

Currently, many critical care indices are repetitively assessed and recorded by overburdened nurses, e.g. physical function or facial pain expressions of nonverbal patients. In addition, many essential information on patients and their environment are not captured at all, or are captured in a non-granular manner, e.g. sleep disturbance factors such as bright light, loud background noise, or excessive visitations. In this pilot study, we examined the feasibility of using pervasive sensing technology and artificial intelligence for autonomous and granular monitoring of critically ill patients and their environment in the Intensive Care Unit (ICU). As an exemplar prevalent condition, we also characterized delirious and non-delirious patients and their environment. We used wearable sensors, light and sound sensors, and a high-resolution camera to collected data on patients and their environment. We analyzed collected data using deep learning and statistical analysis. Our system performed face detection, face recognition, facial action unit detection, head pose detection, facial expression recognition, posture recognition, actigraphy analysis, sound pressure and light level detection, and visitation frequency detection. We were able to detect patient's face (Mean average precision (mAP)=0.94), recognize patient's face (mAP=0.80), and their postures (F1=0.94). We also found that all facial expressions, 11 activity features, visitation frequency during the day, visitation frequency during the night, light levels, and sound pressure levels during the night were significantly different between delirious and non-delirious patients (p-value<0.05). In summary, we showed that granular and autonomous monitoring of critically ill patients and their environment is feasible and can be used for characterizing critical care conditions and related environment factors.

[1]  Alex Shenfield,et al.  What Faces Reveal: A Novel Method to Identify Patients at Risk of Deterioration Using Facial Expressions* , 2018, Critical care medicine.

[2]  B. Gehlbach,et al.  Looking for light in the din: An examination of the circadian-disrupting properties of a medical intensive care unit. , 2018, Intensive & critical care nursing.

[3]  J. Gagné.  Literature Review , 2018, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[4]  Kaberi Dasgupta,et al.  The impact of accelerometer wear location on the relationship between step counts and arterial stiffness in adults treated for hypertension and diabetes. , 2017, Journal of science and medicine in sport.

[5]  Maicon Falavigna,et al.  Effectiveness and Safety of an Extended ICU Visitation Model for Delirium Prevention: A Before and After Study* , 2017, Critical care medicine.

[6]  D. Needham,et al.  Measuring Patient Mobility in the ICU Using a Novel Noninvasive Sensor , 2017, Critical care medicine.

[7]  Eric P. Xing,et al.  ZM-Net: Real-time Zero-shot Image Manipulation Network , 2017, ArXiv.

[8]  I. Reiss,et al.  The significance of circadian rhythms and dysrhythmias in critical illness , 2017, Journal of the Intensive Care Society.

[9]  Jonathan Tompson,et al.  Towards Accurate Multi-person Pose Estimation in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[12]  M. Hanna,et al.  Self-Driving Cars , 2016, Computer.

[13]  C. Hough,et al.  ICU-Acquired Weakness. , 2016, Chest.

[14]  Ali Jalali,et al.  Advanced analytics for outcome prediction in intensive care units , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[15]  Michael Bailey,et al.  The ICU Mobility Scale Has Construct and Predictive Validity and Is Responsive. A Multicenter Observational Study. , 2016, Annals of the American Thoracic Society.

[16]  James M. Pivarnik,et al.  Validation and Comparison of Accelerometers Worn on the Hip, Thigh, and Wrists for Measuring Physical Activity and Sedentary Behavior , 2016, AIMS public health.

[17]  P. Brindley,et al.  Patient Isolation Precautions: Are They Worth It? , 2016, Canadian respiratory journal.

[18]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[19]  B. Van Rompaey,et al.  The patient's perception of a delirium: A qualitative research in a Belgian intensive care unit. , 2016, Intensive & critical care nursing.

[20]  Mahadev Satyanarayanan,et al.  OpenFace: A general-purpose face recognition library with mobile applications , 2016 .

[21]  J. Vincent,et al.  Paradigm shifts in critical care medicine: the progress we have made , 2015, Critical Care.

[22]  Benjamin K. Scott,et al.  Disruption of Circadian Rhythms and Sleep in Critical Illness and its Impact on the Development of Delirium. , 2015, Current pharmaceutical design.

[23]  Antonio Martinez-Nicolas,et al.  Disruption of Circadian Rhythms and Delirium, Sleep Impairment and Sepsis in Critically ill Patients. Potential Therapeutic Implications for Increased Light-Dark Contrast and Melatonin Therapy in an ICU Environment. , 2015, Current pharmaceutical design.

[24]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Linda Denehy,et al.  Assessment of impairment and activity limitations in the critically ill: a systematic review of measurement instruments and their clinimetric properties , 2015, Intensive Care Medicine.

[26]  Mao Zhang,et al.  Real-time continuous glucose monitoring versus conventional glucose monitoring in critically ill patients: a systematic review study protocol , 2015, BMJ Open.

[27]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Guillermo I Carrillo,et al.  A Controlled Clinical Trial , 2015 .

[30]  Anthony J. Culyer,et al.  Before and After Study , 2014 .

[31]  J Patel,et al.  The effect of a multicomponent multidisciplinary bundle of interventions on sleep and delirium in medical and surgical intensive care patients , 2014, Anaesthesia.

[32]  David W Bates,et al.  Continuous monitoring in an inpatient medical-surgical unit: a controlled clinical trial. , 2014, The American journal of medicine.

[33]  P. Trzepacz,et al.  Development of an abbreviated version of the Delirium Motor Subtyping Scale (DMSS-4) , 2014, International Psychogeriatrics.

[34]  J. D. Young,et al.  An investigation of sound levels on intensive care units with reference to the WHO guidelines , 2013, Critical Care.

[35]  H. Grocott,et al.  Effect of intensive care unit environment on in-hospital delirium after cardiac surgery. , 2013, The Journal of thoracic and cardiovascular surgery.

[36]  S. Intille,et al.  Estimating activity and sedentary behavior from an accelerometer on the hip or wrist. , 2013, Medicine and science in sports and exercise.

[37]  Taise Carneiro Araújo,et al.  Music: a care strategy for patients in intensive care unit , 2013 .

[38]  R. Jaeschke,et al.  Clinical Practice Guidelines for the Management of Pain, Agitation, and Delirium in Adult Patients in the Intensive Care Unit , 2013, Critical care medicine.

[39]  R. Polomano,et al.  Preliminary validation of the Defense and Veterans Pain Rating Scale (DVPRS) in a military population. , 2013, Pain medicine.

[40]  Linda M. Peelen,et al.  Intensive care unit environment may affect the course of delirium , 2013, Intensive Care Medicine.

[41]  L. Napolitano,et al.  Common complications in the critically ill patient. , 2012, The Surgical clinics of North America.

[42]  A. Thrush,et al.  The Clinical Utility of the Functional Status Score for the Intensive Care Unit (FSS-ICU) at a Long-Term Acute Care Hospital: A Prospective Cohort Study , 2012, Physical Therapy.

[43]  Paolo Pelosi,et al.  Delirium: clinical approach and prevention. , 2012, Best practice & research. Clinical anaesthesiology.

[44]  Maxime Cannesson,et al.  Monitoring in the Intensive Care , 2012, Critical care research and practice.

[45]  Bernt Schiele,et al.  Articulated people detection and pose estimation: Reshaping the future , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  J Mocco,et al.  The effect of increased mobility on morbidity in the neurointensive care unit. , 2012, Journal of neurosurgery.

[47]  Silvio Savarese,et al.  Articulated part-based model for joint object detection and pose estimation , 2011, 2011 International Conference on Computer Vision.

[48]  Dinesh John,et al.  Validation and comparison of ActiGraph activity monitors. , 2011, Journal of science and medicine in sport.

[49]  Sridha Sridharan,et al.  Automatically Detecting Pain in Video Through Facial Action Units , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[50]  D. Needham,et al.  Long-term complications of critical care , 2011, Critical care medicine.

[51]  Daniel McDuff,et al.  Affect valence inference from facial action unit spectrograms , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[52]  G. ÓLaighin,et al.  Motion analysis in delirium: a discrete approach in determining physical activity for the purpose of delirium motoric subtyping. , 2010, Medical engineering & physics.

[53]  N. Halpern,et al.  Critical care medicine in the United States 2000–2005: An analysis of bed numbers, occupancy rates, payer mix, and costs* , 2010, Critical care medicine.

[54]  V. Preedy,et al.  Prospective Cohort Study , 2010 .

[55]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Jitendra Malik,et al.  Poselets: Body part detectors trained using 3D human pose annotations , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[57]  G. ÓLaighin,et al.  A classification system for delirium subtyping with the use of a commercial mobility monitor. , 2009, Gait & posture.

[58]  R. Dittus,et al.  Delirium in older emergency department patients: recognition, risk factors, and psychomotor subtypes. , 2009, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[59]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[60]  Maeve Leonard,et al.  The delirium experience: a review. , 2008, Journal of psychosomatic research.

[61]  P. Trzepacz,et al.  Motor symptoms in 100 patients with delirium versus control subjects: comparison of subtyping methods. , 2008, Psychosomatics.

[62]  Mark A. Moskowitz,et al.  Complications of care in a medical intensive care unit , 1990, Journal of General Internal Medicine.

[63]  P. Mahadevan,et al.  An overview , 2007, Journal of Biosciences.

[64]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[65]  L. Hirsch.  Continuous EEG Monitoring in the Intensive Care Unit: An Overview , 2004, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[66]  Robert A. Dobie,et al.  Hearing Loss: Determining Eligibility for Social Security Benefits , 2004 .

[67]  Daniel P. Huttenlocher,et al.  Pictorial Structures for Object Recognition , 2004, International Journal of Computer Vision.

[68]  Theodore Speroff,et al.  Evaluation of delirium in critically ill patients: Validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) , 2001, Critical care medicine.

[69]  Russ B. Altman,et al.  Missing value estimation methods for DNA microarrays , 2001, Bioinform..

[70]  R. Schwab,et al.  Patient perception of sleep quality and etiology of sleep disruption in the intensive care unit. , 1999, American journal of respiratory and critical care medicine.

[71]  David F. O'connell.  Dual Disorders: Essentials for Assessment and Treatment , 1997 .

[72]  Mark J. Smith,et al.  The Memorial Delirium Assessment Scale. , 1997, Journal of pain and symptom management.

[73]  D. Lundberg,et al.  Intensive care syndrome: a literature review. , 1996, Intensive & critical care nursing.

[74]  C M Wollschlager,et al.  Common complications in critically ill patients. , 1988, Disease-a-month : DM.

[75]  Robert C. Wolpert,et al.  A Review of the , 1985 .