Multi-Task Convolutional Neural Network for Patient Detection and Skin Segmentation in Continuous Non-Contact Vital Sign Monitoring

Patient detection and skin segmentation are important steps in non-contact vital sign monitoring as skin regions contain pulsatile information required for the estimation of vital signs such as heart rate, respiratory rate and peripheral oxygen saturation (SpO2). Previous methods based on face detection or colour-based image segmentation are less reliable in a hospital setting. In this paper, we develop a multi-task convolutional neural network (CNN) for detecting the presence of a patient and segmenting the patient’s skin regions. The multi-task model has a shared core network with two branches: a segmentation branch which was implemented using a fully convolutional network, and a classification branch which was implemented using global average pooling. The whole network was trained using images from a clinical study conducted in the neonatal intensive care unit (NICU) of the John Radcliffe hospital, Oxford, UK. Our model can produce accurate results and is robust to changes in different skin tones, pose variations, lighting variations, and routine interaction of clinical staff.