Deep features class activation map for thermal face detection and tracking

Recently, capabilities of many computer vision tasks have significantly improved due to advances in Convolutional Neural Networks. In our research, we demonstrate that it can be also used for face detection from low resolution thermal images, acquired with a portable camera. The physical size of the camera used in our research allows for embedding it in a wearable device or indoor remote monitoring solution for elderly and disabled people. The benefits of the proposed architecture were experimentally verified on the thermal video sequences, acquired in various scenarios to address possible limitations of remote diagnostics: movements of the person performing a diagnose and movements of the examined person. The achieved short processing time (42.05±0.21ms) along with high model accuracy (false positives - 0.43%; true positives for the patient focused on a certain task - 89.2%) clearly indicates that the current state of the art in the area of image classification and face tracking in thermography was significantly outperformed.

[1]  Pradeep Buddharaju,et al.  Physiology-Based Face Recognition in the Thermal Infrared Spectrum , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[3]  Marios Anthimopoulos,et al.  Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis , 2016, IEEE journal of biomedical and health informatics.

[4]  Mariusz Marzec,et al.  Detection of selected face areas on thermograms with elimination of typical problems , 2010 .

[5]  Jacek Ruminski,et al.  Real-time facial feature tracking in poor quality thermal imagery , 2016, 2016 9th International Conference on Human System Interactions (HSI).

[6]  Xulong Liu,et al.  Quantitative assessment of facial paralysis using infrared thermal imaging , 2015, 2015 8th International Conference on Biomedical Engineering and Informatics (BMEI).

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Jacek Ruminski,et al.  Evaluation of Respiration Rate Using Thermal Imaging in Mobile Conditions , 2017 .

[9]  Tomasz Kocejko,et al.  The data exchange between smart glasses and healthcare information systems using the HL7 FHIR standard , 2016, 2016 9th International Conference on Human System Interactions (HSI).

[10]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Jacek Ruminski,et al.  Quality of graphical markers for the needs of eyewear devices , 2015, 2015 8th International Conference on Human System Interaction (HSI).

[12]  Andreas Uhl,et al.  CNN transfer learning for the automated diagnosis of celiac disease , 2016, 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[13]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[14]  Svetlana Lazebnik,et al.  Active Object Localization with Deep Reinforcement Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Tong Zhang,et al.  A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , 2005, J. Mach. Learn. Res..

[16]  Bao Lei,et al.  Reuse of Motion Processing for Camera Stabilization and Video Coding , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[17]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  J. Rumiński,et al.  Face detection in image sequences using a portable thermal camera , 2016 .

[21]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[22]  Inas A. Yassine,et al.  Automatic localization of the left ventricle in cardiac MRI images using deep learning , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[23]  J. Rumiński Evaluation of respiration rate and pattern using a portable thermal camera , 2016 .

[24]  Sebastian Thrun,et al.  Is Learning The n-th Thing Any Easier Than Learning The First? , 1995, NIPS.