Automated approaches for ROIs extraction in medical thermography: a review and future directions

Historically, the skin temperature has been used as a functional indicator of the processes undergoing inside the body, where the anomalous change in temperature gives indication of illness. Infrared thermography (IRT) is a non-invasive approach which radiometrically measures the temperature distribution on the object’s surface based on emanating IR radiation. In medical field, IRT has been used to diagnose the various problems associated with superficial body parts since 1960s. The clinical abnormalities can be diagnosed by visual and subjective analysis of the thermograms, but human based diagnoses are likely to be influenced by narcissus effect, negligence, visual exhaustion and mental workload. Apart, the deep and objective diagnostic information, like severity and type of a disease can only be obtained by statistically analyse the region of interest (ROI). So, the segmentation of ROI has often been an initial step in medical diagnosis, where the automated approaches lead to fast and highly reproducible analysis. Recent advancement in deep learning opens a new way for effective computer-aided medical diagnosis, several studies have proposed deep learning based automatic segmentation and data classification. This review focuses on the wide range of image processing techniques adopted for automatic segmentation of various clinically significant ROIs, like breast, face, foot, hand etc. In addition, the present efforts are targeted on the selection of characteristic points for an effective automatic segmentation and identification of new areas to carry out the further research. Besides, the basics of medical IRT, need of automatic segmentation and factors which hinder the automatic segmentation in IRT are also discussed.

[1]  I. Cuevas,et al.  Reliability and reproductibility of skin temperature of overweight subjects by an infrared thermograpy software designed for human beings , 2012 .

[2]  Javad Haddadnia,et al.  Segmenting breast cancerous regions in thermal images using fuzzy active contours , 2016, EXCLI journal.

[3]  G. L. Nandagopan,et al.  Implementation and comparison of two image segmentation techniques on thermal foot images and detection of ulceration using asymmetry , 2016, 2016 International Conference on Communication and Signal Processing (ICCSP).

[4]  A. Conci,et al.  Topological Derivative Applied to Automatic Segmentation of Frontal Breast Thermograms , 2013 .

[5]  J. Selfe,et al.  An accurate and reliable method of thermal data analysis in thermal imaging of the anterior knee for use in cryotherapy research. , 2006, Archives of physical medicine and rehabilitation.

[6]  Payel Ghosh,et al.  Segmentation of medical images using a genetic algorithm , 2006, GECCO.

[7]  U. Rajendra Acharya,et al.  Thermography Based Breast Cancer Detection Using Texture Features and Support Vector Machine , 2012, Journal of Medical Systems.

[8]  Naima Kaabouch,et al.  Enhancement of the asymmetry-based overlapping analysis through features extraction , 2011, J. Electronic Imaging.

[9]  Naima Kaabouch,et al.  Alternative Technique to Asymmetry Analysis-Based Overlapping for Foot Ulcer Examination: Scalable Scanning , 2016, ArXiv.

[10]  T. Jakubowska,et al.  Thermal signatures for breast cancer screening comparative study , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[11]  Robert Koprowski,et al.  Automatic analysis of the trunk thermal images from healthy subjects and patients with faulty posture , 2015, Comput. Biol. Medicine.

[12]  Naima Kaabouch,et al.  Asymmetry analysis based on genetic algorithms for the prediction of foot ulcers , 2009, Electronic Imaging.

[13]  D. Hernandez-Contreras,et al.  Automatic classification of thermal patterns in diabetic foot based on morphological pattern spectrum , 2015 .

[14]  Mohamed Abdel-Nasser,et al.  Automatic nipple detection in breast thermograms , 2016, Expert Syst. Appl..

[15]  Valentyn Novosad,et al.  Study of functional infrared imaging for early detection of mucositis in locally advanced head and neck cancer treated with chemoradiotherapy. , 2013, Oral oncology.

[16]  Ajat Shatru Arora,et al.  Thermographic evaluation of mindfulness meditation using dynamic IR imaging , 2018, Infrared Physics & Technology.

[17]  Alamelu Nachiappan,et al.  Feature Extraction Algorithms for Abnormality Quantification from Medical Thermograms , 2009 .

[18]  Leonardo Trujillo,et al.  Automatic Feature Localization in Thermal Images for Facial Expression Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[19]  Mario Ignacio Chacon Murguia,et al.  Automatic Segmentation of Regions of Interest in Breast Thermographic Images , 2015, Mexican Conference on Pattern Recognition.

[20]  Aura Conci,et al.  A New Database for Breast Research with Infrared Image , 2014 .

[21]  Pragati Kapoor,et al.  Image Segmentation and Asymmetry Analysis of Breast Thermograms for Tumor Detection , 2012 .

[22]  Adrian A. Hopgood,et al.  Automated overlay of infrared and visual medical images , 2008 .

[23]  Shangfei Wang,et al.  Eye Localization from Infrared Thermal Images , 2012, MPRSS.

[24]  Ajat Shatru Arora,et al.  An automated approach to enhance the thermographic evaluation on orofacial regions in lateral facial thermograms. , 2018, Journal of thermal biology.

[25]  Mariusz Marzec,et al.  Automatic method for detection of characteristic areas in thermal face images , 2013, Multimedia Tools and Applications.

[26]  S. S. Suganthi,et al.  Anisotropic diffusion filter based edge enhancement for segmentation of breast thermogram using level sets , 2014, Biomed. Signal Process. Control..

[27]  M. Frize,et al.  Analysis of breast thermography with an artificial neural network , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[28]  Kenneth P. Camilleri,et al.  Automated Segmentation and Temperature Extraction from Thermal Images of Human Hands, Shins and Feet , 2016 .

[29]  Jun Zhang,et al.  Dynamic infrared imaging for analysis of fingertip temperature after cold water stimulation and neurothermal modeling study , 2010, Comput. Biol. Medicine.

[30]  Walmir M. Caminhas,et al.  A Combined Method for Segmentation and Registration for an Advanced and Progressive Evaluation of Thermal Images , 2014, Sensors.

[31]  Pradeepkumar Dhage,et al.  Rheumatoid Arthritis Detection Using Thermal Imaging and Fuzzy-C-Means algorithm , 2014 .

[32]  Elham Mahmoudzadeh,et al.  Directional SUSAN image boundary detection of breast thermogram , 2016, IET Image Process..

[33]  Carl F. Pieper,et al.  Patellar Skin Surface Temperature by Thermography Reflects Knee Osteoarthritis Severity , 2010, Clinical medicine insights. Arthritis and musculoskeletal disorders.

[34]  M. Iqbal Saripan,et al.  Breast boundary segmentation in thermography images based on random walkers , 2017 .

[35]  Victor Machado Reis,et al.  The use of thermal imaging to monitoring skin temperature during cryotherapy: A systematic review , 2015 .

[36]  Chua Kuang Chua,et al.  Automated diagnosis of dry eye using infrared thermography images , 2015 .

[37]  Ricardo Vardasca,et al.  Towards the Automatic Detection of Hand Fingertips and Phalanges in Thermal Images , 2017 .

[38]  F. Ring,et al.  Standard Procedures for Infrared Imaging in Medicine , 2006 .

[39]  N. M. Nandhitha,et al.  Performance Evaluation of Image Processing Algorithms for Automatic Detection and Quantification of Abnormality in Medical Thermograms , 2008 .

[40]  U. Rajendra Acharya,et al.  Infrared thermography on ocular surface temperature: A review , 2009 .

[41]  T. Jayakumar,et al.  Medical applications of infrared thermography: A review , 2012, Infrared Physics & Technology.

[42]  Roshan Joy Martis,et al.  Asymmetry analysis of breast thermograms using automated segmentation and texture features , 2017, Signal Image Video Process..

[43]  Delaine Rodrigues-Bigaton,et al.  Intra and inter-rater reliability of infrared image analysis of masticatory and upper trapezius muscles in women with and without temporomandibular disorder. , 2013, Brazilian journal of physical therapy.

[44]  M. Etehadtavakol,et al.  Level set method for segmentation of infrared breast thermograms , 2014, EXCLI journal.

[45]  E. Y. K. Ng,et al.  Application of K- and Fuzzy c-Means for Color Segmentation of Thermal Infrared Breast Images , 2010, Journal of Medical Systems.

[46]  R.A. Goubran,et al.  Segmentation and Landmark Identification in Infrared Images of the Human Body , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[47]  Maja Pantic,et al.  Facial component detection in thermal imagery , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[48]  E. F. J. Ring,et al.  The historical development of temperature measurement in medicine , 2007 .

[49]  Ari Karppinen,et al.  Multimedia Tools and Applications for Environmental & Biodiversity Informatics , 2018, Multimedia Systems and Applications.

[50]  Keerthana Prasad,et al.  Segmentation of Breast Thermogram Images for the Detection of Breast Cancer: A Projection Profile Approach , 2015 .

[51]  Joonseok Lee,et al.  Thermographic assessment of inferior alveolar nerve injury in patients with dentofacial deformity. , 2007, Journal of oral and maxillofacial surgery : official journal of the American Association of Oral and Maxillofacial Surgeons.

[52]  Hairong Qi,et al.  Asymmetry analysis in breast cancer detection using thermal infrared images , 2002, Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology.

[53]  P. N. S. Rao,et al.  Correlation between Plantar Foot Temperature and Diabetic Neuropathy: A Case Study by Using an Infrared Thermal Imaging Technique , 2010, Journal of diabetes science and technology.

[54]  Saeed Setayeshi,et al.  A vascular network matching in dynamic thermography for breast cancer detection , 2015 .

[55]  Nizami Mohiyuddin Segmentation of Thermal Images for Evaluation of Rheumatoid Arthritis Disease , 2014 .

[56]  Pragati Kapoor,et al.  Image processing for early diagnosis of breast cancer using infrared images , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[57]  P Plassmann,et al.  Digital infrared thermal imaging of human skin. , 2002, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[58]  Silva Hecimovic,et al.  Loss of Cathepsin B and L Leads to Lysosomal Dysfunction, NPC-Like Cholesterol Sequestration and Accumulation of the Key Alzheimer's Proteins , 2016, PloS one.

[59]  S. Mashohor,et al.  Random walkers based segmentation method for breast thermography , 2012, 2012 IEEE-EMBS Conference on Biomedical Engineering and Sciences.

[60]  George Havenith,et al.  A method for whole-body skin temperature mapping in humans (Eine Methode zur Aufzeichnung der Hautemperatur des gesamten menschlichen Korpers) , 2012 .

[61]  Václav Snásel,et al.  Thermogram breast cancer prediction approach based on Neutrosophic sets and fuzzy c-means algorithm , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[62]  KOWSKI,et al.  Automatic eye corners detection and tracking algorithm in sequence of thermal medical images , 2015 .

[63]  Ferdi van der Heijden,et al.  Automatic detection of diabetic foot complications with infrared thermography by asymmetric analysis , 2015, Journal of biomedical optics.

[64]  Vinod Chandran,et al.  Breast cancer detection from thermal images using bispectral invariant features , 2013 .

[65]  Jangho Yoon,et al.  Fever Screening and Detection of Febrile Arrivals at an International Airport in Korea: Association among Self-reported Fever, Infrared Thermal Camera Scanning, and Tympanic Temperature , 2014, Epidemiology and health.

[66]  Ajat Shatru Arora,et al.  A framework for enhancing the thermographic evaluation on characteristic areas for paranasal sinusitis detection , 2017 .

[67]  Henrique A. Almeida,et al.  Segmentation Algorithms for Thermal Images , 2014 .

[68]  U. Rajendra Acharya,et al.  Automated study of ocular thermal images: Comprehensive analysis of corneal health with different age group subjects and validation , 2010, Digit. Signal Process..

[69]  Naima Kaabouch,et al.  Predicting neuropathic ulceration: analysis of static temperature distributions in thermal images. , 2010, Journal of biomedical optics.

[70]  Rajendra Acharya U,et al.  Automated detection of eye and cornea on infrared thermogram using snake and target tracing function coupled with genetic algorithm , 2009 .

[71]  Naima Kaabouch,et al.  Early detection of foot ulcers through asymmetry analysis , 2009, Medical Imaging.

[72]  B. Wiecek,et al.  Review on thermal image processing for passive and active thermography , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[73]  M. Strąkowska,et al.  Automatic eye corners detection and tracking algorithm in sequence of thermal medical images , 2015 .

[74]  Kelvin E. Donne,et al.  Optimized laser application in dermatology using infrared thermography , 2002, SPIE Defense + Commercial Sensing.

[75]  Payel Ghosh,et al.  Segmentation of thermographic images of hands using a genetic algorithm , 2010, Electronic Imaging.

[76]  Mariusz Marzec,et al.  Fast eye localization from thermal images using neural networks , 2016, Multimedia Tools and Applications.

[77]  G. R. Suresh,et al.  Automatic Diagnosis of Breast Cancer using Thermographic Color Analysis and SVM Classifier , 2016 .

[78]  Edward Ng,et al.  SEGMENTATION OF BREAST THERMOGRAM: IMPROVED BOUNDARY DETECTION WITH MODIFIED SNAKE ALGORITHM , 2006 .

[79]  Víctor H. Andaluz,et al.  Segmentation of infrared images: A new technology for early detection of breast diseases , 2015, 2015 IEEE International Conference on Industrial Technology (ICIT).

[80]  B. Venkatraman,et al.  Automated hand thermal image segmentation and feature extraction in the evaluation of rheumatoid arthritis , 2015, Proceedings of the Institution of Mechanical Engineers. Part H, Journal of engineering in medicine.

[81]  H. Qi,et al.  Detecting breast cancer from infrared images by asymmetry analysis , 2000, Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143).

[82]  D. Hernandez-Contreras,et al.  A quantitative index for classification of plantar thermal changes in the diabetic foot , 2017 .

[83]  Hairong Qi,et al.  Asymmetry analysis using automatic segmentation and classification for breast cancer detection in thermograms , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[84]  Maryam Zekri,et al.  Extended hidden Markov model for optimized segmentation of breast thermography images , 2015 .

[85]  Bryan F. Jones,et al.  A reappraisal of the use of infrared thermal image analysis in medicine , 1998, IEEE Transactions on Medical Imaging.

[86]  Yang Yang,et al.  Automated Analysis Method for Screening Knee Osteoarthritis using Medical Infrared Thermography , 2013 .

[87]  Rachid Harba,et al.  Automatic Analysis of Plantar Foot Thermal Images in at-Risk Type II Diabetes by Using an Infrared Camera , 2015 .

[88]  Mita Nasipuri,et al.  Hybrid Intelligent Techniques for Segmentation of Breast Thermograms , 2016 .

[89]  Snekhalatha Umapathy,et al.  Computer Aided Diagnosis Based Hand Thermal Image Analysis: A Potential Tool for the Evaluation of Rheumatoid Arthritis , 2018 .

[90]  M. Faundez-Zanuy,et al.  Thermal hand image segmentation for biometric recognition , 2013, IEEE Aerospace and Electronic Systems Magazine.

[91]  Naima Kaabouch,et al.  Automatic segmentation of thermal images of diabetic-at-risk feet using the snakes algorithm , 2017 .

[92]  Ferdi van der Heijden,et al.  Infrared dermal thermography on diabetic feet soles to predict ulcerations: a case study , 2013, Photonics West - Biomedical Optics.

[93]  Débora C. Muchaluat-Saade,et al.  A new measure for comparing biomedical regions of interest in segmentation of digital images , 2015, Discret. Appl. Math..

[94]  Robert L. Elliot,et al.  Infrared Imaging of the Breast — An Overview , 2006 .