Multi-modality Mobile Image Recognition Based on Thermal and Visual Cameras

The advances of mobile computing and sensor technology have turned the mobile devices into powerful instruments. The integration of thermal and visual cameras extends the capability of computer vision, due to the fact that both images reveal different characteristics in images, however, image alignment is a challenge. This paper proposes an effective approach to align image pairs for event detection on mobile through image recognition. We leverage thermal and visual cameras as multi-modality sources for image recognition. By analyzing the heat pattern, the proposed APP can identify the heating sources and help users inspect their house heating system, on the other hand, with applying image recognition, the proposed APP furthermore can help field workers identify the asset condition and provide the guidance to solve their issues.

[1]  P. Anandan,et al.  Robust multi-sensor image alignment , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[2]  Sharath Pankanti,et al.  Adaptive as-natural-as-possible image stitching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[4]  Shao-Yi Chien,et al.  Baseball and tennis video annotation with temporal structure decomposition , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[5]  Mohan M. Trivedi,et al.  Multiperspective Thermal IR and Video Arrays for 3D Body Tracking and Driver Activity Analysis , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Ruxu Du,et al.  Diagnosis of Sheet Metal Stamping Processes Based on 3-D Thermal Energy Distribution , 2007, IEEE Trans Autom. Sci. Eng..

[8]  J.R. Martinez-De Dios,et al.  Automatic Detection of Windows Thermal Heat Losses in Buildings Using UAVs , 2006, 2006 World Automation Congress.

[9]  Noel E. O'Connor,et al.  Comparison of Fusion Methods for Thermo-Visual Surveillance Tracking , 2006, 2006 9th International Conference on Information Fusion.

[10]  Michael S. Brown,et al.  As-Projective-As-Possible Image Stitching with Moving DLT , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Aly A. Farag,et al.  A Non-invasive Method for Measuring Blood Flow Rate in Superficial Veins from a Single Thermal Image , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Thomas B. Moeslund,et al.  Thermal cameras and applications: a survey , 2013, Machine Vision and Applications.

[13]  Xiang Yi,et al.  Visible and infrared image registration based on visual salient features , 2015, J. Electronic Imaging.

[14]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[15]  Ching-Yung Lin,et al.  Concurrent image query using local random walk with restart on large scale graphs , 2014, 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[16]  Seong G. Kong,et al.  Recent advances in visual and infrared face recognition - a review , 2005, Comput. Vis. Image Underst..

[17]  Xia Liu,et al.  Pedestrian detection and tracking with night vision , 2005, IEEE Transactions on Intelligent Transportation Systems.

[18]  Guillaume-Alexandre Bilodeau,et al.  Fast and Accurate Registration of Visible and Infrared Videos , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.