Multi-View Ground-Based Cloud Recognition by Transferring Deep Visual Information

Since cloud images captured from different views possess extreme variations, multi-view ground-based cloud recognition is a very challenging task. In this paper, a study of view shift is presented in this field. We focus both on designing proper feature representation and learning distance metrics from sample pairs. Correspondingly, we propose transfer deep local binary patterns (TDLBP) and weighted metric learning (WML). On one hand, to deal with view shift, like variations of illuminations, locations, resolutions and occlusions, we first utilize cloud images to train a convolutional neural network (CNN), and then extract local features from the part summing maps (PSMs) based on feature maps. Finally, we maximize the occurrences of regions for the final feature representation. On the other hand, the number of cloud images in each category varies greatly, leading to the unbalanced similar pairs. Hence, we propose a weighted strategy for metric learning. We validate the proposed method on three cloud datasets (the MOC_e, IAP_e, and CAMS_e) that are collected by different meteorological organizations in China, and the experimental results show the effectiveness of the proposed method.

[1]  Yunxue Shao,et al.  Soft-signed sparse coding for ground-based cloud classification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[2]  Andrew Zisserman,et al.  A Statistical Approach to Material Classification Using Image Patch Exemplars , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Subhransu Maji,et al.  Deep filter banks for texture recognition and segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Josep Calbó,et al.  Feature Extraction from Whole-Sky Ground-Based Images for Cloud-Type Recognition , 2008 .

[5]  Xiang Bai,et al.  Robust Scene Text Recognition with Automatic Rectification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Du Tran,et al.  Human Activity Recognition with Metric Learning , 2008, ECCV.

[7]  Josep Calbó,et al.  Retrieving Cloud Characteristics from Ground-Based Daytime Color All-Sky Images , 2006 .

[8]  Gianluca Francini,et al.  Optimized Deep Neural Networks for Real-Time Object Classification on Embedded GPUs , 2017 .

[9]  Zhiguo Cao,et al.  DeepCloud: Ground-Based Cloud Image Categorization Using Deep Convolutional Features , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Jakob Verbeek,et al.  Heterogeneous Face Recognition with CNNs , 2016, ECCV Workshops.

[11]  Yunxue Shao,et al.  Illumination-invariant completed LTP descriptor for cloud classification , 2012, 2012 5th International Congress on Image and Signal Processing.

[12]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Nitesh V. Chawla,et al.  SMOTEBoost: Improving Prediction of the Minority Class in Boosting , 2003, PKDD.

[14]  Apichat Heednacram,et al.  Feature extraction techniques for ground-based cloud type classification , 2015, Expert Syst. Appl..

[15]  Zhao Shi-jun.  Classification of Whole Sky Infrared Cloud Image Based on the LBP Operator , 2009 .

[16]  J. Shields,et al.  The Whole Sky Imager - A Year of Progress , 1998 .

[17]  Wen Liu,et al.  Texture Analysis and Land Cover Classification of Tehran Using Polarimetric Synthetic Aperture Radar Imagery , 2017 .

[18]  A. Heinle,et al.  Automatic cloud classification of whole sky images , 2010 .

[19]  Gang Wang,et al.  Localized Multifeature Metric Learning for Image-Set-Based Face Recognition , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Andrea Vedaldi,et al.  Visualizing Deep Convolutional Neural Networks Using Natural Pre-images , 2015, International Journal of Computer Vision.

[21]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[22]  Matti Pietikäinen,et al.  Local binary features for texture classification: Taxonomy and experimental study , 2017, Pattern Recognit..

[23]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[24]  Jian Sun,et al.  Object Detection Networks on Convolutional Feature Maps , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Taghi M. Khoshgoftaar,et al.  RUSBoost: A Hybrid Approach to Alleviating Class Imbalance , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[26]  Chunheng Wang,et al.  Deep Convolutional Activations-Based Features for Ground-Based Cloud Classification , 2017, IEEE Geoscience and Remote Sensing Letters.

[27]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[28]  Victor S. Lempitsky,et al.  Aggregating Local Deep Features for Image Retrieval , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Yan Zhang,et al.  On the Euclidean distance of images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Andrew Zisserman,et al.  A Statistical Approach to Texture Classification from Single Images , 2004, International Journal of Computer Vision.

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Kohei Mizutani,et al.  Infrared Cloud Imager Deployment at the North Slope of Alaska During Early 2002 , 2002 .

[33]  Prateek Jain,et al.  Fast Similarity Search for Learned Metrics , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Yunxue Shao,et al.  Ground-based cloud classification using multiple random projections , 2012, 2012 International Conference on Computer Vision in Remote Sensing.

[35]  Paul W. Fieguth,et al.  Texture Classification from Random Features , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[37]  Lian Jin-gen.  Whole sky infrared cloud measuring system based on the uncooled infrared focal plane array , 2008 .

[38]  Z. Chen,et al.  Sky Model Study Using Fuzzy Mathematics , 1994 .

[39]  Yunxue Shao,et al.  Salient local binary pattern for ground-based cloud classification , 2013, Acta Meteorologica Sinica.

[40]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[41]  Alex Pentland,et al.  Fractal-Based Description of Natural Scenes , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Yang Xiao,et al.  Ground-based cloud image categorization using deep convolutional visual features , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[43]  Maneesha Singh,et al.  Automated ground-based cloud recognition , 2005, Pattern Analysis and Applications.

[44]  Chunheng Wang,et al.  Cross-View Action Recognition via a Continuous Virtual Path , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Ali Ghodsi,et al.  Distance metric learning vs. Fisher discriminant analysis , 2008, AAAI 2008.

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..