Ground-based cloud image categorization using deep convolutional visual features

Ground-based cloud image categorization is an essential and challenging task in automatic sky and cloud observation field. Till now, it still has not been well addressed in both meteorology and image processing communities, due to the large variation of cloud appearance. One feasible way to solve this is to find more discriminative visual representation to characterize the different kinds of clouds. Many efforts have been paid in this way. However, to our knowledge, most of the existing methods only resort to the hand-craft visual descriptors (e.g., LBP, CENTRIST and color histogram). The resulting performance is unfortunately not satisfied enough. Inspired by the great success of deep convolutional neural networks (CNN) in large-scale image classification task (e.g., ImageNet challenge), we first propose to transfer CNN to solve our relative small-scale cloud classification issue. The experiments on two challenging cloud datasets demonstrate that, using the deep convolutional visual features generated by CNN can significantly outperform all the state-of-the-art methods in most cases. Another important contribution of our work is that, we find that applying Fisher Vector (FV) to encoding the off-the-shelf CNN features can further leverage the performance.