Dataset Refinement for Convolutional Neural Networks via Active Learning

Convolutional neural networks (CNNs) have shown significant advantages in computer vision fields. For the optimizations of CNNs, most research works focus on feature extraction, which creates deeper structures and better activations, but the optimizations on dataset is rarely discussed. Due to the boom of data, most CNNs suffer from serious problems of dataset redundancy and following high computational burden. To this end, this paper brings in an informativeness ranking thought and proposes a new methodology for dataset refinement based on Active Learning. Extensive experiments prove its effectiveness to achieve a higher classification accuracy for CNNs at a less training cost. Moreover, for classification problems with a large number of class, this paper further proposes Entropy Ranking, a new Active Learning method, to enhance the optimization ability.

[1]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Peng Liu,et al.  Ieee Journal of Selected Topics in Applied Earth Observations and Remote Sensing 1 Active Deep Learning for Classification of Hyperspectral Images , 2022 .

[3]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[4]  Wei Liu,et al.  Exploring Representativeness and Informativeness for Active Learning , 2019, IEEE Transactions on Cybernetics.

[5]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[6]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Javier Pérez-Rodríguez,et al.  OligoIS: Scalable Instance Selection for Class-Imbalanced Data Sets , 2013, IEEE Transactions on Cybernetics.

[8]  William J. Emery,et al.  Using active learning to adapt remote sensing image classifiers , 2011 .

[9]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[10]  Hongming Zhou,et al.  Stacked Extreme Learning Machines , 2015, IEEE Transactions on Cybernetics.

[11]  William J. Emery,et al.  Improving active learning methods using spatial information , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[12]  Chun Chen,et al.  Active Learning Based on Locally Linear Reconstruction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Meng Yang,et al.  Large-Margin Softmax Loss for Convolutional Neural Networks , 2016, ICML.

[14]  Melba M. Crawford,et al.  Active Learning: Any Value for Classification of Remotely Sensed Data? , 2013, Proceedings of the IEEE.

[15]  Ishwar K. Sethi,et al.  Confidence-based active learning , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  William J. Emery,et al.  Active Learning Methods for Remote Sensing Image Classification , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[19]  Yann LeCun,et al.  Large-scale Learning with SVM and Convolutional for Generic Object Categorization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).