Greedy AutoAugment

A major problem in data augmentation is to ensure that the generated new samples cover the search space. This is a challenging problem and requires exploration for data augmentation policies to ensure their effectiveness in covering the search space. In this paper, we propose Greedy AutoAugment as a highly efficient search algorithm to find the best augmentation policies. We use a greedy approach to reduce the exponential growth of the number of possible trials to linear growth. The Greedy Search also helps us to lead the search towards the sub-policies with better results, which eventually helps to increase the accuracy. The proposed method can be used as a reliable addition to the current artifitial neural networks. Our experiments on four datasets (Tiny ImageNet, CIFAR-10, CIFAR-100, and SVHN) show that Greedy AutoAugment provides better accuracy, while using 360 times fewer computational resources.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Geoffrey E. Hinton,et al.  Matrix capsules with EM routing , 2018, ICLR.

[3]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Yoshua Bengio,et al.  Better Mixing via Deep Representations , 2012, ICML.

[6]  Dragomir Anguelov,et al.  Capturing Long-Tail Distributions of Object Subcategories , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[9]  Thomas Brox,et al.  Discriminative Unsupervised Feature Learning with Convolutional Neural Networks , 2014, NIPS.

[10]  Yoshua Bengio,et al.  Deep Directed Generative Autoencoders , 2014, ArXiv.

[11]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Dimitris N. Metaxas,et al.  Condensed Silhouette: An Optimized Filtering Process for Cluster Selection in K-Means , 2020, KES.

[14]  Ion Stoica,et al.  Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules , 2019, ICML.

[15]  Ya Le,et al.  Tiny ImageNet Visual Recognition Challenge , 2015 .

[16]  Michael Brown,et al.  Pillow: 3.1.0 , 2016 .

[17]  Fei Yang,et al.  Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Sylvain Paris,et al.  Deep Photo Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Saman Ghili,et al.  Tiny ImageNet Visual Recognition Challenge , 2014 .

[20]  Graham W. Taylor,et al.  Dataset Augmentation in Feature Space , 2017, ICLR.

[21]  Kilian Q. Weinberger,et al.  Deep Networks with Stochastic Depth , 2016, ECCV.

[22]  Yifan Zhang,et al.  Face Clustering in Videos with Proportion Prior , 2015, IJCAI.

[23]  Yi Yang,et al.  Random Erasing Data Augmentation , 2017, AAAI.

[24]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[26]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Kuan-Ta Chen,et al.  Automation of the kidney function prediction and classification through ultrasound-based kidney imaging using deep learning , 2019, npj Digital Medicine.

[28]  Xiangyu Zhang,et al.  ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design , 2018, ECCV.

[29]  Martial Hebert,et al.  Learning to Model the Tail , 2017, NIPS.

[30]  Graham W. Taylor,et al.  Improved Regularization of Convolutional Neural Networks with Cutout , 2017, ArXiv.

[31]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[32]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[33]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[34]  Quoc V. Le,et al.  AutoAugment: Learning Augmentation Policies from Data , 2018, ArXiv.

[35]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[36]  Geoffrey E. Hinton,et al.  Transforming Autoencoders , 2011 .

[37]  Hiroshi Inoue,et al.  Data Augmentation by Pairing Samples for Images Classification , 2018, ArXiv.

[38]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[40]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[41]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.