Active and Incremental Learning with Weak Supervision

Large amounts of labeled training data are one of the main contributors to the great success that deep models have achieved in the past. Label acquisition for tasks other than benchmarks can pose a challenge due to requirements of both funding and expertise. By selecting unlabeled examples that are promising in terms of model improvement and only asking for respective labels, active learning can increase the efficiency of the labeling process in terms of time and cost. In this work, we describe combinations of an incremental learning scheme and methods of active learning. These allow for continuous exploration of newly observed unlabeled data. We describe selection criteria based on model uncertainty as well as expected model output change (EMOC). An object detection task is evaluated in a continuous exploration context on the PASCAL VOC dataset. We also validate a weakly supervised system based on active and incremental learning in a real-world biodiversity application where images from camera traps are analyzed. Labeling only 32 images by accepting or rejecting proposals generated by our method yields an increase in accuracy from 25.4 to 42.6%.

[1]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[2]  Xingquan Zhu,et al.  I don't know the label: Active learning with blind knowledge , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[3]  Joachim Denzler,et al.  Large-scale gaussian process multi-class classification for semantic segmentation and facade recognition , 2013, Machine Vision and Applications.

[4]  Trevor Darrell,et al.  Latent Task Adaptation with Large-Scale Hierarchies , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  C. Lee Giles,et al.  Learning on the border: active learning in imbalanced data classification , 2007, CIKM '07.

[6]  Cordelia Schmid,et al.  Incremental Learning of Object Detectors without Catastrophic Forgetting , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Joachim Denzler,et al.  Towards Automated Visual Monitoring of Individual Gorillas in the Wild , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[8]  Peng Liu,et al.  Ieee Journal of Selected Topics in Applied Earth Observations and Remote Sensing 1 Active Deep Learning for Classification of Hyperspectral Images , 2022 .

[9]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Vladlen Koltun,et al.  Geodesic Object Proposals , 2014, ECCV.

[11]  Kristen Grauman,et al.  Large-scale live active learning: Training object detectors with crawled data and crowds , 2011, CVPR.

[12]  Rong Jin,et al.  Large-scale text categorization by batch mode active learning , 2006, WWW '06.

[13]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[14]  Andrew McCallum,et al.  Toward Optimal Active Learning through Sampling Estimation of Error Reduction , 2001, ICML.

[15]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Nikolaos Papanikolopoulos,et al.  Multi-class active learning for image classification , 2009, CVPR.

[17]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[18]  Vinay P. Namboodiri,et al.  Active learning with version spaces for object detection , 2016, ArXiv.

[19]  Daphne Koller,et al.  Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..

[20]  Jhony-Heriberto Giraldo-Zuluaga,et al.  Recognition of Mammal Genera on Camera-Trap Images Using Multi-layer Robust Principal Component Analysis and Mixture Neural Networks , 2017, 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI).

[21]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Ashish Kapoor,et al.  Active learning for large multi-class problems , 2009, CVPR.

[23]  Frank Keller,et al.  We Don’t Need No Bounding-Boxes: Training Object Class Detectors Using Only Human Verification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Margaret Kosmala,et al.  Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning , 2017, Proceedings of the National Academy of Sciences.

[25]  Kristen Grauman,et al.  Cost-Sensitive Active Visual Category Learning , 2010, International Journal of Computer Vision.

[26]  Joachim Denzler,et al.  Active and Continuous Exploration with Deep Neural Networks and Expected Model Output Changes , 2016, ArXiv.

[27]  Ramakrishnan Mukundan,et al.  HER2 challenge contest: a detailed assessment of automated HER2 scoring algorithms in whole slide images of breast cancer tissues , 2017, Histopathology.

[28]  Derek Hoiem,et al.  Learning without Forgetting , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Andrew McCallum,et al.  Toward Optimal Active Learning through Monte Carlo Estimation of Error Reduction , 2001, ICML 2001.

[30]  Luc Van Gool,et al.  Interactive object detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Margaret Kosmala,et al.  Automatically identifying wild animals in camera trap images with deep learning , 2017, ArXiv.

[32]  Joachim Denzler,et al.  Selecting Influential Examples: Active Learning with Expected Model Output Changes , 2014, ECCV.

[33]  Joachim Denzler,et al.  Large-Scale Active Learning with Approximations of Expected Model Output Changes , 2016, GCPR.

[34]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[35]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[36]  Joachim Denzler,et al.  Active learning and discovery of object categories in the presence of unnameable instances , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Yang Wang,et al.  A Discriminative Latent Model of Object Classes and Attributes , 2010, ECCV.

[38]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[39]  Joachim Denzler,et al.  Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding , 2015, VISAPP.

[40]  Frank Keller,et al.  Extreme Clicking for Efficient Object Annotation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[41]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[42]  Trevor Darrell,et al.  Gaussian Processes for Object Categorization , 2010, International Journal of Computer Vision.

[43]  Jesús Francisco Vargas-Bonilla,et al.  Towards automatic wild animal monitoring: Identification of animal species in camera-trap images using very deep convolutional neural networks , 2016, Ecol. Informatics.

[44]  Trevor Darrell,et al.  LSDA: Large Scale Detection through Adaptation , 2014, NIPS.

[45]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[46]  Li Fei-Fei,et al.  Crowdsourcing in Computer Vision , 2016, Found. Trends Comput. Graph. Vis..

[47]  Joachim Denzler,et al.  Fine-Tuning Deep Neural Networks in Continuous Learning Scenarios , 2016, ACCV Workshops.

[48]  Y. Freund,et al.  Active learning for visual object detection , 2005 .

[49]  Darryl I. MacKenzie,et al.  Occupancy as a surrogate for abundance estimation , 2004, Animal Biodiversity and Conservation.

[50]  Christoph H. Lampert,et al.  iCaRL: Incremental Classifier and Representation Learning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Joachim Denzler,et al.  Watch, Ask, Learn, and Improve: a lifelong learning cycle for visual recognition , 2016, ESANN.

[52]  Daniel Cremers,et al.  CAPTCHA Recognition with Active Deep Learning , 2015 .

[53]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[54]  Yu-Pu Yang,et al.  A batch-mode active learning SVM method based on semi-supervised clustering , 2015, Intell. Data Anal..

[55]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[56]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[57]  Joachim Denzler,et al.  Active Learning for Regression Tasks with Expected Model Output Changes , 2018, BMVC.

[58]  Dan Wang,et al.  A new active labeling method for deep learning , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[59]  Ruimao Zhang,et al.  Cost-Effective Active Learning for Deep Image Classification , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[60]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[61]  Joachim Denzler,et al.  Deep bilinear features for Her2 scoring in digital pathology , 2017 .

[62]  Joachim Denzler,et al.  Active Learning for Deep Object Detection , 2018, VISIGRAPP.

[63]  Hairong Liu,et al.  Active Learning for Speech Recognition: the Power of Gradients , 2016, ArXiv.

[64]  German Díez,et al.  Animal Identification in Low Quality Camera-Trap Images Using Very Deep Convolutional Neural Networks and Confidence Thresholds , 2016, ISVC.

[65]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Jhony-Heriberto Giraldo-Zuluaga,et al.  Camera-trap images segmentation using multi-layer robust principal component analysis , 2017, The Visual Computer.

[67]  C. Lintott,et al.  Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna , 2015, Scientific Data.

[68]  Ming-Yu Liu,et al.  Deep Active Learning for Civil Infrastructure Defect Detection and Classification , 2017 .

[69]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  Joachim Denzler,et al.  Labeling Examples That Matter: Relevance-Based Active Learning with Gaussian Processes , 2013, GCPR.

[71]  Xin Li,et al.  Multi-level Adaptive Active Learning for Scene Classification , 2014, ECCV.

[72]  Zoubin Ghahramani,et al.  Deep Bayesian Active Learning with Image Data , 2017, ICML.