Learning Placeholders for Open-Set Recognition

Traditional classifiers are deployed under closed-set setting, with both training and test classes belong to the same set. However, real-world applications probably face the input of unknown categories, and the model will recognize them as known ones. Under such circumstances, open-set recognition is proposed to maintain classification performance on known classes and reject unknowns. The closed-set models make overconfident predictions over familiar known class instances, so that calibration and thresholding across categories become essential issues when extending to an open-set environment. To this end, we proposed to learn PlaceholdeRs for Open-SEt Recognition (Proser), which prepares for the unknown classes by allocating placeholders for both data and classifier. In detail, learning data placeholders tries to anticipate open-set class data, thus transforms closed-set training into open-set training. Besides, to learn the invariant information between target and non-target classes, we reserve classifier placeholders as the class-specific boundary between known and unknown. The proposed Proser efficiently generates novel class by manifold mixup, and adaptively sets the value of reserved open-set classifier during training. Experiments on various datasets validate the effectiveness of our proposed method.

[1]  Ioannis Mitliagkas,et al.  Manifold Mixup: Better Representations by Interpolating Hidden States , 2018, ICML.

[2]  Zhi-Hua Zhou,et al.  Learnware: on the future of machine learning , 2016, Frontiers of Computer Science.

[3]  Yang Yu,et al.  Open Category Classification by Adversarial Sample Generation , 2017, IJCAI.

[4]  De-Chuan Zhan,et al.  Detecting Sequentially Novel Classes with Stable Generalization Ability , 2021, PAKDD.

[5]  De-Chuan Zhan,et al.  Adaptive Deep Models for Incremental Learning: Considering Capacity Scalability and Sustainability , 2019, KDD.

[6]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[7]  Vishal M. Patel,et al.  C2AE: Class Conditioned Auto-Encoder for Open-Set Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  R. Srikant,et al.  Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks , 2017, ICLR.

[9]  Weng-Keen Wong,et al.  Open Set Learning with Counterfactual Images , 2018, ECCV.

[10]  Takeshi Naemura,et al.  Classification-Reconstruction Learning for Open-Set Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Rahil Garnavi,et al.  Generative OpenMax for Multi-Class Open Set Classification , 2017, BMVC.

[12]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[13]  John McCarthy,et al.  SOME PHILOSOPHICAL PROBLEMS FROM THE STANDPOINT OF ARTI CIAL INTELLIGENCE , 1987 .

[14]  Zhi-Hua Zhou,et al.  Multi-Instance Learning With Emerging Novel Class , 2021, IEEE Transactions on Knowledge and Data Engineering.

[15]  De-Chuan Zhan,et al.  Learning Adaptive Classifiers Synthesis for Generalized Few-Shot Learning , 2019, International Journal of Computer Vision.

[16]  Gabriela Csurka,et al.  Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[18]  Matthias Hein,et al.  Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Vishal M. Patel,et al.  Generative-Discriminative Feature Representations for Open-Set Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Zhi-Hua Zhou,et al.  Classification Under Streaming Emerging New Classes: A Solution Using Completely-Random Trees , 2016, IEEE Transactions on Knowledge and Data Engineering.

[21]  Terrance E. Boult,et al.  Towards Open World Recognition , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Zhi-Hua Zhou,et al.  Multi-Label Learning with Emerging New Labels , 2018, IEEE Transactions on Knowledge and Data Engineering.

[23]  S. Nadarajah,et al.  Extreme Value Distributions: Theory and Applications , 2000 .

[24]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[25]  Lei Shu,et al.  DOC: Deep Open Classification of Text Documents , 2017, EMNLP.

[26]  Zhenguo Li,et al.  DecAug: Out-of-Distribution Generalization via Decomposed Feature Representation and Semantic Augmentation , 2020, AAAI.

[27]  Yinda Zhang,et al.  LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop , 2015, ArXiv.

[28]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[29]  Songcan Chen,et al.  Recent Advances in Open Set Recognition: A Survey , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Kilian Q. Weinberger,et al.  On Calibration of Modern Neural Networks , 2017, ICML.

[31]  Anderson Rocha,et al.  Toward Open Set Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Hongyi Zhang,et al.  mixup: Beyond Empirical Risk Minimization , 2017, ICLR.

[33]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[34]  Tatsuya Harada,et al.  Between-Class Learning for Image Classification , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Ya Le,et al.  Tiny ImageNet Visual Recognition Challenge , 2015 .

[36]  Harry Wechsler,et al.  Open set face recognition using transduction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Zhi-Hua Zhou,et al.  Rectify Heterogeneous Models with Semantic Mapping , 2018, ICML.

[38]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[39]  Terrance E. Boult,et al.  Towards Open Set Deep Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Vishal M. Patel,et al.  Sparse Representation-Based Open Set Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Kevin Gimpel,et al.  A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks , 2016, ICLR.