Hybrid incremental learning of new data and new classes for hand-held object recognition

Abstract Intelligence technology is an important research area. As a very special yet important case of object recognition, hand-held object recognition plays an important role in intelligence technology for its many applications such as visual question-answering and reasoning. In real-world scenarios, the datasets are open-ended and dynamic: new object samples and new object classes increase continuously. This requires the intelligence technology to enable hybrid incremental learning, which supports both data-incremental and class-incremental learning to efficiently learn the new information. However, existing work mainly focuses on one side of incremental learning, either data-incremental or class-incremental learning while do not handle two sides of incremental learning in a unified framework. To solve the problem, we present a Hybrid Incremental Learning (HIL) method based on Support Vector Machine (SVM), which can incrementally improve its recognition ability by learning new object samples and new object concepts during the interaction with humans. In order to integrate data-incremental and class-incremental learning into one unified framework, HIL adds the new classification-planes and adjusts existing classification-planes under the setting of SVM. As a result, our system can simultaneously improve the recognition quality of known concepts by minimizing the prediction error and transfer the previous model to recognize unknown objects. We apply the proposed method into hand-held object recognition and the experimental results demonstrated its advantage of HIL. In addition, we conducted extensive experiments on the subset of ImageNet and the experimental results further validated the effectiveness of the proposed method.

[1]  Motoaki Kawanabe,et al.  On-line learning in changing environments with applications in supervised and unsupervised learning , 2002, Neural Networks.

[2]  Gabriela Csurka,et al.  Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  C. Brodley,et al.  Decision tree classification of land cover from remotely sensed data , 1997 .

[4]  Liqiang Nie,et al.  Predicting Image Memorability Through Adaptive Transfer Learning From External Sources , 2017, IEEE Transactions on Multimedia.

[5]  Xuelong Li,et al.  Incremental learning of weighted tensor subspace for visual tracking , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[6]  Yuting Su,et al.  Human Action Recognition Based on Selected Spatio-Temporal Features via Bidirectional LSTM , 2018, IEEE Access.

[7]  R. Brits,et al.  A clustering approach to incremental learning for feedforward neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[8]  Charu C. Aggarwal,et al.  Towards cross-category knowledge propagation for learning visual concepts , 2011, CVPR 2011.

[9]  Xiaofeng Ren,et al.  Figure-ground segmentation improves handled object recognition in egocentric video , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Jialie Shen,et al.  On Effective Location-Aware Music Recommendation , 2016, ACM Trans. Inf. Syst..

[11]  Yuxin Peng,et al.  Error-Driven Incremental Learning in Deep Convolutional Neural Network for Large-Scale Image Classification , 2014, ACM Multimedia.

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[14]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Vincent Lepetit,et al.  Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes , 2011, 2011 International Conference on Computer Vision.

[16]  Gang Wang,et al.  Large-Margin Multi-Modal Deep Learning for RGB-D Object Recognition , 2015, IEEE Transactions on Multimedia.

[17]  Davide Maltoni,et al.  Comparing Incremental Learning Strategies for Convolutional Neural Networks , 2016, ANNPR.

[18]  Charu C. Aggarwal,et al.  Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Jason Weston,et al.  Fast Kernel Classifiers with Online and Active Learning , 2005, J. Mach. Learn. Res..

[20]  Meng Wang,et al.  Low-Rank Multi-View Embedding Learning for Micro-Video Popularity Prediction , 2018, IEEE Transactions on Knowledge and Data Engineering.

[21]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[22]  Shuang Wang,et al.  Hand-Object Sense: A Hand-held Object Recognition System Based on RGB-D Information , 2015, ACM Multimedia.

[23]  J. Pine,et al.  Chunking mechanisms in human learning , 2001, Trends in Cognitive Sciences.

[24]  Mahardhika Pratama,et al.  An Incremental Learning of Concept Drifts Using Evolving Type-2 Recurrent Fuzzy Neural Networks , 2017, IEEE Transactions on Fuzzy Systems.

[25]  Heng Ji,et al.  Exploring Context and Content Links in Social Media: A Latent Space Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  R. Tyrrell Rockafellar,et al.  Lagrange Multipliers and Optimality , 1993, SIAM Rev..

[27]  Stefan Rüping,et al.  Incremental Learning with Support Vector Machines , 2001, ICDM.

[28]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[29]  Philip M. Long,et al.  On-line learning of linear functions , 1991, STOC '91.

[30]  Rosane Minghim,et al.  An Approach to Supporting Incremental Visual Data Classification , 2015, IEEE Transactions on Visualization and Computer Graphics.

[31]  Minh Tue Vo,et al.  Incremental learning using the time delay neural network , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  A. Kuh,et al.  A smart algorithm for incremental learning , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[33]  Mohan S. Kankanhalli,et al.  A^3NCF: An Adaptive Aspect Attention Model for Rating Prediction , 2018, IJCAI.

[34]  Shuang Wang,et al.  RGB-D Hand-Held Object Recognition Based on Heterogeneous Feature Fusion , 2015, Journal of Computer Science and Technology.

[35]  Xian-Sheng Hua,et al.  Two-Dimensional Multilabel Active Learning with an Efficient Online Adaptation Model for Image Classification , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[37]  Jinhui Tang,et al.  RGB-D Object Recognition via Incorporating Latent Data Structure and Prior Knowledge , 2015, IEEE Transactions on Multimedia.

[38]  Michael Beetz,et al.  Leaving Flatland: Toward real-time 3D navigation , 2009, 2009 IEEE International Conference on Robotics and Automation.

[39]  Ilja Kuzborskij,et al.  From N to N+1: Multiclass Transfer Incremental Learning , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[41]  Matthieu Guillaumin,et al.  Incremental Learning of Random Forests for Large-Scale Image Classification , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  François Poulet,et al.  Large Scale Image Classification: Fast Feature Extraction, Multi-codebook Approach and Multi-core SVM Training , 2012, EGC.

[43]  Christoph H. Lampert,et al.  Learning to detect unseen object classes by between-class attribute transfer , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  M. Madheswaran,et al.  Hybrid Medical Image Classification Using Association Rule Mining with Decision Tree Algorithm , 2010, ArXiv.

[45]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[46]  Byoung-Tak Zhang,et al.  An incremental learning algorithm that optimizes network size and sample size in one trial , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[47]  Vasant Honavar,et al.  Learn++: an incremental learning algorithm for supervised neural networks , 2001, IEEE Trans. Syst. Man Cybern. Part C.