A New Framework for Multiclass Classification Using Multiview Assisted Adaptive Boosting

Multiview representation of data is common in disciplines such as computer vision, bio-informatics, etc. Traditional fusion methods train independent classifiers on each view and finally conglomerate them using weighted summation. Such approaches are void from inter-view communications and thus do not guarantee to yield the best possible ensemble classifier on the given sample-view space. This paper proposes a new algorithm for multiclass classification using multi-view assisted supervised learning (MA-AdaBoost). MA-AdaBoost uses adaptive boosting for initially training baseline classifiers on each view. After each boosting round, the classifiers share their classification performances. Based on this communication, weight of an example is ascertained by its classification difficulties across all views. Two versions of MA-AdaBoost are proposed based on the nature of final output of baseline classifiers. Finally, decisions of baseline classifiers are agglomerated based on a novel algorithm of reward assignment. The paper then presents classification comparisons on benchmark UCI datasets and eye samples collected from FERET database. Kappa-error diversity diagrams are also studied. In majority instances, MA-AdaBoost outperforms traditional AdaBoost, variants of AdaBoost, and recent works on supervised collaborative learning with respect to convergence rate of training set and generalization errors. The error-diversity results are also encouraging.

[1]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Friedhelm Schwenker,et al.  Co-training by Committee: A New Semi-supervised Learning Framework , 2008, 2008 IEEE International Conference on Data Mining Workshops.

[3]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[4]  Frédéric Béchet,et al.  Applying multiview learning algorithms to human-human conversation classification , 2012, INTERSPEECH.

[5]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[6]  Sokol Koço,et al.  A Boosting Approach to Multiview Classification with Cooperation , 2011, ECML/PKDD.

[7]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Chun-Xia Zhang,et al.  An efficient modified boosting method for solving classification problems , 2008 .

[9]  Mikhail Belkin,et al.  Semi-Supervised Learning , 2021, Machine Learning.

[10]  Zhi-Hua Zhou,et al.  On multi-view active learning and the combination with semi-supervised learning , 2008, ICML '08.

[11]  Jing Huang,et al.  Multi-View and Multi-Objective Semi-Supervised Learning for HMM-Based Automatic Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[12]  Sushmita Mitra,et al.  Natural computing methods in bioinformatics: A survey , 2009, Inf. Fusion.

[13]  Terry Windeatt,et al.  Relevant and Redundant Feature Analysis with Ensemble Classification , 2009, 2009 Seventh International Conference on Advances in Pattern Recognition.

[14]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[15]  Weifeng Liu,et al.  Multiview Hessian Regularization for Image Annotation , 2013, IEEE Transactions on Image Processing.

[16]  James Orwell,et al.  Probabilistic Classification from a K-Nearest-Neighbour Classifier , 2013 .

[17]  Peter A. Flach,et al.  Evaluation Measures for Multi-class Subgroup Discovery , 2009, ECML/PKDD.

[18]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[19]  Charles D. Mallah,et al.  PLANT LEAF CLASSIFICATION USING PROBABILISTIC INTEGRATION OF SHAPE, TEXTURE AND MARGIN FEATURES , 2013 .

[20]  Zhi-Hua Zhou,et al.  Semi-supervised learning by disagreement , 2010, Knowledge and Information Systems.

[21]  Jin Liu,et al.  An Embedded Co-AdaBoost based construction of software document relation coupled resource spaces for cyber-physical society , 2014, Future Gener. Comput. Syst..

[22]  Tolga Tasdizen,et al.  Fast AdaBoost training using weighted novelty selection , 2011, The 2011 International Joint Conference on Neural Networks.

[23]  Steven C. H. Hoi,et al.  Multiview Semi-Supervised Learning with Consensus , 2012, IEEE Transactions on Knowledge and Data Engineering.

[24]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[25]  Erkki Oja,et al.  Affective Abstract Image Classification and Retrieval Using Multiple Kernel Learning , 2013, ICONIP.

[26]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[27]  G. Michailidis,et al.  On multi-view learning with additive models , 2009, 0906.1117.

[28]  Thomas G. Dietterich,et al.  Pruning Adaptive Boosting , 1997, ICML.

[29]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  Weijian Ni,et al.  Boosting over Groups and Its Application to Acronym-Expansion Extraction , 2008, ADMA.

[31]  Jin Liu,et al.  An Embedded Co-AdaBoost and Its Application in Classification of Software Document Relation , 2012, 2012 Eighth International Conference on Semantics, Knowledge and Grids.