Multilabel Classification Using Error-Correcting Codes of Hard or Soft Bits

We formulate a framework for applying error-correcting codes (ECCs) on multilabel classification problems. The framework treats some base learners as noisy channels and uses ECC to correct the prediction errors made by the learners. The framework immediately leads to a novel ECC-based explanation of the popular random k-label sets (RAKEL) algorithm using a simple repetition ECC. With the framework, we empirically compare a broad spectrum of off-the-shelf ECC designs for multilabel classification. The results not only demonstrate that RAKEL can be improved by applying some stronger ECC, but also show that the traditional binary relevance approach can be enhanced by learning more parity-checking labels. Our research on different ECCs also helps to understand the tradeoff between the strength of ECC and the hardness of the base learning tasks. Furthermore, we extend our research to ECC with either hard (binary) or soft (real-valued) bits by designing a novel decoder. We demonstrate that the decoder improves the performance of our framework.

[1]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[2]  Ling Li,et al.  Multiclass boosting with repartitioning , 2006, ICML.

[3]  Grigorios Tsoumakas,et al.  Random k -Labelsets: An Ensemble Method for Multilabel Classification , 2007, ECML.

[4]  A. Kouzani,et al.  Multilabel Classification by BCH Code and Random Forests , 2009 .

[5]  John Langford,et al.  Multi-Label Prediction via Compressed Sensing , 2009, NIPS.

[6]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[7]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[8]  Eyke Hüllermeier,et al.  On label dependence in multilabel classification , 2010, ICML 2010.

[9]  Hsuan-Tien Lin,et al.  Feature-aware Label Space Dimension Reduction for Multi-label Classification , 2012, NIPS.

[10]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[11]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[12]  Grigorios Tsoumakas,et al.  MULAN: A Java Library for Multi-Label Learning , 2011, J. Mach. Learn. Res..

[13]  Feiping Nie,et al.  Discriminative Least Squares Regression for Multiclass Classification and Feature Selection , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Richard W. Hamming,et al.  Error detecting and error correcting codes , 1950 .

[15]  Hsuan-Tien Lin,et al.  Multilabel Classification with Principal Label Space Transformation , 2012, Neural Computation.

[16]  Robert E. Schapire,et al.  Using output codes to boost multiclass learning problems , 1997, ICML.

[17]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[18]  Dwijendra K. Ray-Chaudhuri,et al.  Binary mixture flow with free energy lattice Boltzmann methods , 2022, arXiv.org.

[19]  Yuhong Yang,et al.  Information Theory, Inference, and Learning Algorithms , 2005 .

[20]  Abbas Z. Kouzani,et al.  Multilabel Classification Using Error Correction Codes , 2010, ISICA.

[21]  K. Dembczynski,et al.  On Label Dependence in Multi-Label Classification , 2010 .

[22]  Robert H. Morelos-Zaragoza,et al.  The Art of Error Correcting Coding: Morelos-Zaragoza/The Art of Error Correcting Coding, Second Edition , 2006 .

[23]  Jeff G. Schneider,et al.  Multi-Label Output Codes using Canonical Correlation Analysis , 2011, AISTATS.

[24]  Jeff G. Schneider,et al.  Maximum Margin Output Coding , 2012, ICML.

[25]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[26]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[27]  Hsuan-Tien Lin,et al.  Multi-label Classification with Error-correcting Codes , 2011, ACML.

[28]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[29]  R. Morelos-Zaragoza The art of error correcting coding , 2002 .

[30]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..