Category coding with neural network application

In many applications of neural network, it is common to introduce huge amounts of input categorical features, as well as output labels. However, since the required network size should have rapid growth with respect to the dimensions of input and output space, there exists huge cost in both computation and memory resources. In this paper, we present a novel method called category coding (CC), where the design philosophy follows the principle of minimal collision to reduce the input and output dimension effectively. In addition, we introduce three types of category coding based on different Euclidean domains. Experimental results show that all three proposed methods outperform the existing state-of-the-art coding methods, such as standard cut-off and error-correct output coding (ECOC) methods.

[1]  Sergio Escalera,et al.  ECOC-ONE: A Novel Coding and Decoding Strategy , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[2]  Eleonora Guerrini A classification of MDS binary systematic codes , 2006 .

[3]  Paolo Frasconi,et al.  New results on error correcting output codes of kernel machines , 2004, IEEE Transactions on Neural Networks.

[4]  John Langford,et al.  Sensitive Error Correcting Output Codes , 2005, COLT.

[5]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[6]  Sergio Escalera,et al.  An incremental node embedding technique for error correcting output codes , 2008, Pattern Recognit..

[7]  Rayid Ghani,et al.  Using Error-Correcting Codes for Text Classification , 2000, ICML.

[8]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[9]  N. Jacobson,et al.  Lectures in Abstract Algebra : vol. III, Theory of Fields and Galois Theory. By N. Jacobson. Pp. xi, 323. 76s. (Van Nostrand) , 1966, The Mathematical Gazette.

[10]  F. Moore,et al.  Polynomial Codes Over Certain Finite Fields , 2017 .

[11]  I. Reed,et al.  Polynomial Codes Over Certain Finite Fields , 1960 .

[12]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[13]  B. Riemann,et al.  Ueber die Anzahl der Primzahlen unter einer gegebenen Grösse , 2013 .

[14]  Lekh R. Vermani Elements of Algebraic Coding Theory , 1996 .

[15]  Jordi Vitrià,et al.  Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[17]  S. Lang Algebraic Number Theory , 1971 .

[18]  Richard C. Singleton,et al.  Maximum distance q -nary codes , 1964, IEEE Trans. Inf. Theory.

[19]  Kenneth W. Shum,et al.  Deep Representation Learning with Target Coding , 2015, AAAI.

[20]  Shu Lin,et al.  Error Control Coding , 2004 .

[21]  Cheng-Lin Liu,et al.  Error-correcting output codes based ensemble feature extraction , 2013, Pattern Recognit..

[22]  Springer-Verlag London Limited Joint learning of error-correcting output codes and dichotomizers from data , 2012 .

[23]  Adam L. Berger,et al.  ERROR-CORRECTING OUTPUT CODING FOR TEXT CLASSIFICATION , 1999 .

[24]  Mohamed Cheriet,et al.  Adaptive Error-Correcting Output Codes , 2013, IJCAI.

[25]  Sergio Escalera,et al.  Recoding Error-Correcting Output Codes , 2009, MCS.