Neural Information Processing

Mixture modeling is one of the simplest ways to represent complicated probability density functions, and to integrate information from different sources. There are two typical mixtures in the context of information geometry, the mand e-mixtures. This paper proposes a novel framework of non-parametric e-mixture modeling by using a simple estimation algorithm based on geometrical insights into the characteristics of the e-mixture. An experimental result supports the proposed framework.

[1]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[2]  B. Liu,et al.  Error analysis of digital filters realized with floating-point arithmetic , 1969 .

[3]  J. Yackel,et al.  Consistency Properties of Nearest Neighbor Density Function Estimators , 1977 .

[4]  M. Rosenblatt,et al.  Multivariate k-nearest neighbor density estimates , 1979 .

[5]  M. Rudemo.  Empirical Choice of Histograms and Kernel Density Estimators , 1982 .

[6]  L. Györfi,et al.  Density-free convergence properties of various estimators of entropy , 1987 .

[7]  H. Joe.  Estimation of entropy and other functionals of a multivariate density , 1989 .

[8]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[9]  L. Györfi,et al.  Nonparametric entropy estimation. An overview , 1997 .

[10]  James T. Kwok,et al.  Objective functions for training new hidden units in constructive neural networks , 1997, IEEE Trans. Neural Networks.

[11]  Masashi Sugiyama,et al.  Optimal design of regularization term and regularization parameter by subspace information criterion , 2002, Neural Networks.

[12]  Liam Paninski,et al.  Estimation of Entropy and Mutual Information , 2003, Neural Computation.

[13]  Ignacio Rojas,et al.  Obtaining Fault Tolerant Multilayer Perceptrons Using an Explicit Regularization , 2000, Neural Processing Letters.

[14]  M. N. Goria,et al.  A new class of random vector entropy estimators and its applications in testing statistical hypotheses , 2005 .

[15]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[16]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[17]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[18]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[19]  S. Saigal,et al.  Relative performance of mutual information estimation methods for quantifying the dependence among short and noisy data. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[20]  Nicolas Pinto,et al.  How far can you get with a modern face recognition test set using only simple features? , 2009, CVPR.

[21]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Andrew Chi-Sing Leung,et al.  On the Selection of Weight Decay Parameter for Faulty Networks , 2010, IEEE Transactions on Neural Networks.

[23]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[24]  Andrew Chi-Sing Leung,et al.  RBF Networks Under the Concurrent Fault Situation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Zia Ul-Qayyum,et al.  Paraphrase Identification using Semantic Heuristic Features , 2012 .

[26]  Gang Hua,et al.  Probabilistic Elastic Matching for Pose Variant Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Josef Kittler,et al.  Efficient processing of MRFs for unconstrained-pose face recognition , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[28]  Andrew Zisserman,et al.  Fisher Vector Faces in the Wild , 2013, BMVC.

[29]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[30]  Josef Kittler,et al.  Class-Specific Kernel Fusion of Multiple Descriptors for Face Verification Using Multiscale Binarised Statistical Image Features , 2014, IEEE Transactions on Information Forensics and Security.

[31]  Dimitri Kartsaklis,et al.  Evaluating Neural Word Representations in Tensor-Based Compositional Settings , 2014, EMNLP.

[32]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[33]  Christopher Meek,et al.  Semantic Parsing for Single-Relation Question Answering , 2014, ACL.

[34]  David Zhang,et al.  Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification , 2014, International Journal of Computer Vision.

[35]  Cícero Nogueira dos Santos,et al.  Learning Character-level Representations for Part-of-Speech Tagging , 2014, ICML.

[36]  Marios Savvides,et al.  Spartans: Single-Sample Periocular-Based Alignment-Robust Recognition Technique Applied to Non-Frontal Scenarios , 2015, IEEE Transactions on Image Processing.

[37]  Gang Hua,et al.  Hierarchical-PEP model for real-world face recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Kensuke Koshijima,et al.  Non-parametric entropy estimators based on simple linear regression , 2015, Comput. Stat. Data Anal..

[39]  Alexander M. Rush,et al.  Character-Aware Neural Language Models , 2015, AAAI.