A New Equilibrium Criterion for Learning the Cardinality of Latent Variables

Mining high dimensional data-sets extracted from real world problems is a challenging task due to the large features' space. The latent variables are used to reduce the dimensions of this space by representing highly dependent features. They simplify the creation of probabilistic models and they clarify the semantic of the inferred knowledge. Learning these variables for Bayesian network, as the most generic probabilistic model, is problematic. Actually, there is not a direct way that leads to finding their cardinalities. The precision of the inferred model is highly dependent on the accuracy of the latent variable's cardinality. Therefore, choosing a small value leads to a generalized model having a high rate of information loss. Moreover, a high cardinality tend to over-fit the data, to generate complex latent variables and to burden the parameter learning of the probabilistic model. In this paper, we propose a new criterion based on the mutual information and the log likelihood, called the equilibrium criterion. We mathematically and experimentally validate its efficiency for estimating the cardinality of the latent variable. We also demonstrate its performance in finding the hidden cause of a set of observed variables. The experimental analysis shows that our method succeeded in restoring the original cardinality of intentionally deleted variables in known networks.