Information Dependency and Its Applications

Independence is a basic concept of probability theory and statistics. In a lot of elds of sciences, dependency of di erent variables is gained lots of attention from scientists. A measure, named information dependency, is proposed to express the dependency of a group of random variables. This measure is de ned as the Kullback-Leibler divergence of a joint distribution with respect to a product-marginal distribution of these random variables. In the bivariate case, this measure is known as mutual information of two random variables. Thus, the measure information dependency has a strong relationship with the Information Theory. The thesis aims to give a thorough study of the information dependency from both mathematical and practical viewpoints. Concretely, we would like to research three following problems: i. Proving that the information dependency is a useful tool to express the dependency of a group of random variables by comparing it with other measures of dependency. ii. Studying the methods to estimate the information dependency based on the samples of a group of random variables. iii. Investigating how the Independent Component Analysis problem, an interesting problem in statistics, can be solved using information dependency.