Approximating Csiszár f-divergence by the use of Taylor's formula with integral remainder

Some approximations of the Csiszár f − divergence by the use of Taylor’s formula and perturbed Taylor’s formula and some applications for Kullback-Leibler distance are given. Mathematics subject classification (2000): 26D15.

[1]  A. Bhattacharyya.  On a measure of divergence between two statistical populations defined by their probability distributions , 1943 .

[2]  H. Jeffreys.  An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[3]  C. Kraft.  Some conditions for consistency and uniform consistency of statistical procedures , 1955 .

[4]  V. Volkonskii,et al.  Some Limit Theorems for Random Functions. II , 1959 .

[5]  A. Rényi.  On Measures of Entropy and Information , 1961 .

[6]  Amiel Feinstein,et al.  Information and information stability of random variables and processes , 1964 .

[7]  S. M. Ali,et al.  A General Class of Coefficients of Divergence of One Distribution from Another , 1966 .

[8]  H. McKean.  Speed of approach to equilibrium for Kac's caricature of a Maxwellian gas , 1966 .

[9]  S. Kullback,et al.  A lower bound for discrimination information in terms of variation (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[10]  W. E. Silver,et al.  Economics and Information Theory , 1967 .

[11]  T. T. Kadota,et al.  On the best finite set of linear observables for discriminating two Gaussian signals , 1967, IEEE Trans. Inf. Theory.

[12]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[13]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[14]  Solomon Kullback,et al.  Correction to A Lower Bound for Discrimination Information in Terms of Variation , 1970, IEEE Trans. Inf. Theory.

[15]  Igor Vajda,et al.  Note on discrimination information and variation (Corresp.) , 1970, IEEE Trans. Inf. Theory.

[16]  D. H. Freeman.  Statistical Decomposition Analysis , 1974 .

[17]  École d'été de probabilités de Saint-Flour,et al.  École d'été de probabilités de Saint-Flour. VII-1977 , 1978 .

[18]  Moshe Ben-Bassat,et al.  f-Entropies, probability of Error, and Feature Selection , 1978, Inf. Control..

[19]  M. Nei.  The theory of genetic distance and evolution of human races , 1978, Japanese Journal of Human Genetics.

[20]  J. A. Romero.  The information in contingency tables , 1979 .

[21]  Demetrios Kazakos,et al.  A Decision Theory Approach to the Approximation of Discrete Probability Densities , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Calyampudi R. Rao.  Diversity and dissimilarity coefficients: A unified approach☆ , 1982 .

[23]  C. R. Rao,et al.  On the convexity of some divergence measures based on entropy functions , 1982, IEEE Trans. Inf. Theory.

[24]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[25]  I. Vajda.  Theory of statistical inference and information , 1989 .

[26]  J. Lin,et al.  A NEW DIRECTED DIVERGENCE MEASURE AND ITS CHARACTERIZATION , 1990 .

[27]  Jianhua Lin,et al.  Divergence measures based on the Shannon entropy , 1991, IEEE Trans. Inf. Theory.

[28]  B. M. Fulk.  MATH , 1992 .

[29]  Hiroyuki Shioya,et al.  A generalization of lin divergence and the derivation of a new information divergence , 1995 .

[30]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[31]  C. Simkin..  About Economic Inequality , 1998 .

[32]  J. Pečarić,et al.  ON NEW ESTIMATION OF THE REMAINDER IN GENERALIZED TAYLOR'S FORMULA , 1999 .

[33]  NEW ESTIMATION OF THE REMAINDER IN TAYLOR'S FORMULA USING GR¨ USS' TYPE INEQUALITIES AND APPLICATIONS , 1999 .

[34]  An Improvement of the Remainder Estimate in the Generalised Taylor Formula , 2000 .

[35]  Flemming Topsøe,et al.  Some inequalities for information divergence and related measures of discrimination , 2000, IEEE Trans. Inf. Theory.

[36]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.