Renyi's entropy and the probability of error

The basic properties of Renyi's entropy are reviewed, and its concavity properties are characterized. New bounds (referred to as I_{\alpha} bounds) on the probability of error are derived from Renyi's entropy and are compared with known bounds. It is proved that for the two-class case, the I_{2} bound is sharper than many of the previously known bounds. The difference between the I_{2} bound and the real value of the probability of error is at most 0.09.

[1]  Godfried Theodore Patrick Toussaint,et al.  Feature evaluation criteria and contextual decoding algorithms in statistical pattern recognition. , 1972 .

[2]  L. L. Campbell,et al.  A Coding Theorem and Rényi's Entropy , 1965, Inf. Control..

[3]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[4]  Godfried T. Toussaint,et al.  Some Upper Bounds on Error Probability for Multiclass Pattern Recognition , 1971, IEEE Transactions on Computers.

[5]  A. Rényi.  On Measures of Entropy and Information , 1961 .

[6]  Demetrios G. Lainiotis,et al.  A class of upper bounds on probability of error for multihypotheses pattern recognition (Corresp.) , 1969, IEEE Trans. Inf. Theory.

[7]  Laveen N. Kanal,et al.  Patterns in pattern recognition: 1968-1974 , 1974, IEEE Trans. Inf. Theory.

[8]  O. Mangasarian.  PSEUDO-CONVEX FUNCTIONS , 1965 .

[9]  A. Kolmogorov.  ON THE APPROXIMATION OF DISTRIBUTIONS OF SUMS OF INDEPENDENT SUMMANDS BY INFINITELY DIVISIBLE DISTRIBUTIONS , 1965 .

[10]  D. E. Daykin,et al.  Generalization of Hölder's and Minkowski's inequalities , 1968 .

[11]  Pierre A. Devijver,et al.  On a New Class of Bounds on Bayes Risk in Multihypothesis Pattern Recognition , 1974, IEEE Transactions on Computers.

[12]  King-Sun Fu,et al.  Error estimation in pattern recognition via LAlpha -distance between posterior density functions , 1976, IEEE Trans. Inf. Theory.

[13]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[14]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[15]  Martin E. Hellman,et al.  Probability of error, equivocation, and the Chernoff bound , 1970, IEEE Trans. Inf. Theory.

[16]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.