A general class of lower bounds on the probability of error in multiple hypothesis testing

In this paper, a new class of lower bounds on the probability of error for m-ary hypothesis tests is proposed. Computation of the minimum probability of error which is attained by the maximum a-posteriori probability (MAP) criterion, is usually not tractable. The new class is derived using Holder¿s inequality. The bounds in this class are continuous and differentiable function of the conditional probability of error and they provide good prediction of the minimum probability of error in multiple hypothesis testing. It is shown that for binary hypothesis testing problem this bound asymptotically coincides with the optimum probability of error provided by the MAP criterion. This bound is compared with other existing lower bounds in several typical detection and classification problems in terms of tightness and computational complexity.

[1]  Sergio Verdú,et al.  Generalizing the Fano inequality , 1994, IEEE Trans. Inf. Theory.

[2]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[3]  Neri Merhav,et al.  Relations between entropy and error probability , 1994, IEEE Trans. Inf. Theory.

[4]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[5]  Yossef Steinberg,et al.  Extended Ziv-Zakai lower bound for vector parameter estimation , 1997, IEEE Trans. Inf. Theory.

[6]  C. H. Chen,et al.  On information and distance measures, error bounds, and feature selection , 1976, Information Sciences.

[7]  Jacob Ziv,et al.  Improved Lower Bounds on Signal Parameter Estimation , 1975, IEEE Trans. Inf. Theory.

[8]  Pierre A. Devijver,et al.  On a New Class of Bounds on Bayes Risk in Multihypothesis Pattern Recognition , 1974, IEEE Transactions on Computers.

[9]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[10]  Hadar I. Avi-Itzhak,et al.  Arbitrarily Tight Upper and Lower Bounds on the Bayesian Probability of Error , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Peter F. Swaszek.  A lower bound on the error probability for signals in white Gaussian noise , 1995, IEEE Trans. Inf. Theory.

[12]  H. V. Poor,et al.  Applications of Ali-Silvey Distance Measures in the Design of Generalized Quantizers for Binary Decision Systems , 1977, IEEE Trans. Commun..

[13]  Godfried Theodore Patrick Toussaint,et al.  Feature evaluation criteria and contextual decoding algorithms in statistical pattern recognition. , 1972 .

[14]  Godfried T. Toussaint,et al.  An upper bound on the probability of misclassification in terms of Matusita's measure of affinity , 1982 .

[15]  Claude E. Shannon,et al.  Certain Results in Coding Theory for Noisy Channels , 1957, Inf. Control..

[16]  Pramod K. Varshney,et al.  ALGORITHMS FOR SENSOR FUSION: Applications of Distance Measures and Probability of Error Bounds to Distributed. Detection Systems. Volume 2 , 1991 .

[17]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[18]  Dick E. Boekee,et al.  Some aspects of error bounds in feature selection , 1979, Pattern Recognit..

[19]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[20]  Martin E. Hellman,et al.  Probability of error, equivocation, and the Chernoff bound , 1970, IEEE Trans. Inf. Theory.

[21]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[22]  Alexander Vardy,et al.  On an Improvement over Rényi's Equivocation Bound , 2006, ArXiv.

[23]  M. Basseville.  Distance measures for signal processing and pattern recognition , 1989 .

[24]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[25]  Gérald E. Séguin.  A Lower Bound on the Error Probability for Signals in White Gaussian Noise , 1998, IEEE Trans. Inf. Theory.

[26]  Dick E. Boekee,et al.  Divergence Bounds on Key Equivocation and Error Probability in Cryptanalysis , 1985, CRYPTO.

[27]  H. Vincent Poor,et al.  A lower bound on the probability of error in multihypothesis testing , 1995, IEEE Trans. Inf. Theory.

[28]  A. G. Greenhill,et al.  Handbook of Mathematical Functions with Formulas, Graphs, , 1971 .