A class of lower bounds for the generalized mean-square error (MSE), the probability of error, and the accuracy of domain decisions is derived. The derivation is based on the well-known properties of the functional \begin{equation} \Lambda \equiv \int _ {\mathcal{H}} - [\ln p_2 (x) - \ln p_1 (x)]p_1(x)d \gamma. \end{equation} The bounds depend on the mutual information between the channel's input and output signals and on the loss of information due to making the decision. It is shown that further bounding of mutual information gives as special cases bounds for the generalized MSE. These special cases almost coincide with a special case of the we!l-known Shannon bound, which is based on information rate relative to a fidelity criterion, and with a special case of the Cramer-Rao bound. The derived bound for the probability of error and the accuracy is valid both in the continuous case, in particular for interval decisions and in the discrete case for list decisions. In another special case the bound turns out to be the well-known Fano bound. The problem of the uniqueness of the derived bounds is also considered and it is shown that in a certain sense they are unique bounds based only on mutual information. As by-products a new extremal property of the entropy and a uniqueness property of functional \Lambda are proved.
[1]
C. E. SHANNON,et al.
A mathematical theory of communication
,
1948,
MOCO.
[2]
R. Hartley.
Transmission of information
,
1928
.
[3]
David S. Slepian,et al.
Estimation of signal parameters in the presence of noise
,
1954,
Trans. IRE Prof. Group Inf. Theory.
[4]
Claude E. Shannon,et al.
The Mathematical Theory of Communication
,
1950
.
[5]
Peter Swerling,et al.
Maximum Angular Accuracy of a Pulsed Search Radar
,
1956,
Proceedings of the IRE.
[6]
Solomon Kullback,et al.
Information Theory and Statistics
,
1960
.
[7]
C. Shannon.
Probability of error for optimal codes in a Gaussian channel
,
1959
.
[8]
Thomas J. Goblick,et al.
Theoretical limitations on the transmission of data from analog sources
,
1965,
IEEE Trans. Inf. Theory.
[9]
R. A. Leibler,et al.
On Information and Sufficiency
,
1951
.
[10]
Jacob Ziv,et al.
Some lower bounds on signal parameter estimation
,
1969,
IEEE Trans. Inf. Theory.