The entropy of ordered sequences and order statistics

The entropy of a sequence of random variables under order restrictions is examined. A theorem that shows the amount of entropy reduction when the sequence is ordered is presented. Upper and lower bounds to the entropy reduction and conditions under which they are achieved are derived. Some interesting properties of the entropy of the individual order statistics are also presented. It is shown that the difference between the average entropy of the individual order statistics and the entropy of a member of the original independent identically distributed (IID) population is a constant, regardless of the original distribution. Finally, the entropies of the individual order statistics are found to be symmetric about the median when the probability density function (PDF) of the original IID sequence is symmetric about its mean. >

[1]  G. Arce,et al.  State description for the root-signal set of median filters , 1982 .

[2]  Yong Hoon Lee,et al.  Generalized median filtering and related nonlinear filtering techniques , 1985, IEEE Trans. Acoust. Speech Signal Process..

[3]  A. E. Sarhan.  Estimation of the Mean and Standard Deviation by Order Statistics. Part II , 1955 .

[4]  T. Nodes,et al.  The Output Distribution of Median Type Filters , 1984, IEEE Trans. Commun..

[5]  K. M. Wong,et al.  Some statistical properties of median filters , 1981 .

[6]  John W. Tukey,et al.  Nonlinear (nonsuperposable) methods for smoothing data , 1974 .

[7]  G. Wise,et al.  A theoretical analysis of the properties of median filters , 1981 .

[8]  F. Kuhlmann,et al.  On Second Moment Properties of Median Filtered Sequences of Independent Data , 1981, IEEE Trans. Commun..

[9]  E. H. Lloyd.  LEAST-SQUARES ESTIMATION OF LOCATION AND SCALE PARAMETERS USING ORDER STATISTICS , 1952 .

[10]  Shuang Chen,et al.  Detection of narrow-band sonar signals using order statistical filters , 1987, IEEE Trans. Acoust. Speech Signal Process..

[11]  Petros Maragos,et al.  A unification of linear, median, order-statistics and morphological filters under mathematical morphology , 1985, ICASSP '85. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[12]  Thomas S. Huang,et al.  A fast two-dimensional median filtering algorithm , 1979 .

[13]  Kon Max Wong,et al.  A fast method for real-time median filtering , 1980 .

[14]  A. E. Sarhan.  ESTIMATION OF THE MEAN AND STANDARD DEVIATION BY ORDER STATISTICS , 1954 .

[15]  J. Bednar,et al.  Alpha-trimmed means and their relationship to median filters , 1984 .

[16]  F. Downton.  Least-Squares Estimates Using Ordered Observations , 1954 .

[17]  Nuggehally Sampath Jayant.  Average- and Median-Based Smoothing Techniques for Improving Digital Speech Quality in the Presence of Transmission Errors , 1976, IEEE Trans. Commun..

[18]  Gonzalo R. Arce,et al.  BTC Image Coding Using Median Filter Roots , 1983, IEEE Trans. Commun..

[19]  Zakkula Govindarajulu.  Certain General Properties of Unbiased Estimates of Location and Scale Parameters Based on Ordered Observations , 1968 .

[20]  Thomas S. Huang,et al.  A generalization of median filtering using linear combinations of order statistics , 1983 .

[21]  Yong Hoon Lee,et al.  An edge gradient enhancing adaptive order statistic filter , 1986, IEEE Trans. Acoust. Speech Signal Process..

[22]  Lawrence R. Rabiner,et al.  Applications of a nonlinear smoothing algorithm to speech processing , 1975 .