Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data

The recent development of more sophisticated remote-sensing systems enables the measurement of radiation in many more spectral intervals than was previously possible. An example of this technology is the AVIRIS system, which collects image data in 220 bands. The increased dimensionality of such hyperspectral data greatly enhances the data's information content, but provides a challenge to the current techniques for analyzing such data. Human experience in 3D space tends to mislead our intuition of geometrical and statistical properties in high-dimensional space, properties which must guide our choices in the data analysis process. Using Euclidean and Cartesian geometry, high-dimensional space properties are investigated in this paper, and their implication for high-dimensional data and its analysis is studied in order to illuminate the differences between conventional spaces and hyperdimensional space.

[1]  David A. Landgrebe,et al.  Projection pursuit for high dimensional feature reduction: parallel and sequential approaches , 1995, 1995 International Geoscience and Remote Sensing Symposium, IGARSS '95. Quantitative Remote Sensing for Science and Applications.

[2]  David A. Landgrebe,et al.  Hierarchical Classification In High Dimensional, Numerous Class Cases , 1990, 10th Annual International Symposium on Geoscience and Remote Sensing.

[3]  David A. Landgrebe,et al.  Feature extraction and classification algorithms for high-dimensional data , 1992 .

[4]  D. Freedman,et al.  Asymptotics of Graphical Projection Pursuit , 1984 .

[5]  R. Kettig,et al.  Classification of Multispectral Image Data by Extraction and Classification of Homogeneous Objects , 1976, IEEE Transactions on Geoscience Electronics.

[6]  Anil K. Jain,et al.  On the optimal number of features in the classification of multivariate Gaussian data , 1978, Pattern Recognit..

[7]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[8]  J. Simonoff.  Multivariate Density Estimation , 1996 .

[9]  David A. Landgrebe,et al.  Projection pursuit in high dimensional data reduction: initial conditions, feature selection and the assumption of normality , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[10]  David A. Landgrebe,et al.  Analyzing high-dimensional multispectral data , 1993, IEEE Trans. Geosci. Remote. Sens..

[11]  E. Wegman.  Hyperdimensional Data Analysis Using Parallel Coordinates , 1990 .

[12]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[13]  G. F. Hughes,et al.  On the mean accuracy of statistical pattern recognizers , 1968, IEEE Trans. Inf. Theory.

[14]  Edward J. Wegman,et al.  Statistical Signal Processing , 1985 .

[15]  Jan M. Van Campenhout,et al.  On the Possible Orderings in the Measurement Selection Problem , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Ker-Chau Li,et al.  On almost Linearity of Low Dimensional Projections from High Dimensional Data , 1993 .

[17]  John A. Richards,et al.  Remote Sensing Digital Image Analysis , 1986 .

[18]  R. Kettig.  Computer classification of remotely sensed multispectral image data by extraction and classification of homogeneous objects. , 1975 .

[19]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[20]  Maurice G. Kendall,et al.  A Course in the Geometry of n Dimensions , 1962 .

[21]  Ronald L. Rivest,et al.  Constructing Optimal Binary Decision Trees is NP-Complete , 1976, Inf. Process. Lett..

[22]  David A. Landgrebe,et al.  High dimensional feature reduction via projection pursuit , 1994, Proceedings of IGARSS '94 - 1994 IEEE International Geoscience and Remote Sensing Symposium.

[23]  Jenq-Neng Hwang,et al.  Nonparametric multivariate density estimation: a comparative study , 1994, IEEE Trans. Signal Process..

[24]  Maurice G. Kendall,et al.  A Course in the Geometry of n Dimensions , 1962 .

[25]  David A. Landgrebe,et al.  The development of a spectral-spatial classifier for earth observational data , 1980, Pattern Recognit..

[26]  David A. Landgrebe,et al.  Feature Extraction Based on Decision Boundaries , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Philip H. Swain,et al.  Remote Sensing: The Quantitative Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  P. Hall.  Estimating the direction in which a data set is most interesting , 1988 .

[29]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[30]  L. Scharf,et al.  Statistical Signal Processing: Detection, Estimation, and Time Series Analysis , 1991 .

[31]  J. Friedman,et al.  PROJECTION PURSUIT DENSITY ESTIMATION , 1984 .

[32]  Keinosuke Fukunaga,et al.  Effects of Sample Size in Classifier Design , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .

[34]  John A. Richards,et al.  Remote Sensing Digital Image Analysis: An Introduction , 1999 .