Supervised dimensionality reduction and contextual pattern recognition in medical image processing

The past few years have witnessed a significant increase in the number of supervised methods employed in diverse image processing tasks. Especially in medical image analysis the use of, for example, supervised shape and appearance modelling has increased considerably and has proven to be successful. This thesis focuses on applying supervised pattern recognition methods in medical image processing. We consider a local, pixel-based approach in which image segmentation, regression, and filtering tasks are solved using descriptors of the local image content (features) based on which decisions are made that provide a class label (in case of image segmentation) or a gray value (in case of filtering or regression) for every pixel. The basic probabilistic decision problem, underlying---implicitly or explicitly---all the methods presented in this thesis, can be stated in terms of a conditional probability optimization problem u = argmax_y P(y|x) in which x is a d-dimensional vector of measurements, i.e., a feature vector, describing the local image content and y is a quantity that takes values from a set Y. Typically, in a classification task, Y is a discrete set of labels and in case of regression, Y equals R. Based on the maximization in the previous equation, to every vector x (which is associated to a pixel in an image), a particular u from Y is associated. This approach is---because of its local nature---quite different from the shape and appearance methods mentioned in the beginning of this chapter which try to solve image processing tasks in a more global way. A recent comparative study [B. van Ginneken, M. B. Stegmann, and M. Loog. Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database, Medical Image Analysis, accepted, 2005] shows that in image segmentation, pixel-based approaches can compete with shape and appearance models, providing an interesting alternative to the latter. The principal methodological part of the thesis consists of three dimensionality reduction methods that can aid the extraction of relevant features to be used for performing image segmentation or regression. Furthermore, an iterative segmentation scheme is developed which draws from classical pattern recognition and machine learning methods. Finally, two applications of these techniques in two problems related to computer-aided diagnosis (CAD) in chest radiography are presented. Firstly, the task of segmenting the posterior ribs is considered. Secondly, a regression framework is presented, which aims at suppressing bony structures in chest radiographs.

[1]  Gerhard Winkler,et al.  Image analysis, random fields and dynamic Monte Carlo methods: a mathematical introduction , 1995, Applications of mathematics.

[2]  Thomas G. Dietterich Machine Learning for Sequential Data: A Review , 2002, SSPR/SPR.

[3]  C. R. Rao,et al.  The Utilization of Multiple Measurements in Problems of Biological Classification , 1948 .

[4]  G. Sapiro,et al.  Geometric partial differential equations and image analysis [Book Reviews] , 2001, IEEE Transactions on Medical Imaging.

[5]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[6]  Harry Wechsler,et al.  Image processing algorithms applied to rib boundary detection in chest radiographs , 1978 .

[7]  C. T. Ng,et al.  Measures of distance between probability distributions , 1989 .

[8]  Azriel Rosenfeld,et al.  Scene Labeling by Relaxation Operations , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Max A. Viergever,et al.  Gaussian Scale Space from Insufficient Image Information , 2003, Scale-Space.

[11]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[12]  H Yoshida,et al.  Contralateral subtraction: a novel technique for detection of asymmetric abnormalities on digital chest radiographs. , 2000, Medical physics.

[13]  K Doi,et al.  Localization of inter-rib spaces for lung texture analysis and computer-aided diagnosis in digital chest images. , 1988, Medical physics.

[14]  Robert M. Gray,et al.  Multiresolution image classification by hierarchical modeling with two-dimensional hidden Markov models , 2000, IEEE Trans. Inf. Theory.

[15]  Steven W. Zucker,et al.  On the Foundations of Relaxation Labeling Processes , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  M. Loog Approximate Pairwise Accuracy Criteria for Multiclass Linear Dimension Reduction: Generalisations of the Fisher Criterion , 1999 .

[17]  Bram van Ginneken,et al.  A computer-aided diagnosis system for detection of lung nodules in chest radiographs with an evaluation on a public database , 2006, Medical Image Anal..

[18]  Robert A. Schowengerdt,et al.  Remote sensing, models, and methods for image processing , 1997 .

[19]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[20]  Peter de Souza Automatic rib detection in chest radiographs , 1983, Comput. Vis. Graph. Image Process..

[21]  Subhasis Chaudhuri,et al.  Detection of Rib Shadows in Digital Chest Radiographs , 1997, ICIAP.

[22]  Shingo Tomita,et al.  An extended fisher criterion for feature extraction ‐ Malina's method and its problems , 1984 .

[23]  N. A. Campbell,et al.  ALLOCATION OF REMOTELY SENSED DATA USING MARKOV MODELS FOR IMAGE DATA AND PIXEL LABELS , 1992 .

[24]  J. Austin,et al.  Missed non-small cell lung cancer: radiographic findings of potentially resectable lesions evident only in retrospect. , 2003, Radiology.

[25]  B. M. ter Haar Romeny,et al.  Automatic segmentation of lung fields in chest radiographs. , 2000, Medical physics.

[26]  Sunil Arya,et al.  ANN: library for approximate nearest neighbor searching , 1998 .

[27]  Bram van Ginneken,et al.  Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database , 2006, Medical Image Anal..

[28]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Josef Kittler,et al.  Moderating k-NN Classifiers , 2002, Pattern Analysis & Applications.

[30]  Josef Kittler,et al.  Contextual classification of multispectral pixel data , 1984, Image Vis. Comput..

[31]  Max A. Viergever,et al.  Segmenting the posterior ribs in chest radiographs by iterated contextual pixel classification , 2003, SPIE Medical Imaging.

[32]  Robert P. W. Duin,et al.  Multi-class linear feature extraction by nonlinear PCA , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[33]  J. M. Hans du Buf,et al.  A review of recent texture segmentation and feature extraction techniques , 1993 .

[34]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[35]  Jonny Eriksson,et al.  Feature reduction for classification of multidimensional data , 2000, Pattern Recognit..

[36]  John Haslett,et al.  Maximum likelihood discriminant analysis on the plane using a Markovian model of spatial context , 1985, Pattern Recognit..

[37]  F. Marriott The interpretation of multiple observations , 1974 .

[38]  B. Ginneken,et al.  Automatic segmentation of lung fields in chest radiographs. , 2000 .

[39]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[40]  Erik Bølviken,et al.  A general parameter updating approach to image classification , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[41]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[42]  A. D. Gordon A survey of constrained classification , 1996 .

[43]  J. Koenderink The structure of images , 2004, Biological Cybernetics.

[44]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[45]  Wilson S. Geisler,et al.  Multichannel Texture Analysis Using Localized Spatial Filters , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Reinhold Häb-Umbach,et al.  Multi-class linear dimension reduction by generalized Fisher criteria , 2000, INTERSPEECH.

[47]  Max A. Viergever,et al.  Interactive shape models , 2003, SPIE Medical Imaging.

[48]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[51]  K. Doi,et al.  Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists' detection of pulmonary nodules. , 2000, AJR. American journal of roentgenology.

[52]  Bram van Ginneken,et al.  Multi-scale texture classification from generalized locally orderless images , 2003, Pattern Recognit..

[53]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[54]  A.D. Hoover,et al.  Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response , 2000, IEEE Transactions on Medical Imaging.

[55]  R. Fisher THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[56]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[57]  David G. Stork,et al.  Pattern Classification , 1973 .

[58]  Anil K. Jain,et al.  MRF model-based algorithms for image segmentation , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[59]  Bram van Ginneken,et al.  Static posterior probability fusion for signal detection: applications in the detection of interstitial diseases in chest radiographs , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[60]  Linda G. Shapiro,et al.  Image Segmentation Techniques , 1984, Other Conferences.

[61]  J. Besag Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[62]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[63]  Charles A. Bouman,et al.  A multiscale random field model for Bayesian image segmentation , 1994, IEEE Trans. Image Process..

[64]  Bram van Ginneken,et al.  Multi-scale Nodule Detection in Chest Radiographs , 2003, MICCAI.

[65]  Robert P. W. Duin,et al.  Non-iterative Heteroscedastic Linear Dimension Reduction for Two-Class Data , 2002, SSPR/SPR.

[66]  Max A. Viergever,et al.  Model-based segmentation of abdominal aortic aneurysms in CTA images , 2003, SPIE Medical Imaging.

[67]  Robert P. W. Duin,et al.  Dimensionality Reduction by Canonical Contextual Correlation Projections , 2004, ECCV.

[68]  Bayya Yegnanarayana,et al.  Segmentation of Gabor-filtered textures using deterministic relaxation , 1996, IEEE Trans. Image Process..

[69]  Max A. Viergever,et al.  Computer-aided diagnosis in chest radiography: a survey , 2001, IEEE Transactions on Medical Imaging.

[70]  Reinhold Häb-Umbach,et al.  An investigation of cepstral parameterisations for large vocabulary speech recognition , 1999, EUROSPEECH.

[71]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[72]  Bram van Ginneken,et al.  Pixel position regression - application to medical image segmentation , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[73]  J. A. Richards,et al.  Pixel Labeling by Supervised Probabilistic Relaxation , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Allen L. Edwards,et al.  Multiple Regression and Analysis of Variance and Covariance , 1987 .

[75]  Jing-Yu Yang,et al.  Algebraic feature extraction for image recognition based on an optimal discriminant criterion , 1993, Pattern Recognit..

[76]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[77]  A. Willsky Multiresolution Markov models for signal and image processing , 2002, Proc. IEEE.

[78]  J. H. Kulick,et al.  Automatic Rib Detection in Chest Radiographs , 1977, IJCAI.

[79]  A. Ardeshir Goshtasby,et al.  Automatic detection of rib borders in chest radiographs , 1995, IEEE Trans. Medical Imaging.

[80]  Paul Suetens,et al.  Temporal subtraction of thorax CR images using a statistical deformation model , 2003, IEEE Transactions on Medical Imaging.

[81]  Ljubomir J. Buturovic Toward Bayes-Optimal Linear Dimension Reduction , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[82]  H. P. Decell,et al.  Feature combinations and the divergence criterion , 1977 .

[83]  Bram van Ginneken,et al.  Automatic delineation of ribs in frontal chest radiographs , 2000, Medical Imaging: Image Processing.

[84]  Bram van Ginneken,et al.  Supervised segmentation by iterated contextual pixel classification , 2002, Object recognition supported by user interaction for service robots.

[85]  Marvin E. Bauer,et al.  Integrating Contextual Information with per-Pixel Classification for Improved Land Cover Classification , 2000 .

[86]  Kunio Doi,et al.  Automatic detection of abnormalities in chest radiographs using local texture analysis , 2002, IEEE Transactions on Medical Imaging.

[87]  Luc Florack,et al.  On the Behavior of Spatial Critical Points under Gaussian Blurring. A Folklore Theorem and Scale-Space Constraints , 2001, Scale-Space.

[88]  Anuj Srivastava,et al.  Optimal linear representations of images for object recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[89]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[90]  Xie Yuan-dan,et al.  Survey on Image Segmentation , 2002 .

[91]  M. Giger,et al.  Digital image subtraction of temporally sequential chest images for detection of interval change. , 1994, Medical physics.

[92]  J. Rice Mathematical Statistics and Data Analysis , 1988 .

[93]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[94]  S. Klinke,et al.  Exploratory Projection Pursuit , 1995 .

[95]  Andrew P. Witkin,et al.  Scale-Space Filtering , 1983, IJCAI.

[96]  A. L. Edwards,et al.  Multiple Regression and the Analysis of Variance and Covariance , 1986, The Mathematical Gazette.

[97]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[98]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[99]  Gerard V. Trunk,et al.  A Problem of Dimensionality: A Simple Example , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[100]  Olivier Y. de Vel,et al.  Comparative analysis of statistical pattern recognition methods in high dimensional settings , 1994, Pattern Recognit..

[101]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[102]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[103]  Robert P. W. Duin,et al.  Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[104]  A. Guttentag,et al.  Keep your eyes on the ribs: the spectrum of normal variants and diseases that involve the ribs. , 1999, Radiographics : a review publication of the Radiological Society of North America, Inc.

[105]  Nico Karssemeijer,et al.  A relaxation method for image segmentation using a spatially dependent stochastic model , 1990, Pattern Recognit. Lett..

[106]  Jerry L Prince,et al.  Current methods in medical image segmentation. , 2000, Annual review of biomedical engineering.

[107]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[108]  Kannan,et al.  ON IMAGE SEGMENTATION TECHNIQUES , 2022 .

[109]  Frank Fischbach,et al.  Dual-energy chest radiography with a flat-panel digital detector: revealing calcified chest abnormalities. , 2003, AJR. American journal of roentgenology.

[110]  G. McLachlan Discriminant Analysis and Statistical Pattern Recognition , 1992 .

[111]  K. Hirata,et al.  The ribs: anatomic and radiologic considerations. , 1999, Radiographics : a review publication of the Radiological Society of North America, Inc.

[112]  K Doi,et al.  Image feature analysis and computer-aided diagnosis in digital radiography: automated delineation of posterior ribs in chest images. , 1991, Medical physics.

[113]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[114]  Robert P. W. Duin,et al.  Dimensionality reduction of image features using the canonical contextual correlation projection , 2005, Pattern Recognit..

[115]  Timothy F. Cootes,et al.  Statistical models of appearance for medical image analysis and computer vision , 2001, SPIE Medical Imaging.

[116]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[117]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[118]  Ker-Chau Li Sliced inverse regression for dimension reduction (with discussion) , 1991 .

[119]  Alexander A. Sawchuk,et al.  Supervised Textured Image Segmentation Using Feature Smoothing and Probabilistic Relaxation Techniques , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[120]  R.M. McElhaney,et al.  Algorithms for graphics and image processing , 1983, Proceedings of the IEEE.

[121]  Claus Weihs,et al.  Optimal vs. classical linear dimension reduction , 1998 .

[122]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[123]  Bram van Ginneken,et al.  Comparative study of retinal vessel segmentation methods on a new publicly available database , 2004, SPIE Medical Imaging.

[124]  James T Dobbins,et al.  Quantitative , 2020, Psychology through Critical Auto-Ethnography.

[125]  Qiong Jackson,et al.  Adaptive Bayesian contextual classification based on Markov random fields , 2002, IEEE Trans. Geosci. Remote. Sens..

[126]  H. P. Decell,et al.  Linear dimension reduction and Bayes classification , 1981, Pattern Recognit..

[127]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[128]  C. H. Chen,et al.  On information and distance measures, error bounds, and feature selection , 1976, Information Sciences.

[129]  Ker-Chau Li,et al.  Slicing Regression: A Link-Free Regression Method , 1991 .

[130]  Ker-Chau Li,et al.  On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma , 1992 .

[131]  Bram van Ginneken,et al.  Segmentation of the posterior ribs in chest radiographs using iterated contextual pixel classification , 2006, IEEE Transactions on Medical Imaging.

[132]  Josef Kittler,et al.  Relaxation labelling algorithms - a review , 1986, Image Vis. Comput..

[133]  Max A. Viergever,et al.  The MDF discrimination measure: Fisher in disguise , 2004, Neural Networks.

[134]  Mads Nielsen,et al.  Detection of interstitial lung disease in PA chest radiographs , 2004, SPIE Medical Imaging.

[135]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.