Approaches for content-based retrieval of surface defect images ; Pintavirhekuvien sisältöpohjaisesta hausta

There are two properties which all industrial manufacturing processes try to optimize: speed and quality. Speed can also be called throughput and tells how much products can be created in a specified time. The higher speeds you have the better. Quality means the perceived goodness of the finished product. Broken or defective products simply don't sell, so they must be eliminated. These are contradicting goals. The larger the manufacturing volumes, the less time there is to inspect a single product, or the more inspectors are required. A good example is paper manufacturing. A single paper machine can produce a sheet of paper several meters wide and several hundred kilometers long in just a few hours. It is impossible to inspect these kinds of volumes by hand. In this thesis the indexing and retrieval of defect images taken by an automated inspection machine is examined. Some of the images taken contain serious defects such as holes, while others are less grave. The goal is to try to develop automated methods to find the serious fault images from large databases using only the information in the images. This means that there are no annotations. This is called content-based image retrieval, or CBIR. This problem is examined in two different ways. First the PicSOM CBIR tool's suitability for this task is evaluated. PicSOM is a platform for content-based image retrieval developed at the Laboratory of Computer and Information Science, Helsinki University of Technology. PicSOM has earlier been succesfully applied to various different CBIR tasks. The other part involves developing new algorithms for efficient indexing of large, high-dimensional databases. The Evolving Tree (ETree), a novel hierarchical, tree-shaped, self-organizing neural network is presented and analyzed. It is noticeably faster than classical methods, while still obtaining good results. The suitability and performance of both CBIR and ETree on this problem is evaluated using several different experiments. The results show that both approaches are applicable for this real world quality inspection problem with good results.%%%%Kaikki teolliset tuotantolaitokset pyrkivat optimoimaan kahta ominaisuutta: nopeutta ja laatua. Nopeus kertoo kuinka nopeasti tuotteet pystytaan valmistamaan. Mita suurempi valmistunopeus on, sita parempi. Laatu taas mittaa lopullisen tuotteen subjektiivista hyvyytta. Koska viallisia tuotteita ei yleensa saa kaupaksi, ne on poistettava. Nama ovat vastakkaisia tavoitteita. Mita enemman tuotetta valmistetaan, sita vahemman aikaa yksittaisen kappaleen tarkastamiseen jaa. Hyvana esimerkkina kay paperin valmistaminen. Normaali paperikone tuottaa muutamassa tunnissa monta sataa kilometria paperia. Nain suurien tuotantomaarien tarkastaminen kasin on mahdotonta. Tassa vaitoskirjassa tutkitaan automaattisen tarkastuskoneen ottamien kuvien indeksointia ja hakua. Jotkut naista kuvista sisaltavat vakavia virhetilanteita, kuten reikia. Toiset ovat vahemman vakavia. Tavoitteena on kehittaa automaattisia menetelmia, jotka loytavat vakavat viat…

[1]  S. Shaler,et al.  The tensile testing of individual wood fibers using environmental scanning electron microscopy and video image analysis , 1995 .

[2]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[3]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[4]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[5]  David A. Forsyth,et al.  Finding Naked People , 1996, ECCV.

[6]  S. I'Anson.  Identification of periodic marks in paper and board by image analysis using two-dimensional fast Fourier transforms. 1. The basics , 1995 .

[7]  Chihli Hung,et al.  A constructive and hierarchical self-organizing model in a non-stationary environment , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[8]  Paul H. Lewis,et al.  Integrated Image Content and Metadata Search and Retrieval across Multiple Databases , 2003, CIVR.

[9]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[10]  Serkan Kiranyaz.  Advanced Techniques for Content-Based Management of Multimedia Databases , 2005 .

[11]  HongJiang Zhang,et al.  Scheme for visual feature-based image indexing , 1995, Electronic Imaging.

[12]  Helge J. Ritter,et al.  Hyperbolic Self-Organizing Maps for Semantic Navigation , 2001, NIPS.

[13]  Pasi Koikkalainen,et al.  Self-organizing hierarchical feature maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[14]  Tat-Seng Chua,et al.  An integrated color-spatial approach to content-based image retrieval , 1995, MULTIMEDIA '95.

[15]  Stéphane Bressan,et al.  Introduction to Database Systems , 2005 .

[16]  Björn Johansson,et al.  A Survey on : Contents Based Search in Image Databases , 2000 .

[17]  Pasi Koikkalainen,et al.  Progress with the Tree-Structured Self-Organizing Map , 1994, ECAI.

[18]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[19]  P. Sneath,et al.  Numerical Taxonomy , 1962, Nature.

[20]  Gail A. Carpenter,et al.  S-TREE: self-organizing trees for data clustering and online vector quantization , 2001, Neural Networks.

[21]  Horst Eidenberger.  A Video Browsing Application based on visual MPEG-7 Descriptors and Self-Organising Maps , 2004 .

[22]  Stefan Wermter,et al.  A dynamic adaptive self-organising hybrid model for text clustering , 2003, Third IEEE International Conference on Data Mining.

[23]  Ling Guan,et al.  Self-organizing tree map for eliminating impulse noise with random intensity distributions , 1998, J. Electronic Imaging.

[24]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.

[25]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[26]  Gerald Sommer,et al.  Dynamic Cell Structure Learns Perfectly Topology Preserving Map , 1995, Neural Computation.

[27]  Hanan Samet,et al.  The Quadtree and Related Hierarchical Data Structures , 1984, CSUR.

[28]  V. Burzevski,et al.  Hierarchical growing cell structures , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[29]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[30]  J.-P. Bernie,et al.  Local grammage distribution and formation of paper by light transmission image analysis , 1996 .

[31]  Emanuele Trucco,et al.  Computer and Robot Vision , 1995 .

[32]  Jon Louis Bentley,et al.  Multidimensional Binary Search Trees in Database Applications , 1979, IEEE Transactions on Software Engineering.

[33]  A. Rauber,et al.  Investigation of alternative strategies and quality measures for controlling the growth process of the growing hierarchical self-organizing map , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[34]  Carla E. Brodley,et al.  ASSERT: A Physician-in-the-Loop Content-Based Retrieval System for HRCT Image Databases , 1999, Comput. Vis. Image Underst..

[35]  Bernd Fritzke,et al.  Kohonen Feature Maps and Growing Cell Structures - a Performance Comparison , 1992, NIPS.

[36]  Hans Hinterberger,et al.  Content-Based Image Retrieval in Astronomy , 2000, Information Retrieval.

[37]  Goh Wee Leng,et al.  An Autoregressive Approach to Surface Texture Analysis , 1994, Int. J. Pattern Recognit. Artif. Intell..

[38]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[39]  Hong Yan,et al.  Handwritten digit recognition by neural 'gas' model and population decoding , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[40]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[41]  Linda G. Shapiro,et al.  Computer and Robot Vision , 1991 .

[42]  Amarnath Gupta,et al.  Virage image search engine: an open framework for image management , 1996, Electronic Imaging.

[43]  William E. Higgins,et al.  Texture Segmentation using 2-D Gabor Elementary Functions , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Ling Guan,et al.  Automatic machine interactions for content-based image retrieval using a self-organizing tree map architecture , 2002, IEEE Trans. Neural Networks.

[45]  Brian Everitt,et al.  Cluster analysis , 1974 .

[46]  Iivari Kunttu.  Shape and Gray Level Descriptors for Surface Defect Image Retrieval and Classification , 2005 .

[47]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[48]  Sven Behnke,et al.  Competitive neural trees for pattern classification , 1998, IEEE Trans. Neural Networks.

[49]  S. Marshall,et al.  Review of shape coding techniques , 1989, Image Vis. Comput..

[50]  Helge Ritter,et al.  A hierarchically growing hyperbolic self-organizing map for rapid structuring of large data sets , 2005 .

[51]  Ishwar K. Sethi,et al.  Image retrieval using hierarchical self-organizing feature maps , 1999, Pattern Recognit. Lett..

[52]  Forest Baskett,et al.  An Algorithm for Finding Nearest Neighbors , 1975, IEEE Transactions on Computers.

[53]  Bidyut Baran Chaudhuri,et al.  Texture Segmentation Using Fractal Dimension , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Erkki Oja,et al.  PicSOM - content-based image retrieval with self-organizing maps , 2000, Pattern Recognit. Lett..

[55]  E. Cuadros-Vargas,et al.  Introduction to the SAM-S M* and MAM-S M* families , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[56]  Bernd Fritzke.  Growing Grid — a self-organizing network with constant neighborhood range and adaptation strength , 1995, Neural Processing Letters.

[57]  Peter Stanchev,et al.  Content-Based Image Retrieval Systems , 2001 .

[58]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[59]  Tao Li,et al.  A structure-parameter-adaptive (SPA) neural tree for the recognition of large character set , 1995, Pattern Recognit..

[60]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[61]  Christos Faloutsos,et al.  The R+-Tree: A Dynamic Index for Multi-Dimensional Objects , 1987, VLDB.

[62]  Bala Srinivasan,et al.  Dynamic self-organizing maps with controlled growth for knowledge discovery , 2000, IEEE Trans. Neural Networks Learn. Syst..

[63]  Matti Niskanen,et al.  A visual training based approach to surface inspection , 2003 .

[64]  Markus Koskela,et al.  Interactive image retrieval using self-organizing maps , 2003 .

[65]  Andreas Rauber,et al.  Recent Advances with the Growing Hierarchical Self-Organizing Map , 2001, WSOM.

[66]  Ari Visa,et al.  Comparison of Combined Shape Descriptors for Irregular Objects , 1997, BMVC.

[67]  Seong-Whan Lee,et al.  A self-organizing neural tree for large-set pattern classification , 1998, IEEE Trans. Neural Networks.

[68]  Michael Stonebraker,et al.  Chabot: Retrieval from a Relational Database of Images , 1995, Computer.

[69]  Kate Smith-Miles,et al.  HDGSOM: a modified growing self-organizing map for high dimensional data clustering , 2004, Fourth International Conference on Hybrid Intelligent Systems (HIS'04).

[70]  Anil K. Jain,et al.  A Survey of Automated Visual Inspection , 1995, Comput. Vis. Image Underst..

[71]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[72]  Franz Aurenhammer,et al.  Voronoi diagrams—a survey of a fundamental geometric data structure , 1991, CSUR.

[73]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[74]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[75]  Jorma Laaksonen,et al.  PicSOM Experiments in TRECVID 2018 , 2015, TRECVID.

[76]  Axel Funk,et al.  Die GNU General Public License, Version 3 , 2007 .

[77]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[78]  Oliver Günther,et al.  Multidimensional access methods , 1998, CSUR.

[79]  Jukka Iivarinen,et al.  Content-based retrieval of surface defect images with MPEG-7 descriptors , 2003, International Conference on Quality Control by Artificial Vision.

[80]  Hans-Peter Kriegel,et al.  The X-tree : An Index Structure for High-Dimensional Data , 2001, VLDB.

[81]  L. Hubert,et al.  Quadratic assignment as a general data analysis strategy. , 1976 .

[82]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[83]  S. G. Tyan,et al.  Median Filtering: Deterministic Properties , 1981 .

[84]  Tommy W. S. Chow,et al.  Content-based image retrieval using growing hierarchical self-organizing quadtree map , 2005, Pattern Recognit..

[85]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[86]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[87]  Hans-Peter Kriegel,et al.  The R*-tree: an efficient and robust access method for points and rectangles , 1990, SIGMOD '90.

[88]  Dragana Brzakovic,et al.  Designing a defect classification system: A case study , 1996, Pattern Recognit..

[89]  T. Smith.  A digital library for geographically referenced materials , 1996, Computer.

[90]  Thomas Villmann,et al.  Growing a hypercubical output space in a self-organizing feature map , 1997, IEEE Trans. Neural Networks.

[91]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[92]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[93]  Huan Liu,et al.  Learning a Neural Tree , 1992 .

[94]  Linda G. Shapiro,et al.  Computer and Robot Vision (Volume II) , 2002 .

[95]  Kannan Ramchandran,et al.  Multimedia Analysis and Retrieval System (MARS) Project , 1996, Data Processing Clinic.

[96]  Ricardo A. Baeza-Yates,et al.  Searching in metric spaces , 2001, CSUR.

[97]  Jorma Laaksonen,et al.  Content-Based Browsing of Mail-Order Catalogue with PicSOM System , 2003 .

[98]  Jorma Laaksonen,et al.  Interactive Retrieval in Facial Image Database Using Self-Organizing Maps , 2005, MVA.

[99]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[100]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[101]  Suchendra M. Bhandarkar,et al.  Multiscale image segmentation using a hierarchical self-organizing map , 1997, Neurocomputing.

[102]  J. Clark,et al.  Video summarization and browsing using growing cell structures , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[103]  Ari Visa,et al.  An adaptive texture and shape based defect classification , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[104]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[105]  Petri Vuorimaa,et al.  A Defect Detection Scheme for Web Surface Inspection , 2000, Int. J. Pattern Recognit. Artif. Intell..

[106]  Eric J. Pauwels,et al.  Finding Salient Regions in Images: Nonparametric Clustering for Image Segmentation and Grouping , 1999, Comput. Vis. Image Underst..

[107]  J. Dunn.  Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[108]  Kate A. Smith,et al.  Enhancing clustering performance of feature maps using randomness , 2005 .

[109]  Henry Fuchs,et al.  On visible surface generation by a priori tree structures , 1980, SIGGRAPH '80.

[110]  Dieter Merkl,et al.  Visualizing Similarities in High Dimensional Input Spaces with a Growing and Splitting Neural Network , 1996, ICANN.

[111]  Alan F. Murray,et al.  Paper curl prediction and control using neural networks , 1999 .

[112]  A. Roli.  Artificial Neural Networks , 2012, Lecture Notes in Computer Science.

[113]  Topi Mäenpää,et al.  The local binary pattern approach to texture analysis - extensions and applications , 2003 .

[114]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[115]  Victoria J. Hodge,et al.  Hierarchical growing cell structures: TreeGCS , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[116]  N. Milosavljevic,et al.  Modeling a scrubber using feed-forward neural networks , 1999 .

[117]  Antonin Guttman,et al.  R-trees: a dynamic index structure for spatial searching , 1984, SIGMOD '84.

[118]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[119]  R. R. Bresee,et al.  Characterizing nonwoven web structure using image analysis techniques , 1997 .

[120]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[121]  Hirotaka Inoue,et al.  SONG: Self-Organizing Neural Grove , 2008 .

[122]  Risto Miikkulainen,et al.  Incremental grid growing: encoding high-dimensional structure into a two-dimensional feature map , 1993, IEEE International Conference on Neural Networks.

[123]  Erkki Oja,et al.  PicSOM - A Framework for Content-Based Image Database Retrieval using Self-Organizing Maps , 1999 .

[124]  Michael S. Lew,et al.  Principles of Visual Information Retrieval , 2001, Advances in Pattern Recognition.

[125]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[126]  Thierry Pun,et al.  Content-based query of image databases: inspirations from text retrieval , 2000, Pattern Recognit. Lett..

[127]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.