Toward Self-Organizing Search Systems

The huge amount of images, videos, and music clips produced everyday by various digital devices must be processed. Firstly, this kind of data calls for content-based search or similarity search rather than keyword-based or text-based search. Secondly, new scalable and efficient methods capable of storing and querying such data must be developed. Although many distributed approaches exist, one of the most suitable and flexible is provided by self-organizing systems. These systems exhibit high resistance to failures in dynamically changing environments. In this chapter, we propose a general three-layer model for designing and implementing a self-organizing system that aims at searching in multimedia data. This model gives a developer guidelines about what component must be implemented, and how they should behave. The usability of this model is illustrated on a system called Metric Social Network. The architecture of this system is based on the social network theory that is utilized for establishing links between nodes. The system’s properties are verified by organizing and searching in 10 million images.

[1]  Nick Chater,et al.  The Generalized Universal Law of Generalization , 2001, ArXiv.

[2]  Hector Garcia-Molina,et al.  Routing indices for peer-to-peer systems , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[3]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[4]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[5]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[6]  Mark S. Granovetter.  The Strength of Weak Ties , 1973, American Journal of Sociology.

[7]  Jan Paredaens,et al.  Advances in Database Systems , 1994 .

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Gerhard Weikum,et al.  Self-tuning Database Technology and Information Services: from Wishful Thinking to Viable Engineering , 2002, VLDB.

[10]  Pavel Zezula,et al.  Building self-organized image retrieval network , 2008, LSDS-IR '08.

[11]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[12]  Carlos Gershenson,et al.  How can we think the complex , 2004, nlin/0402023.

[13]  Pavel Zezula,et al.  A Self-Organized System for Content-Based Search in Multimedia , 2008, 2008 Tenth IEEE International Symposium on Multimedia.

[14]  Pavel Zezula,et al.  Similarity Grid for Searching in Metric Spaces , 2004, DELOS.

[15]  Andrew B. Kahng,et al.  Cooperative Mobile Robotics: Antecedents and Directions , 1997, Auton. Robots.

[16]  Grégoire Nicolis,et al.  Self-Organization in nonequilibrium systems , 1977 .

[17]  Luca Maria Gambardella,et al.  AntHocNet: an adaptive nature-inspired algorithm for routing in mobile ad hoc networks , 2005, Eur. Trans. Telecommun..

[18]  Hanan Samet,et al.  A distributed quadtree index for peer-to-peer settings , 2005, 21st International Conference on Data Engineering (ICDE'05).

[19]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM 2001.

[20]  Timothy W. Finin,et al.  A Framework for Distributed Trust Management , 2001 .

[21]  Steven Johnson,et al.  Emergence: The Connected Lives of Ants, Brains, Cities, and Software , 2001 .

[22]  Pavel Zezula,et al.  Querying Similarity in Metric Social Networks , 2007, NBiS.

[23]  Pavel Zezula,et al.  Similarity Search - The Metric Space Approach , 2005, Advances in Database Systems.

[24]  W. Foster,et al.  Group transmission of predator avoidance behaviour in a marine insect: The trafalgar effect , 1981, Animal Behaviour.

[25]  Pavel Zezula,et al.  Adaptive Approximate Similarity Searching through Metric Social Networks , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[26]  Andrea Esuli,et al.  CoPhIR: a Test Collection for Content-Based Image Retrieval , 2009, ArXiv.

[27]  Peter N. Yianilos,et al.  Data structures and algorithms for nearest neighbor search in general metric spaces , 1993, SODA '93.

[28]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[29]  Pavel Zezula,et al.  D-Index: Distance Searching Index for Metric Data Sets , 2003, Multimedia Tools and Applications.

[30]  Guy Theraulaz,et al.  Self-Organization in Biological Systems , 2001, Princeton studies in complexity.

[31]  Angelos D. Keromytis,et al.  Trust management for IPsec , 2002, TSEC.

[32]  Christos Doulkeridis,et al.  Peer-to-Peer Similarity Search in Metric Spaces , 2007, VLDB.

[33]  David Novak,et al.  M-Chord: a scalable distributed similarity search structure , 2006, InfoScale '06.

[34]  Joan Feigenbaum,et al.  Compliance Checking in the PolicyMaker Trust Management System , 1998, Financial Cryptography.

[35]  Hanan Samet,et al.  Foundations of multidimensional and metric data structures , 2006, Morgan Kaufmann series in data management systems.

[36]  Michael Stonebraker,et al.  The Morgan Kaufmann Series in Data Management Systems , 1999 .

[37]  Sandhya Dwarkadas,et al.  Peer-to-peer information retrieval using self-organizing semantic overlay networks , 2003, SIGCOMM '03.

[38]  Joan Feigenbaum,et al.  Decentralized trust management , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[39]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[40]  Jeffrey K. Uhlmann,et al.  Satisfying General Proximity/Similarity Queries with Metric Trees , 1991, Inf. Process. Lett..

[41]  Hanan Samet,et al.  The Quadtree and Related Hierarchical Data Structures , 1984, CSUR.

[42]  David Novak,et al.  On scalability of the similarity search in the world of peers , 2006, InfoScale '06.

[43]  Pavel Zezula,et al.  Similarity Search: The Metric Space Approach (Advances in Database Systems) , 2005 .

[44]  W. Ross Ashby,et al.  Principles of the Self-Organizing System , 1991 .

[45]  Joan Feigenbaum,et al.  The Role of Trust Management in Distributed Systems Security , 2001, Secure Internet Programming.

[46]  Daniel Stutzbach,et al.  Understanding churn in peer-to-peer networks , 2006, IMC '06.

[47]  Stafford Beer,et al.  Decision and Control: The Meaning of Operational Research and Management Cybernetics , 1966 .

[48]  Beng Chin Ooi,et al.  iDistance: An adaptive B+-tree based indexing method for nearest neighbor search , 2005, TODS.

[49]  Hector Garcia-Molina,et al.  Semantic Overlay Networks for P2P Systems , 2004, AP2PC.

[50]  Zhu Han,et al.  A trust evaluation framework in distributed networks: Vulnerability analysis and defense against attacks , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[51]  Francis Heylighen,et al.  The Science of Self-Organization and Adaptivity , 1999 .

[52]  Pavel Zezula,et al.  M-tree: An Efficient Access Method for Similarity Search in Metric Spaces , 1997, VLDB.

[53]  Claudio Gennaro,et al.  MRoute: A Peer-to-Peer Routing Index for Similarity Search in Metric Spaces , 2007 .

[54]  Pavel Zezula,et al.  A Content-Addressable Network for Similarity Search in Metric Spaces , 2005, DBISP2P.

[55]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[56]  Hanan Samet,et al.  Foundations of Multidimensional and Metric Data Structures (The Morgan Kaufmann Series in Computer Graphics and Geometric Modeling) , 2005 .

[57]  Sharon L. Milgram,et al.  The Small World Problem , 1967 .

[58]  Farnoush Banaei Kashani,et al.  SWAM: a family of access methods for similarity-search in peer-to-peer data networks , 2004, CIKM '04.

[59]  Christian Bettstetter,et al.  Self-organization in communication networks: principles and design paradigms , 2005, IEEE Communications Magazine.