Selectively materializing data in mediators by analyzing user queries

There is currently great interest in building information mediators that can integrate information from multiple data sources such as databases or Web sources. The query response time in such mediators is typically quite high. We present an approach for optimizing the performance of information mediators by selectively materializing data. We first present our overall framework for materialization in a mediator environment. We then stress the need to materialize data selectively and outline the factors that must be considered to select data to materialize. We present an algorithm for identifying classes of data to materialize by analyzing one of the factors which is the distribution of user queries. We present results with an implemented version of our optimization system for the Ariadne information mediator which show significant performance improvement over no materialization and existing schemes such as page level caching.

[1]  Jennifer Widom,et al.  Change detection in hierarchically structured information , 1996, SIGMOD '96.

[2]  Alon Y. Halevy,et al.  An adaptive query execution system for data integration , 1999, SIGMOD '99.

[3]  Jian Yang,et al.  Algorithms for Materialized View Design in Data Warehousing Environment , 1997, VLDB.

[4]  Peter B. Danzig,et al.  A Hierarchical Internet Object Cache , 1996, USENIX ATC.

[5]  Robert M. MacGregor,et al.  A Deductive Pattern Matcher , 1988, AAAI.

[6]  K. Selçuk Candan,et al.  Query caching and optimization in distributed mediator systems , 1996, SIGMOD '96.

[7]  Gio Wiederhold,et al.  Mediators in the architecture of future information systems , 1992, Computer.

[8]  Arthur M. Keller,et al.  A predicate-based caching scheme for client-server database architectures , 1994, Proceedings of 3rd International Conference on Parallel and Distributed Information Systems.

[9]  Zachary G. Ives,et al.  An adaptive query execution engine for data integration , 1999 .

[10]  Jennifer Widom,et al.  Information translation, mediation, and mosaic-based browsing in the TSIMMIS system , 1995, SIGMOD '95.

[11]  Gang Zhou,et al.  A framework for supporting data integration using the materialized and virtual approaches , 1996, SIGMOD '96.

[12]  Arthur M. Keller,et al.  A predicate-based caching scheme for client-server database architectures , 1994, Proceedings of 3rd International Conference on Parallel and Distributed Information Systems.

[13]  Craig A. Knoblock,et al.  Intelligent caching: selecting, representing, and reusing data in an information server , 1994, CIKM '94.

[14]  Vipul Kashyap,et al.  InfoSleuth: Semantic Integration of Information in Open and Dynamic Environments (Experience Paper) , 1997, SIGMOD Conference.

[15]  Tiziana Catarci,et al.  Materializing the Web , 1998, Proceedings. 3rd IFCIS International Conference on Cooperative Information Systems (Cat. No.98EX122).

[16]  Maria-Esther Vidal,et al.  Optimization of Wrappers and Mediators for Web Accessible Data Sources (WebSources) , 1998, Workshop on Web Information and Data Management.

[17]  Inderpal Singh Mumick,et al.  Selection of Views to Materialize Under a Maintenance Cost Constraint , 1999, ICDT.

[18]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[19]  Oren Etzioni,et al.  A softbot-based interface to the Internet , 1994, CACM.

[20]  Timos K. Sellis,et al.  Designing Data Warehouses , 1999, Data Knowl. Eng..

[21]  Craig A. Knoblock,et al.  Modeling Web Sources for Information Integration , 1998, AAAI/IAAI.

[22]  Divesh Srivastava,et al.  The Information Manifold , 1995 .

[23]  Michael R. Genesereth,et al.  Infomaster: an information integration system , 1997, SIGMOD '97.

[24]  Divesh Srivastava,et al.  Semantic Data Caching and Replacement , 1996, VLDB.

[25]  Yi-Shin Chen,et al.  TheaterLoc: A Case Study in Building An Information Integration System , 1999, Intelligent Information Integration.

[26]  Peter Scheuermann,et al.  WATCHMAN : A Data Warehouse Intelligent Cache Manager , 1996, VLDB.

[27]  Ramakrishnan Srikant,et al.  Mining generalized association rules , 1995, Future Gener. Comput. Syst..