Deadline and QoS Aware Data Warehouse

A data warehouse infrastructure needs to support the requirement of (day time) ad hoc query response time and (night time) batch workload completion time. The following tasks need to be finished in a batch window: (1) Apply one day's delta data to the base tables; (2) refresh MQTs (Materialized Query Tables) for ad hoc queries and batch workloads; (3) run batch queries. Tools are available to optimize each step; however, many factors need to be considered for improving the overall performance of a data warehouse (i.e. meeting batch window deadline and ad hoc query response time). We have prototyped a Data Warehouse Operation Advisor to systematically study each component contributing to the batch window problem, and then perform global optimization to achieve desired results!

[1]  K. Selçuk Candan,et al.  Load and network aware query routing for information integration , 2005, 21st International Conference on Data Engineering (ICDE'05).

[2]  Hamid Pirahesh,et al.  fAST refresh using mass query optimization , 2001, Proceedings 17th International Conference on Data Engineering.

[3]  Surajit Chaudhuri,et al.  Automated Selection of Materialized Views and Indexes in SQL Databases , 2000, VLDB.

[4]  Hamid Pirahesh,et al.  Recommending materialized views and indexes with the IBM DB2 design advisor , 2004, International Conference on Autonomic Computing, 2004. Proceedings..

[5]  Hamid Pirahesh,et al.  Recommending materialized views and indexes with the IBM DB2 design advisor , 2004 .

[6]  Sam Lightstone,et al.  DB2 Design Advisor: Integrated Automatic Physical Database Design , 2004, VLDB.

[7]  Daniel C. Zilio,et al.  DB2 advisor: an optimizer smart enough to recommend its own indexes , 2000, Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073).