Validating a Cost Model for Wide Area Applications

Recent technology advances have enabled wide area application (WAA) processing with WebSources that are accessible via the WWW. One challenge to query processing in wide area environments is the unpredictable behavior of WebSources in the dynamic WAN. There can be wide variability in the latency (delay) of accessing these sources, and the delay could depend on the network and server workloads. These workloads are often affected by parameters such as the Time of Day, the Day of Week, etc. Another challenge is that autonomous WebSources may not provide metrics needed for accurate cost estimation. In this paper, we describe a case study in developing a cost model for WebSources in the context of a wrapper mediator architecture. We document our experiences in validating this cost model, and note successes and lessons learned. Using experimental data of query feedback from several WebSources, we characterize sources as having High or Low Prediction Accuracy, with respect to the ability of the cost model to predict access costs. We also identify WebSource characteristics of the query feedback that are correlated with High or Low prediction accuracy. A cost model is an important component of the mediator query optimizer. It uses the cost estimations of the Wrapper cost model to obtain the cost of a mediator query accessing multiple WebSources simultaneously. We examine how our research can be used to develop robust query optimization techniques for WAA processing in noisy environments.