Efficiently tracking provenance in scientific workflows

Tracking the provenance of data produced by a workflow execution involves answering reachability queries over large provenance graphs, which can be expensive. For that, we present compact labeling schemes for efficiently answering reachability queries over provenance graphs that are derived from executions of a given workflow specification. The idea is to assign each node a reachability label such that using only the labels of any two nodes, we can quickly decide if one can reach the other. Our proposed schemes build logarithmic-size labels in linear time, and answer any query in constant time. In this dissertation, we consider the reachability labeling problem for a variety of workflow settings. First, we study the static labeling problem, where the entire provenance graph is given as input. For workflows with well-nested loops and forks (i.e., parallel executions), we develop a skeleton-based labeling approach which uses the labeling for the specification as an effective skeleton for designing the labeling for its executions. Next, we turn to the dynamic labeling problem, where the input provenance graph grows over time but the nodes must be labeled on-the-fly. We first show that, in general, for workflows that contain arbitrary recursion, dynamic labeling of their executions requires long (linear-size) labels. Nevertheless, we identify a natural class of workflows with linear recursion, for which dynamic, yet compact (logarithmic-size) labeling is possible. Finally, we revisit the dynamic labeling problem when fine-grained dependencies between inputs and outputs of modules are defined over multiple workflow views. It turns out that the restriction of linear recursion, which suffices to reduce the label length before, is no longer helpful. However, for a more restricted class of workflows with strictly linear recursion and safe views, we propose a novel view-adaptive dynamic labeling approach.