Debugging Scientific Workflows with Provenance: Achievements and Lessons Learned

1Scientific Workflow Management Systems manage experiments in large-scale and deliver provenance data. Provenance data represents the workflow execution behavior, allowing for tracing the data-flow generation. When provenance is extended with performance execution data, it becomes an important asset to identify and analyze errors that occurred during the workflow execution (i.e. debugging). Debugging is essential for workflows that execute in parallel in large-scale distributed environments since the incidence of errors in this type of execution is high and difficult to track. By debugging at runtime, scientists can identify errors and take the necessary actions, while the workflow is still running. We present provenance based debugging, in real use cases, running in parallel, with virtual machines in clouds. In these experiences scientists use provenance data to query domain and execution data to detect errors, especially when the execution concludes without any system error message.