Tracing Lineage of Array Data

Arrays are a common and important class of data in many applications. Arrays can model data such as digital images, digital video, scientific and experimental data, matrices, and finite element grids. Although array manipulations are diverse and domain-specific, they often exhibit structural regularities. This paper describes an algorithm called sub-pushdown to trace data lineage in such array computations. Lineage tracing is a type of data-flow analysis that relates parts of a result array to those parts of the argument (base) arrays that have bearings on the result array parts. Sub-pushdown can be used to trace data lineage in array-manipulating computations expressed in the Array Manipulation Language (AML) that was introduced previously. Sub-pushdown has several useful features. First, the lineage computation is expressed as an AML query. Second, it is not necessary to evaluate the AML lineage query to compute the array data lineage. Third, sub-pushdown never gives false-negative answers. Sub-pushdown has been implemented as part of the ArrayDB prototype array database system that we have built.

[1]  Kenneth Salem,et al.  Query processing techniques for arrays , 1999, SIGMOD '99.

[2]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[3]  Laks V. S. Lakshmanan,et al.  A Foundation for Multi-dimensional Databases , 1997, VLDB.

[4]  Sunita Sarawagi,et al.  Modeling multidimensional databases , 1997, Proceedings 13th International Conference on Data Engineering.

[5]  Michael Stonebraker,et al.  Supporting fine-grained data lineage in a database visualization environment , 1997, Proceedings 13th International Conference on Data Engineering.

[6]  Kenneth Salem,et al.  A Language for Manipulating Arrays , 1997, VLDB.

[7]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1992 .

[8]  Peter Baumann,et al.  Storage of multidimensional arrays based on arbitrary tiling , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[9]  Leonidas J. Guibas,et al.  Compilation and delayed evaluation in APL , 1978, POPL.

[10]  Michael Stonebraker,et al.  Efficient organization of large multidimensional arrays , 1994, Proceedings of 1994 IEEE 10th International Conference on Data Engineering.

[11]  Jennifer Widom,et al.  Practical lineage tracing in data warehouses , 2000, Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073).

[12]  V. Vianu,et al.  Edinburgh Why and Where: A Characterization of Data Provenance , 2017 .

[13]  Arunprasad P. Marathe.  Tracing lineage of array data , 2001, Proceedings Thirteenth International Conference on Scientific and Statistical Database Management. SSDBM 2001.

[14]  Jennifer Widom,et al.  Database System Implementation , 2000 .