Incremental Discovery of Order Dependencies on Tuple Insertions

Order dependencies (ODs) are recently proposed to describe a relationship of ordering between lists of attributes. It is typically too costly to design ODs manually, since the number of possible ODs is of a factorial complexity in the number of attributes. To this end, automatic discovery techniques for ODs are developed. In practice, data is frequently updated, especially with tuple insertions. Existing techniques do not lend themselves well to these situations, since it is prohibitively expensive to recompute all ODs from scratch after every update. In this paper, we make a first effort to investigate incremental OD discovery techniques in response to tuple insertions. Given a relation D, a set \(\varSigma \) of valid and minimal ODs on D, and a set \(\triangle D\) of tuple insertions to D, it is to find, changes \(\triangle \varSigma \) to \(\varSigma \) that makes \(\varSigma \oplus \triangle \varSigma \) a set of valid and minimal ODs on \(D + \triangle D\). Note that \(\triangle \varSigma \) contains both new ODs to be added to \(\varSigma \) and outdated ODs to be removed from \(\varSigma \). Specifically, (1) We formalize the incremental OD discovery problem. Although the incremental discovery problem has a same complexity as its batch (non-incremental) counterpart in terms of traditional complexity, we show that it has good data locality. It is linear in the size of \(\triangle D\) to validate on \(D+\triangle D\) any OD \(\varphi \) that is valid on D. (2) We present effective incremental OD discovery techniques, leveraging an intelligent traversal strategy for finding \(\triangle \varSigma \) and chosen indexes to minimize access to D. Our approach computes \(\triangle \varSigma \) based on ODs in \(\varSigma \), and is independent of the size of D. (3) Using real-life data, we experimentally verify that our approach substantially outperforms its batch counterpart by orders of magnitude.