Pathfinder: XQuery Off the Relational Shelf

The Pathfinder project makes inventive use of relational database technology—originally developed to process data of strictly tabular shape—to construct efficient database-supported XML and XQuery processors. Pathfinder targets database engines that implement a set-oriented mode of query execution: many off-the-shelf traditional database systems make for suitable XQuery runtime environments, but a number of off-beat storage back-ends fit that bill as well. While Pathfinder has been developed with a close eye on the XQuery semantics, some of the techniques that we will review here will be generally useful to evaluate XQuery-style iterative languages on database back-ends. 1 The Rectangularization of XQuery: Purely Relational XML Processing If you zoom back in time to dig for the semantic roots of XQuery [5], you will find that the language’s core construct, the for–let–where–order by–return (FLWOR) block is one particular incarnation of a very general idea: the comprehension [26]. Many language-related concepts may be uniformly understood in comprehension form, but comprehensions provide a particularly concise and elegant way to express iteration over collections of objects—in the case of XQuery: finite, ordered sequences of XML nodes and atomic values (or items) [1]. Any program or query expressed in comprehension form is subject to a number of useful equivalencepreserving rewriting rules (the monad laws) and so is XQuery’s FLWOR block. Once you look closely, a wide range of seemingly XQuery-specific optimizations realized by compilers and interpreters today, e.g., for loop fusion or unnesting, in fact put the monad laws to work. The family of programming and query languages whose semantic core may be cast in comprehension form is large. Among its members, specifically, is SQL, the relational database language. This observation sparked a whole line of work that we will review in the following pages: Exploit the common semantic ground of XQuery and SQL and try to turn relational database systems (i.e., processors for strictly tabular, or rectangular, data) into efficient and scalable XQuery processors. XQuery processors of this type should be able to benefit from the 30+ years of research and engineering experience that shaped relational database technology. This is, in fact, what we repeatedly observed in the course of Copyright 2008 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering