Efficient data structures for internal queries in texts

This thesis is devoted to internal queries in texts, which ask to solve classic text-processing problems for substrings of a given text. More precisely, the task is to preprocess a static string T of length n (called the text) and construct a data structure answering certain questions about the substrings of T . The substrings involved in each query are specified in constant space by their occurrences in T , called fragments of T , identified by the start and the end positions. Components for internal queries often become parts of more complex data structures, and they are used in many algorithms for text processing. Longest Common Extension Queries, asking for the length of the longest common prefix of two substrings of the text T , are by far the most popular internal queries. They are used for checking if two fragments match (represent the same string) and for lexicographic comparison of substrings. Due to an optimal solution in the standard setting of texts over polynomially-bounded integer alphabets, with O(1)-time queries, O(n) size, and O(n) construction time, they have found numerous applications across stringology. In this dissertation, we provide the first optimal data structure for smaller alphabets of size σ n: it handles queries in O(1) time, takes O(n/ logσ n) space, and admits an O(n/ logσ n)-time construction (from the packed representation of T with Θ(logσ n) characters in each machine word). We then go back to alphabets of size σ polynomial in n and focus on more complex internal queries. Our first data structure supports Internal Pattern Matching Queries, which ask for the occurrences of one substring x within another substring y. After O(n)-time preprocessing of the text T , it answers these queries in time proportional to the quotient |y|/|x| of substrings’ lengths, which is required due to the information content of the output. We also use this data structure for Period Queries, asking for the periods of a given substring. Here, our logarithmic query time is also optimal by a similar information-theoretic argument. Further data structures are designed for Minimal Suffix and Minimal Rotation Queries, asking to compute the lexicographically smallest non-empty suffix and cyclic rotation of a given substring, respectively. They are answered in O(1) time after O(n)-time preprocessing. We also consider a more general problem of simulating the suffix array of a given substring (Substring Suffix Selection Queries, asking for the kth lexicographically smallest suffix of a substring) and its inverse suffix array (Substring Suffix Rank Queries, asking for the lexicographic rank of a substring’s suffix). Our data structure supports these queries in O(logn) time, takes O(n) space, and can be constructed in O(n √ logn) time. The tools developed in this dissertation additionally yield improved results for several kinds of Substring Compression Queries, which ask for the compressed representation of a given substring obtained using a specific method; we consider schemes based on the Lempel–Ziv parsing and the Burrows–Wheeler transform. Our results combine text-processing tools with combinatorics on words and state-of-the-art general-purpose data structures. The key technical contribution is a novel locally consistent symmetry-breaking scheme, formalized in terms of synchronizing functions, which is central to our solutions for Longest Common Extension Queries and Internal Pattern Matching Queries. 2012 ACM Subject Classification: Theory of computation→ Pattern matching