Adaptive processing of structural data: from sequences to trees and beyond

There are several challenging real-world problems, for instance in the fields of medical and technical diagnosis, molecular biology, or document and image processing, where the objects of interest are significantly structured, and their component parts have a continuous nature and are subjected to different types of noise. For a number of practical tasks, the solution can be described by example data, but there is no or only partial and uncertain prior expert knowledge about the relevant structural concepts. Thus, it would be advantageous to have methods and tools to (automatically) infer the solutions, i.e. the desired input-output mapping, from the given example data. In computer science, structural (for example causal, topological, or hierarchical) relationships between parts of a object are commonly represented by symbolic formalisms such as graphs, terms or diagrams. Symbolic machine learning approaches can deal with these representations, but fail if the range of the intended mapping is of continuous nature. On the other hand, existing analog models of computation and learning are tailored to the processing of continuous information. However, these models assume that data are organized according relatively poor structures, by and large, arrays and sequences. This work contributes in bridging this gap. We propose tree-recursive dynamical systems (TRDS), a new class of deterministic state machines that operate in a continuous state space. These machines enable the representation and the inductive inference of structure mappings. The most general admissible domain is characterized by rooted labeled ordered trees (and a certain class of rooted labeled directed ordered acyclic graphs) whose vertices can be labeled by continuous feature vectors. The range of these mappings may either be (a subspace of) the Euclidean vector space or a finite set of categorical values. Adaptivity is incorporated into TRDS by choosing parameterized functions for the state transition map and the output map. Inductive learning tasks, such as the classification or the regression of tree structures, can be re-formulated as the optimization (minimization) of an error criterion. If the given error criterion is stated by a continuously differentiable function, then gradient-based optimization methods are usually taken into account to solve the learning task. We develop and analyze two different algorithms for the calculation of gradient information, backpropagation through structure (BPTS) and tree-recursive gradient computation (TRGC). Both algorithms can be used to calculate the first-order gradient for arbitrary continuous and differentiable criteria where tree structures are embedded via TRDS mappings. This enables attacking inductive learning tasks on structural data by means of TRDS and a variety of