Towards Unifying Perception and Cognition : The Ubiquity of Trees

Is there a single mechanism that underlies all perceptual and cognitive processing? This paper aims to solve a small part of Newell's challenge (A. Newell 1990, Unified Theories of Cognition , Harvard University Press) and proposes a model that unifies three different modalities: language, music and problem-solving. In doing so, we will focus on tree structures. Trees are ubiquitous in modeling high-level perception and cognition and have been used to represent grouping structures in linguistic, musical and visual perception and deductive structures in reasoning, learning and problem solving. We will show that an instantiation of the Data-Oriented Parsing (DOP) framework can accurately predict the correct tree structure for linguistic utterances, musical pieces and physics problems. The key idea of the DOP framework is that new input is analyzed by combining subtrees from a representative corpus of previous trees. While the labeling of the trees and the details of the combination operation may differ across the modalities, we argue that there is one model for predicting the tree that humans come up with. We report on experiments with manually annotated corpora for the three modalities, showing that the best performing model is the one which takes into account subtrees of arbitrary size and which selects the most probable tree from among the shortest derivations of an input.