Geometric Ordering of Concepts, Logical Disjunction, and Learning by Induction

In many of the abstract geometric models which have been used to represent concepts and their relationships, regions possessing some cohesive property such as convexity or linearity have played a significant role. When the implication or containment relationship is used as an ordering relationship in such models, this gives rise to logical operators for which the disjunction of two concepts is often larger than the set union obtained in Boolean models. This paper describes some of the characteristic properties of such broad non-distributive composition operations and their applications to learning algorithms and classification structures. As an example we describe a quad-tree representation which we have used to provide a structure for indexing objects and composition of regions in a spatial database. The quad-tree combines logical, algebraic and geometric properties in a naturally non-distributive fashion. The lattice of subspaces of a vector space is presented as a special example, which draws a middle-way between ‘noninductive’ Boolean logic and ‘overinductive’ tree-structures. This gives rise to composition operations that are already used as models in physics and cognitive science. Closure conditions in geometric models The hypothesis that concepts can be represented by points and more general regions in spatial models has been used by psychologists and cognitive scientists to simulate human language learning (Landauer & Dumais 1997) and to represent sensory stimuli such as tastes and colors (Gardenfors 2000, §1.5). Of the traditional practical applications of such a spatial approach, the vector space model for information retrieval (Salton & McGill 1983) is notable, and its generalizations such as latent semantic analysis (Landauer & Dumais 1997), in which distributions of word usage learned from corpora become condensed into a lowdimensional representation and used, among other things, for discriminating between different senses of ambiguous words (Schutze 1998). Schutze’s (1998) paper exemplifies some of the opportunities and challenges involved in such a spatial approach — these include learning to represent individual objects as Copyright c © 2004, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: Two non-convex sets (dark gray) and the points added to form their convex closures (light gray) points in a geometric space (in this case, word vectors), combining these points into appropriate sentence or document vectors (in this case, using addition of vectors), and extrapolating from observed points of information to apportion the geometric space into cohesive regions corresponding to recognizable concepts (in this case, using clustering). The last question — how are empirical observations gathered into classes described by the same word or represented by the same concept? — is of traditional importance in philosophy and many related disciplines. The extrapolation from observed data to classifying previously unexperienced situations is implemented in a variety of theoretical models and practical applications, using smoothing and clustering, by exploiting a natural general-to-specific ordering on the space of observations (Mitchell 1997, Ch. 6, 7, 2), and by using similarity or distance measures to gauge the influence exerted by a cluster of observations upon its conceptual hinterland (Gardenfors 2000, Ch. 3,4). Mathematically, such extrapolation techniques are related to closure conditions, a set being closed if it has no tendency to include new members. A traditional example of closure is in the field of topology, which describes a set as being closed if it contains the limit point of every possible sequence of elements. A more easily-grasped example is given by the property of convexity. A set S is said to be convex if for any two pointsA andB in S, the straight lineAB lies entirely within S. The convex closure of S is formed by taking the initial set and all such straight lines, this being the smallest convex set containing S. Figure 1 shows two simple non-convex regions and their convex closures. One of the best developed uses of such closure methods for obtaining stable conceptual representations is in Formal Figure 2: The convex closure of the union of two sets. Concept Analysis, where the closure operation is given by the relationship between the intent and the extent of a concept (Ganter & Wille 1999, §1.1). An important closure operation we will consider later is the linear span of a set of vectors, which can also be thought of as the smallest subspace containing those vectors. Ordering, containment, implication and