Reactive Agents Learn to Add Epistemic Structures to the World

Reactive Agents Learn to Add Epistemic Structures to the World Sanjay Chandrasekharan (schandra@sce.carleton.ca) Terry Stewart (tcstewar@connect.carleton.ca) Institute of Cognitive Science, Carleton University, Ottawa, Canada, K1S 5B6 is almost endless. Humans also add structures to the world to reduce cognitive complexity for artifacts. Examples include bar codes (makes check-out machines’ decisions easier), content-based tags in web pages (makes Web agents’ decisions easier), sensors on roads (helps the traffic light program’s decision-making), etc. The pervasiveness of such structures across species indicates that adding structure to the world is a fundamental cognitive strategy (Kirsh, 1996). Note that these structures predominantly serve a task-smoothening function – they make tasks easier for agents. Some of these structures have referential properties, but they do not exist for the purpose of reference. From here onwards, we will term such stable structures that provide “cognitive congeniality” (Kirsh, 1996), epistemic structures. The term is derived from a distinction between epistemic and pragmatic action made by Kirsh (1994). How do organisms generate and use such structures? Can this generation of structures be captured computationally? These are the questions we address in this paper. Abstract We provide a computationally tractable model of how organisms can learn to add structures to the world to reduce cognitive complexity. This model is then implemented using two techniques: first using a genetic algorithm, and then using the Q-learning algorithm. The results clearly show that organisms with only reactive behavior can learn to systematically add structures to the world to reduce their cognitive load. We show that such learning can happen in both evolutionary time and within an agent’s lifetime. An extension of this model (currently being implemented) is then illustrated, where organisms with just reactive behavior learn to systematically generate and use internal structures akin to representations. Many organisms generate stable structures in the world to reduce cognitive complexity (minimize search or inference), for themselves, for others, or both. Wood mice (Apodemus sylvaticus) distribute small objects, such as leaves or twigs, as points of reference while foraging. They do this even under laboratory conditions, using plastic discs. Such ‘way- marking’ diminishes the likelihood of losing interesting locations during foraging (Stopka & MacDonald, 2003). Red foxes (Vulpes vulpes) use urine to mark food caches they have emptied. This marking acts as a memory aid and helps them avoid unnecessary search (Henry, 1977, reported in Stopka & MacDonald, 2003). The male bower bird builds colorful bowers (nest-like structures), which are used by females to make mating decisions (Zahavi & Zahavi, 1997). Ants drop pheromones to trace a path to a food source. Many mammals mark their territories. At the most basic level, cells in the immune system use antibodies that bind to attacking microbes, thereby ‘marking’ them. Macrophages use this ‘marking’ to identify and destroy invading microbes. Bacterial colonies use a strategy called ‘quorum sensing’ to know that they have reached critical mass (to attack, to emit light, etc.). This strategy involves individual bacteria secreting molecules known as auto-inducers into the environment. The auto- inducers accumulate in the environment, and when it reaches a threshold, the colony moves into action (Silberman, 2003). Such ‘doping’ of the world is commonly seen in lower animals. Most large animals (large body & brain size) do not exploit this strategy. Humans, however, do so to a tremendous degree. Markers, color-codes, page numbers, credit-ratings, badges, shelf-talkers, speed bugs, road signs, post-it notes, the list of epistemic structures used by humans A Taxonomy and a Property Most of the literature on epistemic structures is by David Kirsh, and from the field of Distributed Cognition in general. Kirsh’s work explores the structural and computational properties of such structures, and how they function. We are interested in the other half of the problem, i.e., how such structures are generated and used. We use Kirsh’s model to develop a situated cognition model of how such structures are generated. We then outline two simulations we implemented to test this model. An extension of this model (currently in progress) is then described. Epistemic structures can be classified into three types, based on whom they are generated for. (examples of each in brackets). Structures generated for oneself (Cache marking, bookmarks) Structures generated for oneself and others (Pheromones, color codes) Structures generated exclusively for others (Warning smells, badges) A central feature of such structures is their task-specificity (more broadly, function/goal-orientedness). To illustrate this concept, consider the following example. Think of a