From a Real Chair to a Negative Chair

In the fall of 1977, I was invited by Raj Reddy to spend a year at the Computer Science Department of Carnegie Mellon University as a visiting scientist from Kyoto University in Japan. My plan for research during the stay was to develop a model-based object recognition program. Upon arrival, I chose an image of an office scene (Fig. 1 ) as an example image; the image was one of a set that Ron Ohlander had used in his research on color image segmentation. The task I set for my program was to recognize the chair in this image. I began to write a "knowledge-based" program for chair recognition by creating a set of heuristic rules for the task. It seemed that in addition to geometric relationships, a good source of constraints was color information, such as "the back and the seat of a chair have the same color". The effort of creating heuristic rules one after another, however, was not a satisfying game, since every time I came up with a reasonably functioning program, I could also find a chair that was an exception to the rules.