Expanding the breadth and detail of object recognition

Object recognition systems today see the world as a collection of object categories, each existing as a separate isolated entity. They exist in a closed world, never expecting to come across a new and unfamiliar object. This bleak view of the world leads to brittle systems that are limited to recognizing a few predefined categories such as airplanes, bicycles, and potted plants. Instead, we adopt a broader view of recognition and try to move toward recognition systems that can survive in an open world. Here they might encounter any object, even ones that humans have not yet named. Toward this end, we want to say more than just “here is an object”, but instead give detailed insight into the state of this object, even if it cannot be categorized. By considering tasks beyond categorization, which partitions objects into disjoint sets, we can instead relate objects to one another and consider ways to generalize to new objects in our open world. We present how to relate novel objects to known training examples by capturing the a variety of shared commonalities, such as named attributes, generic low-level object properties, and shared appearance and spatial layout. For each of these new learning tasks, we provide the datasets necessary to explore these exciting new problems. Ultimately, this leads to methods that can give rich discriptions of any object, predict what is unusual about known objects, segment and localize objects from broad domains while giving detailed localized predictions of their parts, and quickly learning new categories from few, or even no visual examples.