Large-scale discovery of visual features for object recognition

To rectify this, we developed Clicktionary, a web-based game used for measuring visual feature importance for recognizing real-world objects. Pairs of participants play together in different roles to identify objects: A “teacher” reveals image regions diagnostic of the object’s category while a “student” tries to recognize the object as quickly as possible. Aggregating game data across players yields importance maps for individual object images, in which each pixel is scored by its contribution to object recognition. We found that these importance maps are consistent across participants and identify object features that are distinct from those used by state-of-the-art deep convolutional networks (DCNs) for object recognition or those predicted by salience maps derived from both human participants and models.