Friends with Faces: How Social Networks Can Enhance Face Recognition and Vice Versa

The “friendship” relation, a social relation among individuals, is one of the primary relations modeled in some of the world’s largest online social networking sites, such as “FaceBook.” On the other hand, the “co-occurrence” relation, as a relation among faces appearing in pictures, is one that is easily detectable using modern face detection techniques. These two relations, though appearing in different realms (social vs. visual sensory), have a strong correlation: faces that co-occur in photos often belong to individuals who are friends. Using real-world data gathered from “Facebook,” which were gathered as part of the “FaceBots” project, the world’s first physical face-recognizing and conversing robot that can utilize and publish information on “Facebook” was established. We present here methods as well as results for utilizing this correlation in both directions. Both algorithms for utilizing knowledge of the social context for faster and better face recognition are given, as well as algorithms for estimating the friendship network of a number of individuals given photos containing their faces. The results are quite encouraging. In the primary example, doubling of the recognition accuracy as well as a sixfold improvement in speed is demonstrated. Various improvements, interesting statistics, as well as an empirical investigation leading to predictions of scalability to much bigger data sets are discussed.