Challenges and Opportunities in Building Socially Intelligent Machines

AT the recent ACM Multimedia conference in Florence, Ramesh Jain boldly and succinctly stated, “Content without context is meaningless.” He presented a paper on the topic at the conference’s Brave New Ideas session, and challenged multimedia researchers to “stop ignoring the elephant in the room” (context) and to recognize that by reducing problems to being entirely content-focused, they are doing both their research and their community a disservice [1]. Further, he writes that many intractable problems in multimedia analysis have been substantially aided by taking context into account, such as object recognition, image search, and photo management. Multimedia is not the only field that can benefit from this mindset. Researchers working in social computing, social signal processing, human-machine interaction, robotics, computer vision, computer security, or any other field concerned with the automatic analysis of (and response to) human behavior may be greatly aided by understanding the role context plays. Indeed, some might say that understanding social context is one of the grand challenges of these fields. But how does context affect social behavior? And, further, as researchers how can we build autonomous systems that take advantage this contextual information? This paper will broadly introduce social context, and discuss some of the challenges involved in building real-time systems that can process and respond to this contextual information. By clearly articulating these challenges, our hope is that researchers will be better equipped to confront them in their work, and can transform them into opportunities.