Behavioral informatics from multimodal human interaction cues

shri@sipi.usc.edu Abstract The confluence of advances in sensing, communication and computing technologies is allowing capture and access to data of human interaction and its context, in diverse forms and modalities, in ways that were unimaginable even a few years ago. Importantly, these data afford the analysis and interpretation of multi-modal cues of verbal and non-verbal human behaviour. These signals carry crucial information about not only a person’s intent and identity but also underlying attitudes and emotions. Automatically capturing these cues, although vastly challenging, offers the promise of not just efficient data processing but in tools for discovery that enable hitherto unimagined insights. Recent computational approaches that have leveraged judicious use of both data and domain knowledge have shown promising advances in this regards, for example in deriving rich information about behaviour constructs. This talk will focus on some of the advances (and challenges) in gathering such data and creating algorithms for machine processing of such cues. It will highlight some of our ongoing efforts in Behavioural Signal Processing (BSP)—technology and algorithms for quantitatively and objectively understanding typical, atypical and distressed human behaviour—with a specific focus on communicative, affective and social behaviour. The talk will illustrate Behavioural Informatics applications of these techniques that contribute to quantifying higher-level, often subjectively described, human behaviour in a domain-sensitive fashion using examples from Autism, Couple therapy and Addiction counselling.