Applying Probabilistic Programming to Affective Computing

Affective Computing is a rapidly growing field spurred by advancements in artificial intelligence, but often, held back by the inability to translate psychological theories of emotion into tractable computational models. To address this, we propose a probabilistic programming approach to affective computing, which models psychological-grounded theories as generative models of emotion, and implements them as stochastic, executable computer programs. We first review probabilistic approaches that integrate reasoning about emotions with reasoning about other latent mental states (e.g., beliefs, desires) in context. Recently-developed probabilistic programming languages offer several key desidarata over previous approaches, such as: (i) flexibility in representing emotions and emotional processes; (ii) modularity and compositionality; (iii) integration with deep learning libraries that facilitate efficient inference and learning from large, naturalistic data; and (iv) ease of adoption. Furthermore, using a probabilistic programming framework allows a standardized platform for theory-building and experimentation: Competing theories (e.g., of appraisal or other emotional processes) can be easily compared via modular substitution of code followed by model comparison. To jumpstart adoption, we illustrate our points with executable code that researchers can easily modify for their own models. We end with a discussion of applications and future directions of the probabilistic programming approach

[1]  Clifford Nass,et al.  Emotion in human-computer interaction , 2002 .

[2]  Joost Broekens,et al.  Formal models of appraisal: Theory, specification, and computational model , 2008, Cognitive Systems Research.

[3]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[4]  Ana Paiva,et al.  FAtiMA Modular: Towards an Agent Architecture with a Generic Appraisal Framework , 2014, Emotion Modeling.

[5]  A. Manstead,et al.  An interpersonal approach to emotion in social decision making: the emotions as social information model , 2010 .

[6]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[7]  Roger Levy,et al.  A Computational Model of Linguistic Humor in Puns , 2015, Cogn. Sci..

[8]  Michael R. Waldmann,et al.  The Oxford handbook of causal reasoning , 2017 .

[9]  A. Gopnik,et al.  Words, thoughts, and theories , 1997 .

[10]  Mika Asaba,et al.  Young children and adults integrate past expectations and current outcomes to reason about others' emotions , 2016, CogSci.

[11]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[12]  Michael C. Frank,et al.  "I won't lie, it wasn't amazing": Modeling polite indirect speech , 2017, CogSci.

[13]  P. Ellsworth,et al.  An appraisal theory of empathy and other vicarious emotional experiences. , 2015, Psychological review.

[14]  Noah D. Goodman,et al.  Nonliteral understanding of number words , 2014, Proceedings of the National Academy of Sciences.

[15]  J. Jacko,et al.  The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications , 2002 .

[16]  Katherine B. Martin,et al.  Facial Action Coding System , 2015 .

[17]  H. Wellman,et al.  Cognitive development: foundational theories of core domains. , 1992, Annual review of psychology.

[18]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[19]  J. Cohn,et al.  Automated Face Analysis for Affective Computing , 2015 .

[20]  Jamil Zaki,et al.  Affective cognition: Exploring lay theories of emotion , 2015, Cognition.

[21]  Hatice Gunes,et al.  Automatic, Dimensional and Continuous Emotion Recognition , 2010, Int. J. Synth. Emot..

[22]  Noah D. Goodman,et al.  Happier Than Thou? A Self-Enhancement Bias in Emotion Attribution , 2018, Emotion.

[23]  Eric Woldridge Promoting Probabilistic Programming System (PPS) Development in Probabilistic Programming for Advancing Machine Learning (PPAML) , 2018 .

[24]  Kostas Karpouzis,et al.  The HUMAINE Database: Addressing the Collection and Annotation of Naturalistic and Induced Emotional Data , 2007, ACII.

[25]  Noah D. Goodman,et al.  Pyro: Deep Universal Probabilistic Programming , 2018, J. Mach. Learn. Res..

[26]  Noah D. Goodman,et al.  Eye-Tracking Causality , 2017, Psychological science.

[27]  J. Stainer,et al.  The Emotions , 1922, Nature.

[28]  Siobhan Chapman Logic and Conversation , 2005 .

[29]  Marc Spraragen,et al.  Computational Models of Emotion and Cognition , 2012 .

[30]  A. Mehrabian Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament , 1996 .

[31]  Klaus R. Scherer,et al.  Facial expressions allow inference of both emotions and their components , 2008 .

[32]  H. Wellman,et al.  Young children's understanding of perception, desire, and emotion. , 2000, Child development.

[33]  Dustin Tran,et al.  Deep Probabilistic Programming , 2017, ICLR.

[34]  P. Petta,et al.  Computational models of emotion , 2010 .

[35]  J. Tsai,et al.  Cultural models of shame and guilt. , 2007 .

[36]  K. Scherer,et al.  Appraisal processes in emotion. , 2003 .

[37]  Jonathan Gratch,et al.  Interpersonal effects of expressed anger and sorrow in morally charged negotiation , 2014, Judgment and Decision Making.

[38]  Joshua B. Tenenbaum,et al.  The Naïve Utility Calculus: Computational Principles Underlying Commonsense Psychology , 2016, Trends in Cognitive Sciences.

[39]  Jamil Zaki,et al.  Computational Models of Emotion Inference in Theory of Mind: A Review and Roadmap , 2018, Top. Cogn. Sci..

[40]  Rebecca Saxe,et al.  Formalizing emotion concepts within a Bayesian model of theory of mind. , 2017, Current opinion in psychology.

[41]  L. Schulz,et al.  Inferring Beliefs and Desires From Emotional Reactions to Anticipated and Observed Events , 2017, Child development.

[42]  Ipke Wachsmuth,et al.  Affect Simulation with Primary and Secondary Emotions , 2008, IVA.

[43]  Carlos Busso,et al.  The ordinal nature of emotions , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII).

[44]  Craig A. Smith,et al.  Appraisal components, core relational themes, and the emotions , 1993 .

[45]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[46]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Jordi Vallverdú,et al.  Handbook of Research on Synthetic Emotions and Sociable Robotics: New Applications in Affective Computing and Artificial Intelligence , 2009 .

[48]  Charles Kemp,et al.  A decision network account of reasoning about other people’s choices , 2015, Cognition.

[49]  D. Over,et al.  Studies in the Way of Words. , 1989 .

[50]  F. Heider The psychology of interpersonal relations , 1958 .

[51]  Jonathan Gratch,et al.  Reading people's minds from emotion expressions in interdependent decision making. , 2014, Journal of personality and social psychology.

[52]  Kristen A. Lindquist,et al.  The brain basis of emotion: A meta-analytic review , 2012, Behavioral and Brain Sciences.

[53]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[54]  Randall W. Hill,et al.  Toward Virtual Humans , 2006, AI Mag..

[55]  Michael C. Frank,et al.  Embedded Implicatures as Pragmatic Inferences under Compositional Lexical Uncertainty , 2015, J. Semant..

[56]  B. Malle How the Mind Explains Behavior: Folk Explanations, Meaning, and Social Interaction , 2004 .

[57]  A. Newell SOAR as a unified theory of cognition: Issues and explanations , 1992, Behavioral and Brain Sciences.

[58]  Jamil Zaki,et al.  Not As Good as You Think? Trait Positive Emotion Is Associated with Increased Self-Reported Empathy but Decreased Empathic Performance , 2014, PloS one.

[59]  Rebecca Saxe,et al.  Neural Representations of Emotion Are Organized around Abstract Event Features , 2015, Current Biology.

[60]  P. Ekman An argument for basic emotions , 1992 .

[61]  E. Spelke,et al.  Preverbal infants identify emotional reactions that are incongruent with goal outcomes , 2014, Cognition.

[62]  Noah D. Goodman The principles and practice of probabilistic programming , 2013, POPL.

[63]  J. Tenenbaum,et al.  Intuitive Theories , 2020, Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship.

[64]  Mike Wu,et al.  Multimodal Generative Models for Scalable Weakly-Supervised Learning , 2018, NeurIPS.

[65]  S. Brison The Intentional Stance , 1989 .

[66]  Noah D. Goodman,et al.  Concepts in a Probabilistic Language of Thought , 2014 .

[67]  Emiliano Lorini,et al.  Computational Modeling of Emotion: Toward Improving the Inter- and Intradisciplinary Exchange , 2013, IEEE Transactions on Affective Computing.

[68]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[69]  David G. Rand,et al.  Signaling Emotion and Reason in Cooperation , 2017, Journal of experimental psychology. General.

[70]  J. Russell A circumplex model of affect. , 1980 .

[71]  Noah D. Goodman,et al.  Knowledge and implicature: Modeling language understanding as social cognition , 2012, CogSci.

[72]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[73]  Anna Wierzbicka,et al.  Emotions Across Languages and Cultures: Diversity and Universals: Defining emotion concepts: discovering “cognitive scenarios” , 1999 .

[74]  Noah D. Goodman,et al.  A pragmatic theory of generic language , 2016, ArXiv.

[75]  S. Laurence,et al.  The Conceptual Mind: New Directions in the Study of Concepts , 2015 .

[76]  Stacy Marsella,et al.  EMA: A process model of appraisal dynamics , 2009, Cognitive Systems Research.

[77]  L. Ross The Intuitive Psychologist And His Shortcomings: Distortions in the Attribution Process1 , 1977 .

[78]  Richard L. Lewis,et al.  A computational unification of cognitive behavior and emotion , 2009, Cognitive Systems Research.

[79]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[80]  Jamil Zaki,et al.  Tracking the Emotional Highs but Missing the Lows: Hypomania Risk is Associated With Positively Biased Empathic Inference , 2015, Cognitive Therapy and Research.

[81]  P. Niedenthal,et al.  Adult attachment and the perception of facial expression of emotion. , 2002, Journal of personality and social psychology.

[82]  Richard W. Robins,et al.  The self-conscious emotions: Theory and research. , 2007 .

[83]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[84]  Michael C. Frank,et al.  Review Pragmatic Language Interpretation as Probabilistic Inference , 2022 .

[85]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[86]  Chris L. Baker,et al.  Rational quantitative attribution of beliefs, desires and percepts in human mentalizing , 2017, Nature Human Behaviour.

[87]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[88]  Noah D. Goodman,et al.  Deep Amortized Inference for Probabilistic Programs , 2016, ArXiv.

[89]  Honglak Lee,et al.  Deep learning for robust feature generation in audiovisual emotion recognition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[90]  Joshua B. Tenenbaum,et al.  Rational Inference of Beliefs and Desires From Emotional Expressions , 2017, Cogn. Sci..