A Multimodal Workflow for Modeling Personality and Emotions to Enable User Profiling and Personalisation

The Personality Emotion Model (PEM) is a workflow for generating quantifiable and bi-directional mappings between 15 personality traits and the basic emotions. PEM utilises Affective computing methodology to map this relationship across the modalities of self-report, facial expressions, semantic analysis, and affective prosody. The workflow is an end-to-end solution integrating data collection, feature extraction, data analysis, and result generation. PEM results in a real-time model that provides a high-resolution correlated mapping between personality traits and the basic emotions. The robustness of PEM’s model is supported by the workflow’s ability to conduct meta-analytical and multimodal analysis; each state-to-trait mapping is dynamically updated in terms of its magnitude, direction, and statistical significance as data is processed. PEM provides a methodology that can contribute to long-standing research questions in the fields of Psychology and Affective computing. These research questions include (i) quantifying the emotive nature of personality, (ii) minimising the effects of context variance in basic emotions research, and (iii) investigating the role of emotion sequencing effects in relation to individual differences. PEM’s methodology enables direct applications in any domain that requires the provision of individualised and personalised services (e.g. advertising, clinical care, research).