Creative Applications of Human Behavior Understanding

The role of computer science in the creative industries is becoming recognised as an important way to bring forward progress in both domains. There is a need for smarter applications that sense and adapt to their users in arts, creativity, entertainment and edutainment domains. Understanding human behavior in this area is challenging because it forces practitioners to engage in both creative, and perhaps counterintuitively, analytic processes of understanding how people engage with creative phenomena. The systems constructed for this purpose promise to enhance and redefine the scope of creative industries significantly. This paper discusses scientific and technological factors that make this a challenging topic to address, provides a brief survey of related work in this area, and identifies active topics of research. Since arts, creativity, entertainment and edutainment all contribute to significant social and societal benefits, it is vital to tackle the problem of measuring and evaluating the success of automatic behavior analysis solutions as a social and human phenomenon.

[1]  Maurizio Mancini,et al.  Sync'n'Move: Social Interaction Based on Music and Gesture , 2009, UCMedia.

[2]  Matthew Turk,et al.  Multimodal interaction: A review , 2014, Pattern Recognit. Lett..

[3]  Kristina Höök,et al.  Sense and sensibility: evaluation and interactive art , 2003, CHI '03.

[4]  Yiannis Demiris,et al.  Creative Dance: An Approach for Social Interaction between Robots and Children , 2013, HBU.

[5]  L. Umansky.  The Data-Driven Life , 2012 .

[6]  Patrick G. T. Healey,et al.  Performing places , 2011, C&C '11.

[7]  Christian Wolf,et al.  Social Behavior Modeling Based on Incremental Discrete Hidden Markov Models , 2013, HBU.

[8]  Alan F. Smeaton,et al.  Biometric Responses to Music-Rich Segments in Films: The CDVPlex , 2009, 2009 Seventh International Workshop on Content-Based Multimedia Indexing.

[9]  Kim Hartmann,et al.  Human Behaviour in HCI: Complex Emotion Detection through Sparse Speech Features , 2013, HBU.

[10]  Albert Ali Salah,et al.  A Robotic Fitness Coach for the Elderly , 2013, AmI.

[11]  Francesco Setti,et al.  ATTENTO: ATTENTion Observed for Automated Spectator Crowd Analysis , 2013, HBU.

[12]  Kelsey Cohen,et al.  Project for Public Spaces , 2011 .

[13]  Robby van Delden,et al.  Socially Aware Interactive Playgrounds , 2013, IEEE Pervasive Computing.

[14]  Rob Tieben,et al.  Human Behavior Analysis in Ambient Gaming and Playful Interaction , 2011, Computer Analysis of Human Behavior.

[15]  Hatice Gunes,et al.  Human Behavior Understanding , 2016, Lecture Notes in Computer Science.

[16]  Radoslaw Niewiadomski,et al.  MMLI: Multimodal Multiperson Corpus of Laughter in Interaction , 2013, HBU.

[17]  Peter Robinson,et al.  Real-Time Recognition of Affective States from Nonverbal Features of Speech and Its Application for Public Speaking Skill Analysis , 2011, IEEE Transactions on Affective Computing.

[18]  Lale Akarun,et al.  Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests , 2012, ECCV.

[19]  Carlos Medrano,et al.  Transfer Learning of Human Poses for Action Recognition , 2013, HBU.

[20]  Andrea Passerini,et al.  A Fully Unsupervised Approach to Activity Discovery , 2013, HBU.

[21]  Pierre Dillenbourg,et al.  An Interactive Table for Supporting Participation Balance in Face-to-Face Collaborative Learning , 2010, IEEE Transactions on Learning Technologies.

[22]  Daniel Thalmann,et al.  Real-Time Comprehensive Sociometrics for Two-Person Dialogs , 2013, HBU.

[23]  Alan Hanjalic,et al.  Adaptive extraction of highlights from a sport video based on excitement modeling , 2005, IEEE Transactions on Multimedia.

[24]  Nicu Sebe,et al.  Human Behavior Understanding, First International Workshop, HBU 2010, Istanbul, Turkey, August 22, 2010. Proceedings , 2010, International Workshop on Human Behavior Unterstanding.

[25]  A. Ohman,et al.  Emotion drives attention: detecting the snake in the grass. , 2001, Journal of experimental psychology. General.

[26]  Allison Druin,et al.  Robots for Kids: Exploring New Technologies for Learning , 2000 .

[27]  Hilary Johnson,et al.  Proceedings of the seventh ACM conference on Creativity and cognition , 2009 .

[28]  K. Dautenhahn,et al.  Towards interactive robots in autism therapy: background, motivation and challenges , 2004 .

[29]  Albert Ali Salah,et al.  Computer Analysis of Human Behavior , 2011 .

[30]  Isabella Poggi,et al.  Types of Help in the Teacher's Multimodal Behavior , 2010, HBU.

[31]  Fabiane Barreto Vavassori Benitti,et al.  Exploring the educational potential of robotics in schools: A systematic review , 2012, Comput. Educ..

[32]  Petros Daras,et al.  User Centric Media , 2011 .

[33]  Dana Kulic,et al.  Gender Differences in the Perception of Affective Movements , 2012, HBU.

[34]  Marc Leman,et al.  A methodological framework for the development and evaluation of user-centered art installations , 2011 .

[35]  D. Matsumoto,et al.  The Thrill of Victory and the Agony of Defeat: Spontaneous Expressions of Medal Winners of the 2004 Athens Olympic Games the Link between Emotionally Evocative Contexts and Facial Expressions of Emotion , 2022 .

[36]  D. Meyer,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S6 References Evidence for a Collective Intelligence Factor in the Performance of Human Groups , 2022 .

[37]  Mladjan Jovanovic,et al.  Motivation and Multimodal Interaction in Model-Driven Educational Game Design , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[38]  Catherine Pelachaud,et al.  Head, Shoulders and Hips Behaviors during Turning , 2013, HBU.

[39]  M.J. Mataric,et al.  Hands-off assistive robotics for post-stroke arm rehabilitation , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[40]  Hassen Drira,et al.  A Dense Deformation Field for Facial Expression Analysis in Dynamic Sequences of 3D Scans , 2013, HBU.

[41]  Takashi Ozeki,et al.  Extraction of relations between behaviors by lecturer and students in lectures , 2011, Face and Gesture 2011.

[42]  E. Coiera.  Interaction Design , 2002 .

[43]  Mohan S. Kankanhalli,et al.  VIP: A Unifying Framework for Computational Eye-Gaze Research , 2013, HBU.

[44]  Matthieu Guillaumin,et al.  Segmentation Propagation in ImageNet , 2012, ECCV.

[45]  Y. Rogers,et al.  Interaction Design , 2002 .

[46]  Jean-Philippe Thiran,et al.  Multiple Local Curvature Gabor Binary Patterns for Facial Action Recognition , 2013, HBU.

[47]  W. Whyte.  The social life of small urban spaces , 1980 .

[48]  Hatice Gunes,et al.  Measuring Affect for the Study and Enhancement of Co-present Creative Collaboration , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[49]  Hatice Gunes,et al.  Human Nonverbal Behaviour Understanding in the Wild for New Media Art , 2013, HBU.

[50]  Hayes Raffle,et al.  Social immersive media: pursuing best practices for multi-user interactive camera/projector exhibits , 2009, CHI.

[51]  Xenofon Fafoutis,et al.  A Behavioral Study on the Effects of Rock Music on Auditory Attention , 2013, HBU.

[52]  Pierre Dillenbourg,et al.  Interactive tabletops in education , 2011, Int. J. Comput. Support. Collab. Learn..

[53]  Nicu Sebe,et al.  Challenges of Human Behavior Understanding , 2010, HBU.

[54]  Hichem Sahli,et al.  Towards Real-Time Continuous Emotion Recognition from Body Movements , 2013, HBU.

[55]  Antonio Camurri,et al.  A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media , 2010, IEEE Transactions on Multimedia.

[56]  Alexandros Potamianos,et al.  Towards adapting fantasy, curiosity and challenge in multimodal dialogue systems for preschoolers , 2009, ICMI-MLMI '09.

[57]  Tobias Baur,et al.  NovA: Automated Analysis of Nonverbal Signals in Social Interactions , 2013, HBU.

[58]  Dimitrios Makris,et al.  Dynamic Feature Selection for Online Action Recognition , 2013, HBU.

[59]  Pirkko Oittinen,et al.  Stylistic Features for Affect-Based Movie Recommendations , 2013, HBU.

[60]  Andrea Schenone,et al.  Multi-scale entropy analysis of dominance in social creative activities , 2010, ACM Multimedia.

[61]  Tomoo Inoue,et al.  Designing a conversation support system in dining together based on the investigation of actual party , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[62]  P. Beesley.  Hylozoic Soil , 2009 .

[63]  Kosuke Sato,et al.  Table-Top Interface Using Fingernail Images and Real Object Recognition , 2011, AmI.

[64]  Bogdan Raducanu,et al.  Efficient Graph Construction for Label Propagation Based Multi-observation Face Recognition , 2013, HBU.

[65]  Rita Cucchiara,et al.  Human Behavior Understanding with Wide Area Sensing Floors , 2013, HBU.

[66]  T. Baumgartner,et al.  From emotion perception to emotion experience: emotions evoked by pictures and classical music. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[67]  James A. Hendler,et al.  Robots: Exploring new technologies for learning for kids , 2000 .