FARMI: A FrAmework for Recording Multi-Modal Interactions

In this paper we present (1) a processing architecture used to collect multi-modal sensor data, both for corpora collection and real-time processing, (2) an open-source implementation thereof and ( ...

[1]  Sean Andrist,et al.  Rapid development of multimodal interactive systems: a demonstration of platform for situated intelligence , 2017, ICMI.

[2]  Gabriel Skantze,et al.  Furhat: A Back-Projected Human-Like Robot Head for Multiparty Human-Machine Interaction , 2011, COST 2102 Training School.

[3]  H. Gardner,et al.  Frames of Mind: The Theory of Multiple Intelligences , 1983 .

[4]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[5]  Eric Horvitz,et al.  Dialog in the open world: platform and applications , 2009, ICMI-MLMI '09.

[6]  David Schlangen,et al.  Temporal alignment using the incremental unit framework , 2017, ICMI.

[7]  Matthew Turk,et al.  Multimodal interaction: A review , 2014, Pattern Recognit. Lett..

[8]  David Schlangen,et al.  MINT.tools: tools and adaptors supporting acquisition, annotation and analysis of multimodal corpora , 2013, INTERSPEECH.

[9]  Peter Robinson,et al.  OpenFace: An open source facial behavior analysis toolkit , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Ana Paiva,et al.  Feeling and Reasoning: A Computational Model for Emotional Characters , 2005, EPIA.

[11]  Tobias Baur,et al.  The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time , 2013, ACM Multimedia.

[12]  Anne Loomis Thompson,et al.  A framework for multimodal data collection, visualization, annotation and learning , 2013, ICMI '13.

[13]  Gabriel Skantze,et al.  IrisTK: a statechart-based toolkit for multi-party face-to-face interaction , 2012, ICMI '12.

[14]  Giampiero Salvi,et al.  A gaze-based method for relating group involvement to individual engagement in multimodal multiparty dialogue , 2013, ICMI '13.

[15]  Ana Paiva,et al.  FAtiMA Modular: Towards an Agent Architecture with a Generic Appraisal Framework , 2014, Emotion Modeling.