A Web-based system for annotation of dance multimodal recordings by dance practitioners and experts

Recent advances in technologies for capturing, analyzing and visualizing movement can revolutionize the way we create, practice, learn dance, and transmit bodily knowledge. The need for creating meaningful, searchable and re-usable libraries of motion capture and video movement segments can only be fulfilled through the collaboration of both technologists and dance practitioners. Towards this direction, manual annotations of these segments by dance experts can play a four-fold role: a) enrich movement libraries with expert knowledge, b) create "ground-truth" datasets for comparing the results of automated algorithms, c) fertilize a dialogue across dance genres and disciplines on movement analysis and conceptualization, and d) raise questions on the subjectivity and diversity of characterizing movement segments using verbal descriptions. The web-based application presented in this work, is an archival system with, browsing, searching, visualization, personalization and textual annotation functionalities. Its main objective is to provide access to a repository of multimodal dance recordings including motion capture data, video, and audio, with the aim to also support dance education. The tool has been designed and developed within an interdisciplinary project, following a user-centered, iterative design approach involving dance researchers and practitioners of four different dance genres.

[1]  Ann Hutchinson Guest Labanotation: The System of Analyzing and Recording Movement , 1987 .

[2]  Peter Wittenburg,et al.  ELAN: a Professional Framework for Multimodality Research , 2006, LREC.

[3]  Balakrishnan Ramadoss,et al.  SEMI-AUTOMATED ANNOTATION AND RETRIEVAL OF DANCE MEDIA OBJECTS , 2007, Cybern. Syst..

[4]  Michael Neff,et al.  An annotation scheme for conversational gestures: how to economically capture timing and form , 2007, Lang. Resour. Evaluation.

[5]  Yannis E. Ioannidis,et al.  A Labanotation Based Ontology for Representing Dance Movement , 2011, Gesture Workshop.

[6]  Celine Latulipe,et al.  The choreographer's notebook: a video annotation system for dancers and choreographers , 2011, C&C '11.

[7]  Saïd Mahmoudi,et al.  An Ontology for video human movement representation based on Benesh notation , 2012, 2012 International Conference on Multimedia Computing and Systems.

[8]  Diogo Cabral,et al.  Evaluation of a multimodal video annotator for contemporary dance , 2012, AVI.

[9]  Nikolaos Grammalidis,et al.  Capturing the intangible an introduction to the i-Treasures project , 2015, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[10]  Pierfrancesco Bellini,et al.  Modeling performing arts metadata and relationships in content service for institutions , 2014, Multimedia Systems.

[11]  Yannis E. Ioannidis,et al.  From Dance Notation to Conceptual Models: A Multilayer Approach , 2014, MOCO.

[12]  Philippe Pasquier,et al.  Mova: Interactive Movement Analytics Platform , 2014, MOCO.

[13]  Thomas W. Calvert Approaches to the Representation of Human Movement: Notation, Animation and Motion Capture , 2014, Dance Notations and Robot Motion.

[14]  Thecla Schiphorst,et al.  Choreography as Mediated through Compositional Tools for Movement: Constructing A Historical Perspective , 2014, MOCO.

[15]  Florian Jenett Notes on Annotation , 2015 .

[16]  Thecla Schiphorst,et al.  How do experts observe movement? , 2015, MOCO.

[17]  H. Blades Affective Traces in Virtual Spaces , 2015 .

[18]  Frederik Truyen,et al.  What can Europeana bring to Open Education , 2016 .

[19]  Akrivi Katifori,et al.  BalOnSe: Ballet Ontology for Annotating and Searching Video performances , 2016, MOCO.

[20]  Making digital choreographic objects interrelate , 2016, Performing the Digital.

[21]  Rafael Kuffner dos Anjos,et al.  3D Annotation in Contemporary Dance: Enhancing the Creation-Tool Video Annotator , 2016, MOCO.

[22]  Augusto Sarti,et al.  WhoLoDancE: Towards a methodology for selecting Motion Capture Data across different Dance Learning Practice , 2016, MOCO.

[23]  Radoslaw Niewiadomski,et al.  The Dancer in the Eye: Towards a Multi-Layered Computational Framework of Qualities in Movement , 2016, MOCO.

[24]  Augusto Sarti,et al.  Using multi-dimensional correlation for matching and alignment of MoCap and video signals , 2017, 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP).

[25]  Karim Tabia,et al.  Automatic annotation of traditional dance data using motion features , 2017, 2017 International Conference on Digital Arts, Media and Technology (ICDAMT).

[26]  The Duy Bui,et al.  Annotating Movement Phrases in Vietnamese Folk Dance Videos , 2017, IEA/AIE.

[27]  Martina Leeker,et al.  Performing the Digital: Performativity and Performance Studies in Digital Cultures , 2017 .