MIS: Multimodal Interaction Services in a cloud perspective

The Web is becoming more and more a wide software framework on which each one can compose and use contents, software applications and services. It can offer adequate computational resources to manage the complexity implied by the use of the five senses when involved in human machine interaction. The core of the paper describes how SOA (Service Oriented Architecture) can support multimodal interaction by pushing the I/O processing and reasoning to the cloud, improving naturalness. The benefits of cloud computing for multimodal interaction have been identified by emphasizing the flexibility and scalability of a SOA, and its characteristics to provide a more holistic view of interaction according to the variety of situations and users.

[1]  Alexander H. Waibel,et al.  Multimodal interfaces , 1996, Artificial Intelligence Review.

[2]  Patrizia Grifoni,et al.  A Learning Algorithm for Multimodal Grammar Inference , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Alessio Malizia,et al.  The artificiality of natural user interfaces , 2012, CACM.

[4]  Elisa Bertino,et al.  Improving Scalability of Software Cloud for Composite Web Services , 2009, 2009 IEEE International Conference on Cloud Computing.

[5]  Vladimir Pavlovic,et al.  Integration of audio/visual information for use in human-computer intelligent interaction , 1997, Proceedings of International Conference on Image Processing.

[6]  Deborah A. Dahl.  The W3C multimodal architecture and interfaces standard , 2013, Journal on Multimodal User Interfaces.

[7]  Patrizia Grifoni,et al.  A Hybrid Grammar-Based Approach to Multimodal Languages Specification , 2007, OTM Workshops.

[8]  Patrizia Grifoni,et al.  Multimodal Human Computer Interaction and Pervasive Services , 2009 .

[9]  Patrizia Grifoni.  Multimodal Fission , 2016 .

[10]  Sang-Hyeon Jin,et al.  An Implementation of Multimodal User Interface using Speech, Image and EOG , 2011 .

[11]  Claudia M. Eckert,et al.  Against Ambiguity , 2003, Computer Supported Cooperative Work (CSCW).

[12]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[13]  Sharon L. Oviatt,et al.  Ten myths of multimodal interaction , 1999, Commun. ACM.

[14]  Fernando Ferri,et al.  From Modal to Multimodal Ambiguities: a Classification Approach , 2017, ArXiv.

[15]  Gregory D. Abowd,et al.  Error Correction Techniques for Handwriting, Speech, and Other Ambiguous or Error Prone Systems , 1999 .

[16]  Patrizia Grifoni,et al.  Generating Multimodal Grammars for Multimodal Dialogue Processing , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[17]  References , 1971 .

[18]  Fernando Ferri,et al.  InteSe: An Integrated Model for Resolving Ambiguities in Multimodal Sentences , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[19]  Danilo Avola,et al.  Ambiguities in Sketch-Based Interfaces , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[20]  Xiaolong Yang,et al.  Cloud Computing Architecture Based-On SOA , 2012, 2012 Fifth International Symposium on Computational Intelligence and Design.

[21]  Patrizia Grifoni,et al.  Multimodal Language Specification for Human Adaptive Mechatronics , 2017, ArXiv.

[22]  Allison Woodruff,et al.  Making space for stories: ambiguity in the design of personal communication systems , 2005, CHI.

[23]  Alex Waibel,et al.  A framework and toolkit for the construction of multimodal learning interfaces , 1998 .

[24]  Louise E. Moser,et al.  Managing multiple speech-enabled applications in a mobile handheld device , 2009, Int. J. Pervasive Comput. Commun..

[25]  Won Kim.  Cloud computing architecture , 2013, Int. J. Web Grid Serv..

[26]  Steve Evans,et al.  Succeeding through service innovation : a service perspective for education, research, business and government , 2008 .

[27]  Hajar Mousannif,et al.  The cloud is not 'there', we are the cloud! , 2013, Int. J. Web Grid Serv..

[28]  Fernando Ferri,et al.  An Approach for Managing Ambiguities in Multimodal Interaction , 2007, OTM Workshops.

[29]  Patrizia Grifoni,et al.  KRC: KnowInG crowdsourcing platform supporting creativity and innovation , 2017, ArXiv.

[30]  Danilo Avola,et al.  Classifying and resolving ambiguities in sketch-based interaction , 2010 .

[31]  Barry Sandywell,et al.  The medium is the massage , 2015 .