Evaluation of a dynamic classification method for multimodal ambiguities based on Hidden Markov Models

The wide interest in ambiguities is because it represents uncertainty but also a fundamental item of discussion for who is interested in the interpretation of languages also considering that it is functional for communicative purposes. This paper addresses ambiguity issues in terms of identification of the meaningful features of multimodal ambiguities and it evaluates a dynamic HMM-based classification method that is able to classify ambiguities by learning, and progressively adapting the model to the evolution of the interaction, refining the existing classes, or identifying new ones. The comparative evaluation of the considered method of the considered method with other surveyed methods demonstrates an improvement considering the performance evaluation measures.

[1]  Loïc Kessous,et al.  Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis , 2010, Journal on Multimodal User Interfaces.

[2]  Shaogang Gong,et al.  Security and Surveillance , 2011, Visual Analysis of Humans.

[3]  Kostas Karpouzis,et al.  Emerging Artificial Intelligence Applications in Computer Engineering - Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies , 2007, Emerging Artificial Intelligence Applications in Computer Engineering.

[4]  Dimosthenis Kyriazis,et al.  A Comparative Study of Classification Techniques for Managing IoT Devices of Common Specifications , 2017, GECON.

[5]  Eric Horvitz,et al.  A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities , 2005, User Modeling.

[6]  Alaa Tharwat,et al.  Classification assessment methods , 2020, Applied Computing and Informatics.

[7]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[8]  Bashar Nuseibeh,et al.  Analysing anaphoric ambiguity in natural language requirements , 2011, Requirements Engineering.

[9]  Shehzad Khalid,et al.  Comparative study of classification techniques used in skin lesion detection systems , 2014, 17th IEEE International Multi Topic Conference 2014.

[10]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[11]  Maria Chiara Caschera,et al.  Interpretation Methods and Ambiguity Management in Multimodal Systems , 2009 .

[12]  Christine Solnon,et al.  Classification of Images Based on Hidden Markov Models , 2009, 2009 Seventh International Workshop on Content-Based Multimedia Indexing.

[13]  Robert P. Futrelle.  Ambiguity in visual language theory and its role in diagram parsing , 1999, Proceedings 1999 IEEE Symposium on Visual Languages.

[14]  Wen-Jiin Tsai,et al.  Incorporating frequent pattern analysis into multimodal HMM event classification for baseball videos , 2015, Multimedia Tools and Applications.

[15]  Dimitrios Tzovaras,et al.  Multimodal user interface for the communication of the disabled , 2008, Journal on Multimodal User Interfaces.

[16]  David A. Bell,et al.  Learning Bayesian networks from data: An information-theory based approach , 2002, Artif. Intell..

[17]  Allison Woodruff,et al.  Making space for stories: ambiguity in the design of personal communication systems , 2005, CHI.

[18]  Franck Favetta,et al.  About Ambiguities in Visual GIS Query Languages: A Taxonomy and Solutions , 2000, VISUAL.

[19]  Annie I. Antón,et al.  Identifying and classifying ambiguity for regulatory requirements , 2014, 2014 IEEE 22nd International Requirements Engineering Conference (RE).

[20]  Margit Antal,et al.  SPEAKER INDEPENDENT PHONEME CLASSIFICATION IN CONTINUOUS SPEECH , 2004 .

[21]  Mario Malcangi,et al.  Evolving connectionist method for adaptive audiovisual speech recognition , 2017, Evol. Syst..

[22]  Aijun An.  Learning classification rules from data , 2003 .

[23]  Konrad W. Wojciechowski,et al.  Feature Extraction and HMM-Based Classification of Gait Video Sequences for the Purpose of Human Identification , 2013, Vision Based Systemsfor UAV Applications.

[24]  Fernando Ferri,et al.  InteSe: An Integrated Model for Resolving Ambiguities in Multimodal Sentences , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[25]  Peter Sawyer,et al.  The Case for Dumb Requirements Engineering Tools , 2012, REFSQ.

[26]  Plamen P. Angelov,et al.  On line learning fuzzy rule-based system structure from data streams , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[27]  Andrzej Skowron,et al.  Multimodal Classification: Case Studies , 2006, Trans. Rough Sets.

[28]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[29]  Erik Kamsties,et al.  From Contract Drafting to Software Specification: Linguistic Sources of Ambiguity , 2003 .

[30]  E. Jablonka,et al.  The co-evolution of language and emotions , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[31]  Bruce W. Watson,et al.  Incremental construction of minimal acyclic finite state automata , 2000, CL.

[32]  Nicole Novielli,et al.  HMM modeling of user engagement in advice-giving dialogues , 2010, Journal on Multimodal User Interfaces.

[33]  Fernando Ferri,et al.  From Modal to Multimodal Ambiguities: a Classification Approach , 2017, ArXiv.

[34]  Pooja Mittal,et al.  A COMPARATIVE ANALYSIS OF CLASSIFICATION TECHNIQUES ON MEDICAL DATA SETS , 2014 .

[35]  Maria Chiara Caschera.  The Management Of Ambiguities , 2008 .

[36]  Fernando Ferri,et al.  Ambiguity detection in multimodal systems , 2008, AVI '08.

[37]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[38]  T. Benesch,et al.  The Baum--Welch Algorithm for Parameter Estimation of Gaussian Autoregressive Mixture Models , 2001 .

[39]  Andreas Stolcke,et al.  Dialogue act modeling for automatic tagging and recognition of conversational speech , 2000, CL.

[40]  Fernando Ferri,et al.  Multimodal interaction systems: information and time features , 2007, Int. J. Web Grid Serv..

[41]  Manuel Mucientes,et al.  STAC: A web platform for the comparison of algorithms using statistical tests , 2015, 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[42]  Benedikt Gleich,et al.  Ambiguity Detection: Towards a Tool Explaining Ambiguity Sources , 2010, REFSQ.

[43]  M. Mcluhan.  The Medium Is the Massage , 1967 .

[44]  Wei-Yin Loh,et al.  A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms , 2000, Machine Learning.

[45]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[46]  Tatsuya Kawahara,et al.  Multiparty Interaction Understanding Using Smart Multimodal Digital Signage , 2014, IEEE Transactions on Human-Machine Systems.

[47]  Patrizia Grifoni,et al.  Generating Multimodal Grammars for Multimodal Dialogue Processing , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[48]  Fernando Ferri,et al.  DAMA: A Dynamic Classification of Multimodal Ambiguities , 2020, Int. J. Comput. Intell. Syst..

[49]  Patrizia Grifoni,et al.  Towards Evolutionary Multimodal Interaction , 2012, OTM Workshops.

[50]  Fernando Ferri,et al.  An Approach for Managing Ambiguities in Multimodal Interaction , 2007, OTM Workshops.

[51]  G. Vigliocco,et al.  Language as a multimodal phenomenon: implications for language learning, processing and evolution , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[52]  Claudia M. Eckert,et al.  Against Ambiguity , 2003, Computer Supported Cooperative Work (CSCW).

[53]  J. L. Hodges,et al.  Rank Methods for Combination of Independent Experiments in Analysis of Variance , 1962 .

[54]  Cheng Lu,et al.  Classification of summarized videos using hidden markov models on compressed chromaticity signatures , 2001, MULTIMEDIA '01.

[55]  Luisa Mich,et al.  Requirements for tools for ambiguity identification and measurement in natural language requirements specifications , 2008, Requirements Engineering.