Evaluation of multimodal fusion methods using integration patterns modeling

Multimodal interaction has been proven as a key to more natural and flexible human-computer interaction. Research of multimodal fusion algorithms still remains highly dependent on usability testing as the only option for their proper evaluation. Usability testing is, however, a very demanding task with many drawbacks hindering further research of advanced methods and more robust algorithms in the domain. In the present research, a novel approach of automated evaluation and testing of decision-level fusion methods is introduced. The key feature behind the approach is modeling of users' integration patterns derived from a series of existing empirical studies. The proposed solution provides researchers with ability to evaluate their fusion algorithms automatically, and more importantly, without the need of expensive usability testing. Moreover, the method delivers other considerable advantages as a reproducibility of tests, high-level parameterization, and possibility to design a large scale of user models.