Use of Content Analysis Tools for Visual Interaction Design

Automatic media content analysis in multimedia is a very promising field of research bringing in various possibilities for enhancing visual informatics. By computationally analysing the quantitative data contained in text, audio, image and video media, more semantically meaningful and useful information on the media contents can be derived, extracted and visualised, informing human users those facts and patterns initially hidden in the bit streams of data. Insights into how to transform the emerging technological possibilities from these media analysis tools into usable visual interfaces to help people see visual information in novel ways will be an important contribution to visual informatics. In this paper, we outline some of the more promising content analysis techniques currently being researched in multimedia and computer vision and discuss how these could be used to develop visually-oriented end-user interfaces that support searching, browsing and summarization of the media contents in various usage contexts. We illustrate this with a few example applications that we have developed over the years, all of which designed in such a way as to take advantage of the automatic content analysis and to discover and create novel usage scenarios of consuming visually-oriented media contents.