Human-Computer Interfaces to Multimedia Content a Review

In the past few decades, there has been an explosive growth of multimedia content available both online and offline, in the personal collections of users. The majority of online content [522] is authored by individuals that are part-time content creators and is very diverse in style, often lacking semantic annotation and user quality ratings, e.g., blogs, home videos, personal images. In addition, multimedia content created by professionals often also lacks semantic and rating information, e.g., podcasts, mp3 music files, TV video clips. As a result, the user is faced with a tremendous amount of raw multimedia data that lacks annotation and cannot be possibly consumed within a lifetime, if accessed sequentially. It is not surprising that designing interfaces for creating, searching, retrieving and -most importantlyconsuming multimedia content is quickly emerging as a top priority in both research and commercial product development. Interface design is interdisciplinary by nature and requires both scientific expertise and creativity. Modality experts, multimedia experts, device experts, human factors experts, software designers, cognitive psychologists and graphic artists have to collaborate to create a successful interface. Creative thinking is needed in order to select the appropriate design among the numerous interface implementations possible for a specific task. An important interface design choice is the selection and mixing of input and output modalities, i.e., channels of communication, between the user and the system. In addition to traditional human-computer interaction (HCI) modalities, such as keyboard and mouse for input, and text and graphics for output, numerous “novel” modalities are available to today’s interface designer, e.g., speech, gestures, haptics. New devices are becoming increasingly mainstream that can support multi-touch input, augmented reality displays, force feedback gloves, virtual keyboards and eye-tracking. The improved device capabilities and available interaction modalities have increased the freedom of choice for the designer, but also the complexity and challenges of interface design. The purpose of this chapter is to familiarize the reader with fundamental concepts, review the state-of-the-art in multimedia interfaces and identify the