Towards a multi-modal perceptual model

The objective evaluation of the audio performance of nonlinear processes such as data reduction has required the development of a new generation of perception-based assessment algorithms. In the future it is expected that a large proportion of communications traffic will be wideband and multimedia. Objective assessment of perceived performance is required for optimisation during design, and for commissioning. In developing objective techniques to assess multimedia systems it is necessary to investigate, and model, the dependencies which exist between audio and visual perception (other senses may follow). This paper summarises the background of current perceptual modelling work and reports early progress towards a multisensory perceptual model. An experimental investigation into combined audio/video perception is described and an algorithmic basis for a multisensory perceptual model proposed. In the future it is predicted that algorithmic models of perception will be exploited for objective assessment of multimedia systems and by the delivery technology for advanced telepresence applications.