A system for gaze-contingent image analysis and multi-sensorial image display

A novel system for gaze-contingent im- age analysis and multi-sensorial image display is de- scribed. The observer's scanpaths are recorded while viewing and analysing 2-0 or 3-0 (volumetric) images. A region-of-interest (ROI) centred around the current fixation point is simultaneously subjected to real-time image analysis algorithms to compute various image features, e.g. edges, textures (2-0) or surfaces and vol- umetric texture (3-0). This feature information is fed back to the observer using multiple channels, i.e. in visual (replacing the ROI by a visually modified ROI), auditory (generating an auditory display of a computed feature) and tactile (generating a tactile representation of a computed feature) manner. Thus, the observer can use several of his senses to perceive information @om the image which may be otherwise hidden to his eyes, e.g. targets or patterns which are very dificult or im- possible to detect. The human brain then fuses all the information from the multi-sensorial display. The mo- ment the eyes make a saccade to a newjixation loca- tion, the same process is applied to the new ROI centred around it. In this way the observer receives informa- tion from the local real-time image analysis around the point of gaze, hence the term gaze-contingent image analysis. The new system is profiled and several exam- ple applications are discussed.