Making computers look the way we look: exploiting visual attention for image understanding

Human Visual attention (HVA) is an important strategy to focus on specific information while observing and understanding visual stimuli. HVA involves making a series of fixations on select locations while performing tasks such as object recognition, scene understanding, etc. We present one of the first works that combines fixation information with automated concept detectors to (i) infer abstract image semantics, and (ii) enhance performance of object detectors. We develop visual attention-based models that sample fixation distributions and fixation transition distributions in regions-of-interest (ROI) to infer abstract semantics such as expressive faces and interactions (such as look, read, etc.). We also exploit eye-gaze information to deduce possible locations and scale of salient concepts and aid state-of-art detectors. A 18% performance increase with over 80% reduction in computational time for a state-of-art object detector [4].