Gestalt-based Contour Weights Improve Scene Categorization by CNNs

Humans can accurately recognize natural scenes from line drawings, consisting solely of contour-based shape cues. Deep learning strategies for this complex task, however, have thus far been applied directly to photographs, exploiting all the cues available in colour images at the pixel level. Here we report the results of fine tuning off-the-shelf pre-trained Convolutional Neural Networks (CNNs) to perform scene classification given only contour information as input. To do so we exploit the Iverson-Zucker logical/linear framework to obtain line drawings from popular scene categorization databases, including an artist’s scene database and MIT67. We demonstrate a high level of performance despite the absence of colour, texture and shading information. We also show that the inclusion of medial-axis based contour salience weights leads to a further boost, adding useful information that does not appear to be exploited when CNNs are trained to use contours alone.