Leveraging Knowledge-based Inference for Material Classification

Material classification is one of the fundamental problems for multimedia content analysis, computer vision and graphics. Existing efforts mostly focus on extracting representative visual features and training a classifier to recognize unknown materials. Compared with human visual recognition, automatic recognition cannot leverage common sense knowledge regarding material categories and contextual information such as object and scene. In this paper, we propose to first extract such knowledge on material, object and scene from heterogeneous sources, i.e. a public data set of 100 million Flickr images [13] and Bing search results. To improve the material classification task, the knowledge information is further exploited in a probabilistic inference framework. Our method is evaluated on OpenSurfaces [10], the largest public material data set which contains both visual features of physical properties as well as image context information. The quantitative evaluation demonstrates the superior performance of our proposed method.