Using Soft Constraints to Interpret Descriptions of Shapes

The contribution of this paper is to test different models that correctly interpret descriptions of shapes provided by web-users and are able to select which object is being described by them in a language game, a game in which one user describes a selected object of those composing the scene (see figure 1), while another user has to guess which object has been described (see figure 2). The given description needs to be non ambiguous and accurate enough to allow other users to guess the described shape correctly. The descriptions need to be parsed to extract the syntax and the words' classes used, and to interpret the semantics of them. We have modeled the meaning of these descriptions by means of soft constraints, which represent the meaning of words, and an aggregation function which combine the meanings of the words into the meaning of the description. We have proven that if we use the syntax to detect whether a description is simple or complex (those referring to two or more objects) the system can learn better semantic models and improve the results, increasing the precision from 79% to 81%, the recall from 34% to 58% and the f-measure from 48% to 68%.