Spotting Ontological Lacunae through Spectrum Analysis of Retrieved Documents

The world-wide-web has produced a potentially unlimited flow of information of varying quality, the so-called ”information overload”. Information filters have been designed in hopes of letting only the relevant part be delivered to the user. Such filters require that the user formulate a more or less stable information need. Often, however, an ‘anomalous state of knowledge’ occurs: the user needs information but is not aware of this need. With the rapid growth of information available on-line, it is likely that this need could be filled, if only there were a system to detect it on the user’s behalf. As part of our project involving ontology based proactive information filtering (PROFILE), we designed such a system. It consists of: (1) an ontology in which a user’s more or less stable information need is represented, (2) WordNet, to translate concepts in the ontology into a ‘spectrum’ of keywords, and (3) a component to analyze the spectrum of a set of retrieved documents (e.g. through latent semantic indexing). The two spectra are compared, and if a mismatch is found, the latter spectrum is fed back through WordNet to the ontology component. We show an example of how this mechanism leads to the detection of an anomalous state of knowledge, resulting in a new concept to augment the ontology.