Extracting the Dynamic Popularity of Concepts from a Corpus of Short-Sentence Documents

The decomposition of information into smaller bunches of data is a commonly observed process on the Web, Twitter and RSS being manifestations of this process. As a consequence, a shift may be observed from an information world in which information comes in large bunches of data, to a world of short-sentence documents. This shrinking of information chunks goes along with an explosion of the number of these chunks. Therefore, information may often be aggregated in corpuses of documents consisting of many short sentences. The identification of important concepts in corpuses of short-sentence documents is a difficult, but necessary, task to understand the whole information. Understanding the dynamics of the popularity of important concepts is necessary to capture the evolution of the corpus in time. In this paper, a method to extract the important concepts from a corpus of short-sentence documents is proposed. A model of the popularity of concepts and its dynamics is proposed, together with an algorithm to analyze the dynamics of important concepts. Finally, the proposed method is validated with an analysis of the titles of the articles published at eleven IFIP Working Conferences on Virtual Enterprises, from PROVE’99 to PROVE’10.