LDA Topic Modeling Based Dataset Dependency Matrix Prediction

Classification of text based datasets has many applications in the field of Computer Science. Some of the key application areas include scientific article recommendation, news article tagging, multimedia content search assistance, etc. We are interested in the problem of data placement of text based datasets in a distributed storage system. Distributed data placement entails placing related data together at a local site. Thus, classifying related data from the unrelated ones is a pre-requisite for any such data placement system. Classification of datasets can be accomplished using information provided to the system about the relatedness of a pair of dataset. However, when such information are not available, the relatedness of pairs of dataset need to be inferred from content of the dataset itself. In literature, topic modeling has been used to find similarity between text documents and in classifying these documents according to the similarity between them. We intend to develop a novel classification system of text based datasets using topic modeling, as a precursor to a data placement scheme to be developed for distributed data storage system.