Towards an Expressive and Scalable Twitter's Users Profiles

Microblogging websites such as Twitter produce tremendous amount of data each second. Consequently, real-time recommendation systems require very efficient algorithm to quickly proceed this massive amount of data. In this paper we propose a scalable and extensible way of building content-based user profiles. Scalability refers to the relative complexity of algorithms involved in building the users profiles with respect to state-of-the-art solutions. Extensibility consider avoiding to recompute the model for newcomers. We present a tractable algorithm to build user profiles out of their tweets. Our model is a graph of terms cooccurency, driven by the fact that user sharing similar interests will share similar terms. We then present how this model can be encoded as a binary footprint, hence boosting comparison of users. We provide an empirical study to measure how the distance between users in the hash space differs from distance between users using standard Information Retrieval techniques. This experiment is based on a Twitter dataset we crawled, and represents 25K users and 1 million tweets. Our approach is driven by real-time analysis requirements and is thus oriented on a trade-off between expressivity and efficiency. Experimental results shows that our approach outperforms vector space model by three orders of magnitude, with a precision of 58%.