Publish/Subscribe on the Web at Extreme Speed

This demonstration presents Le Subscribe an event notification system for the Web. It is widely accepted that the majority of human information will be on the Web in ten years. As pointed out in [6], besides systems for searching, querying and retrieving information from the Web, there is a need for systems being able to capture the dynamic aspect of the web information by notifying users of interesting events. This functionality is crucial for web users (or applications) who want to exploit highly dynamic web information such as stock markets updates or auctions. A tool that implements this functionality must be scalable and efficient. Indeed, it should manage millions of user demands for notifications (i.e. subscriptions); It should handle high rates of events (several millions per day) and notify the interested users in a short delay. In addition, it should provide a simple and expressive subscription interface and efficiently cope with high volatility of web user demands (new subscriptions, new users and cancellations). Finally, it should facilitate integration of similar kinds of information issued by different publishers (e.g. new auctions coming from distinct auction sites). The classical approach for query subscription is a mediator system where queries are periodically evaluated against static data. This static approach does not scale for high rate of events and a large number of volatile subscriptions, since it requires the storage of large event histories between two successive computations and requires repeated complex multi-query optimization. In Le Subscribe we adopt a different approach where events are processed on-the-fly to discover matching subscriptions. Our main contributions in Le Subscribe are: A semi-structured event model which is well suited for the information published on the Web, and flexible enough to support easy integration of publishers. Founded by “Instituto Superior Técnico” Technical University of Lisbon and by a JNICT fellowship of Program PRAXIS XXI (Portugal) A subscription language which is designed to be simple while supporting the most usual queries on event notifications. An efficient matching algorithm for processing events in real time which can handle a large number of volatile subscriptions and supporting high event rates. Simple interfaces for publishing and subscribing which enable an easy integration of the system in the Web. The system supports both HTTP protocol and Java RMI. The demonstration consists of an application whose goal is to notify interested users of items put up for auction. These items are captured by Le Subscribe from a set of auction Web sites, e.g. ebay [5], amazon [2] or yahoo [15], or can be transmitted to our system using a web interface. Every day, the system has to handle a large number of items. For example, ebay publishes about 560000 new auction items per day.