Enabling Time Sensitive Information Retrieval on the Web through Real Time Search Engines Using Streams

Real time search engines constantly index web content originated by data streams also. This is because, the web sources like social networking sites, news, and tweets provide up to date information through streams. As new content is arrived constantly from those sources, it is very challenging job for search engines to have efficient indexing mechanisms to ensure index freshness and coverage of the index. Such updated index supports faster search whose results also include the latest content available. Latencies such as retrieval latency and indexing latency play an important role in index freshness. The former is the time taken to fetch the content after its publication while the latter is the time taken to make index on the newly fetched content. This paper presents a framework which optimizes indexing latency and also indexing coverage. The empirical results revealed that the proposed framework is capable of achieving index freshness and coverage in order to support faster processing of search queries.