An intelligent adaptive filtering agent based on an on-line learning model (poster abstract)

We are developing an Intelligent Filtering Agent (IFA) which is capable of automatically monitoring information sources to find documents for a particular information need. While more and more information is available electronically, especially on the WWW, an effective information filtering system is essential to help users decide which documents are relevant to users’ preferences. One major feature of IFA is that it can perform adaptive filtering. IFA can refine its model containing filtering knowledge in an on-line fashion when the user provides feedback. The key component in the IFA to deal with adaptive filtering is an on-line machine learning framework. Unlike batch filtering, a good adaptive text filtering algorithm should maintain high performance not only at the end of the filtering process, but also during the filtering task. We study the unique requirements of adaptive filtering and propose a new on-line lesrning algorithm, known aa the REPGER (RElevant term Pool with Good training Example retrieval Rule) algorithm, for this task. REPGER possesses three characteristics. First, it maintains a pool of terms with high predictive power. Second it incorporates a novel mechanism for retrieving good training examples. Third, it can dynamically learn the dissemination threshold. Some researchers [2] recently investigated the use of the Exponentiated-Gradient (EG) algorithm [4] for this filtering problem. However, the EG-based algorithm may give undesirable performance before they process enough documents. In contrast to many previous work, we investigate the continuous performance of the whole filtering process starting from the beginning.