Weaving Entities into Relations: From Page Retrieval to Relation Mining on the Web

With its sheer amount of information, the Web is clearly an important frontier for data mining. While Web mining must start with content on the Web, there is no effective “search-based” mechanism to help sifting through the information on the Web. Our goal is to provide a such online search-based facility for supporting query primitives, upon which Web mining applications can be built. As a first step, this paper aims at entity-relation discovery, or E-R discovery, as a useful function– to weave scattered entities on the Web into coherent relations. To begin with, as our proposal, we formalize the concept of ER discovery. Further, to realize ER discovery, as our main thesis, we abstract tuple ranking– the essential challenge of ER discovery– as pattern-based cooccurrence analysis. Finally, as our key insight, we observe that such relation mining shares the same core functions as traditional page-retrieval systems, which enables us to build the new ER discovery upon today’s search engines, almost for free. We report our system prototype and testbed, WISDM-ER, with real Web corpus. Our case studies have demonstrated a high promise, achieving 83% − 91% accuracy for real benchmark queries– and thus the real possibilities of enabling ad-hoc Web mining tasks with online ER discovery.