论文信息 - A topic-specific crawler with dynamic concept context graph based on FCA

A topic-specific crawler with dynamic concept context graph based on FCA

An excellent topic-specific crawler should retrieve as many high related web pages as possible in the limited time. The historical clicked web pages can express the users’ interest in a certain extent. By using the knowledge of Formal Concept Analysis (FCA), some information is extracted out to construct the concept lattice which is used to build the Concept Context Graph (CCG). In this paper, we construct a knowledge background for the topic-specific crawler by using the relevant web pages. And a Dynamic CCG (DCCG) is proposed in this paper which updates the concepts in a way of elimination mechanism all the crawling process. At last, several different CCGs are taken into the experiment for the comparison in their performance.

[1] Carl Lagoze,et al. Focused Crawls, Tunneling, and Digital Libraries , 2002, ECDL.

[2] Filippo Menczer,et al. Topical web crawlers: Evaluating adaptive algorithms , 2004, TOIT.

[3] Yajun Du,et al. Semantic ranking of web pages based on formal concept analysis , 2013, J. Syst. Softw..

[4] 杜亚军. The research of Concept Context Graph layer division based on six degrees of separation theory , 2013 .

[5] Yajun Du,et al. Topic-Specific Crawling on the Web with Concept Context Graph Based on FCA , 2009, 2009 International Conference on Management and Service Science.

[6] Geert-Jan Houben,et al. Information Retrieval in Distributed Hypertexts , 1994, RIAO.

[7] Yuekui Yang,et al. Focused Web Crawling Based on Incremental Learning , 2010 .

[8] Jun Li,et al. Focused crawling by exploiting anchor text using decision tree , 2005, WWW '05.

[9] Jingyu Sun,et al. A Topic-Specific Web Crawler with Concept Similarity Context Graph Based on FCA , 2008, ICIC.

[10] Xiang Kun. The Shark-Search algorithm based on clustering links , 2006 .

[11] Yuekui Yang,et al. Incrementally Updating Concept Context Graph (CCG) for Focused Web Crawling Based on FCA , 2009, 2009 Asia-Pacific Conference on Information Processing.