Implementation of multiuser personal web crawler

The search engines many times give irrelevant searches which are based on general user preferences. Moreover, they maintain the user search logs and other information which is considered as privacy breach. The personal web crawlers not only magically understand precise requirements, but also they can be scheduled to automatically grab the information at regular intervals. These personal crawlers are not as fast as the commercial crawlers are, but they serve the sole purpose of getting the exact information at the desk. This paper details implementation of such an effective multiuser personal web crawler where one user can manage multiple topics of interest. The work is also supported with experimental evaluation.

[1]  Xin Zhang,et al.  HAWK: A Focused Crawler with Content and Link Analysis , 2008, 2008 IEEE International Conference on e-Business Engineering.

[2]  Ning Zhang,et al.  An Improved Link Selection Algorithm for Vertical Search Engine , 2009, 2009 First International Conference on Information Science and Engineering.

[3]  Anirudha Sahoo,et al.  An 802.11 Based MAC Protocol for Providing QoS to Real Time Applications , 2007 .

[4]  Yan Chun,et al.  An evolutionary relevance calculation measure in topic crawler , 2009, 2009 ISECS International Colloquium on Computing, Communication, Control, and Management.

[5]  David J. DeWitt,et al.  X-Diff: an effective change detection algorithm for XML documents , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[6]  Debajyoti Mukhopadhyay,et al.  A New Approach to Design Domain Specific Ontology Based Web Crawler , 2007, 10th International Conference on Information Technology (ICIT 2007).

[7]  A. K. Sharma,et al.  Architecture for Parallel Crawling and Algorithm for Change Detection in Web Pages , 2007, 10th International Conference on Information Technology (ICIT 2007).

[8]  Debashis Hati,et al.  An Approach for Identifying URLs Based on Division Score and Link Score in Focused Crawler , 2010 .

[9]  Filippo Menczer,et al.  A General Evaluation Framework for Topical Crawlers , 2005, Information Retrieval.

[10]  Sharma Chakravarthy,et al.  Automating Change Detection and Notification of Web Pages (Invited Paper) , 2006, 17th International Workshop on Database and Expert Systems Applications (DEXA'06).

[11]  P. P. Halkarnikar,et al.  A Novel Approach for Web Page Change Detection System , 2010 .