Www Robots and Search Engines

The Web robots are programs that automatically traverse through networks. Currently, their most visible and familiar application is to provide indices for search engines, such as Lycos and Alta Vista, and semi-automatically maintained topic references or subject directories. In this article, we survey the state-of-art of the Web robots, and the search engines that utilize the results of robot searches. We also present notions about robot ethics and distributed Web robots.