Web Crawler Software
Crawl a website with ActiveX Spidering component. Advanced features include caching, "avoid" patterns, and robots.txt compliance.
Spider ActiveX Component for Crawling the web:
-Cache pages so future crawls can fetch from cache;
-Fetch the HTML content of each page crawled;
. Freeware download of Chilkat Spider ActiveX 126.96.36.199, size 1.26 Mb.
This program lets you to substitute URI for symbol codes of the ASCII table. It's very useful to prevent scanning of web pages by the crawlers gathering e-mail addresses for broadcasting spam messages on the next step. Using this software you can protect your mailbox from tons of junk mail, and will not be afraid of publishing contacts on your . Freeware download of URI Crypter 188.8.131.52, size 6.48 Mb.
Automatically search web sites for emails. Simply provide a starting web site or starting keywords and let emailgrabber do the rest. Emailgrabber will crawl the web site and collect all email addresses it finds. It will also follow every link. Specify the web sites you want to visit to allow emailgrabber to stay on top of your goal or block web . Free download of EmailGrabber 1 1, size 2.17 Mb.
Desktop IRIS is an easy-to-use search program that can be successfully downloaded and accessed by anyone. It allows you to intuitively find stored information from your desktop and network without imposing any restrictions on the number of files and folder locations indexed. The system uses the same security model for the desktop and network . Freeware download of Desktop IRIS 3.0.6, size 58.25 Mb.
The Indexing Service Companion allows the Microsoft Windows Indexing Service to index content from remote websites (including those running Apache on Linux/Unix) and ODBC databases. Includes sample code for ASP (VBScript) and .NET Framework (C# and VB.NET). The Indexing Service Companion contains full documentation and code samples for use in a . Free download of Indexing Service Companion 3.0, size 1.43 Mb.
Darcy Ripper is a powerful pure Java multi-platform web crawler (web spider) with great work load and speed capabilities. Darcy is a standalone multi-platform Graphical User Interface Application that can be used by simple users as well as programmers to download web related resources on the fly. . Freeware download of Darcy Ripper 184.108.40.206, size 4.30 Mb.
Arkaif is a simple web crawler designed to help you get the web links for any available website.
Arkaif runs in the command console, enabling you to view a list of the target links from the web archive and save them in a TXT file on your computer.
. Freeware download of Arkaif 220.127.116.11, size 0 b.
FAROO is a web search engine based on peer-to-peer technology. Each visited webpage is automatically displayed in the distributed index that can be accessed in the tool's options.
The ranking of search results is based on a distributed usage statistics of the web pages visited by FAROO users.
The application supports the . Free download of FAROO 3.5.1025.33404, size 0 b.
Avant Prime web Miner is the ultimate data extraction, web content mining and web scraping tool. Using the built in web crawler to crawl a single page, an entire website or a unlimited number of interlinked websites automatically while extracting content using one of the built-in miners which include: emails, images, URLs, feeds, phone numbers, . Free download of Web Miner 1.2.502.1318, size 4.64 Mb.
Content Grabber is a superior web scraping and web automation tool. It is very powerful, yet extremely easy to use. It can extract content from almost any website and save it as structured data in a format of your choice, including Excel reports, XML, CSV and most databases.
Content Grabber has all the bells and whistles that you'd . Free download of Content Grabber 1.17.7, size 56.01 Mb.
Octoparse is a powerful but easy-to-use and free web scraping tool that is being used by tens of thousands of individuals and companies all over the world. The word of Octoparse is the combination of the two words, â€œOctopusâ€ and â€œParseâ€. This implies that Octoparse can extract and parse data around the vast . Freeware download of Octoparse 6.0, size 681.98 Kb.