FetchBot

FetchBot - Gather Intelligence Data for a List of Domains FetchBot gathers summary data for each domain in a list. It collects country/state geographical information by parsing pages such as "Contact" pages.

FetchBot collects Google Adsense IDs, page titles, charset, and builds a word vector containing the most frequent words found on the domain\'s home page (disregarding stop words).

FetchBot has no user interface. To run the program, simply double-click on the FetchBot.exe program. It will read the C:/WorldWideWeb-X.com/FetchBot.xml configuration file. If the directory and/or file does not exist, it will create it and exit. You may then edit the configuration file to specify the input/output files and re-run.