I need of a distributed web crawler + indexing, that can take care of crawls of any size. For example the crawler must be able to crawl & indexing a single website (few web pages) as well as the whole web (over a billion web pages).
Installation & configuration : Apache Nutch
2 freelancers are bidding on average $177 for this job
I have experience in apache Nutch. I am a developer with years of experience in the field of development. I can deliver you the optimal results with reference to crawling and indexing. thanks,