Need an automation script/application to 'crawl' throughout a website and retrieve data for a database. Can either deliver the script or just the data in CSV or other compatible format.
The spider will start by using a base url, which will show 'results' which have links to individual pages.
It will extract these links from each of the pages that results are shown for. Then it will visit each page and search that page to collect the name, address, phone, description, etc.
Will disclose specifics through private messages, but this script will be used to gather approximately ~500 initial records for this project, and speed isn't of great importance. All the pages appear to have unique HTML/XML ids so that the information can be easily identified in the source. If you wish to compile/run the script on your own machine and just return the data, we would potentially like to establish a long term connection for further data retrieval needs, depending on it's accuracy and our direction. Can be done in PHP, Perl, or whatever language you prefer to use, so long as we are able to compile/run it on our systems.