Automate Scraping and Aggregation (1919945)

Cancelled Posted Jul 5, 2012 Paid on delivery
Cancelled Paid on delivery

- A program that visits **hundreds/thousands** of specified websites, reads their pages, processes instructions written in scripts, extracts specific data from those pages [name, location, dates, etc.], then saves that data in another more structured format to incorporation into website or app.

- Program will scan websites for changes in data by comparing previous values stored to what's currently on the page.

- Automated update process (eg, catalog updates every N hours automatically) - no manual intervention required.

- Customer service tool enabling non-programmers to add or remove websites from the list of sites to visit.

- If data is hidden behind registration walls, the program will also need to enter credentials to access and then scrape the data.

MySQL PHP Script Install Shell Script Software Architecture Software Testing Web Hosting Website Management Website Testing

Project ID: #2756937

About the project

Remote project Active Jul 5, 2012