We need to scrap a website to insert data into a text file.
We will be hosting the application on Windows Azure, and would like it to run as an Azure Function. The code should be tested that it can run under that environment.
Target language will be Node.js.
The function will be scheduled to run 4 times a day. Care should be taken that the same data isn’t scraped more than once.
We will need to scrape a static URL that points to different locations.
Output will be to a CSV file stored in an Azure Blob Storage container. Allow this container name and credentials to be set via simple configuration.
Source control will be handed over via pushing to a git repo
We will need a regression suite which removes any dirty data prior to sending to a text file.
Each milestone will be deemed complete once feedback has been given and adequately implemented in the final script.
Please contact us for test URLs, examples of the output files and a description of the milestones.