Find Jobs
Hire Freelancers

Website Monitoring Tool (web crawler, screenshots, email notification)

€18-36 EUR / hour

Closed
Posted almost 5 years ago

€18-36 EUR / hour

The goal is to have a tool that given a website URL (eg. https://www.freelancer.com/) will send me weekly emails about what has changed on the website. I need to know 3 pieces of information: what new pages appeared (list of urls), what pages were deleted (list of urls), and what pages were modified (list of urls AND before and after screenshots so I can see what's been modified). The tool should send me a weekly email with these 3 pieces of information. So the tool should crawl a given website (eg. https://www.freelancer.com/) to get a list of pages (eg. https://www.freelancer.com/post-project, https://www.freelancer.com/jobs/, etc). It should take full page screenshots of the modified pages. By doing this every week, the tool should send me an email about new, deleted, modified (with screenshots) pages.
Project ID: 19988881

About the project

14 proposals
Remote project
Active 5 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
14 freelancers are bidding on average €26 EUR/hour for this job
User Avatar
Hello, We are IT-Engineers and Business Analysts, creating complete web and mobile solutions, for already more than 15 years, supporting our products with custom digital marketing strategies and instruments. Web-crawling systems are especially preferred by our engineers, as we've created a lot of them already, so it seems to be a good match. Before the implementation starts - the initial briefing round takes place, which is free of charge. We are interested in all the details of your specific case. They will allow us to obtain a better vision of your concept and help us create unique solutions for you. Let's schedule a convenient time for the first kick-off conversation. P.S.: Wir sprechen auch Deutsch. :) Best Regards, Andrei Costenco Team Lead, Business Analyst, Senior Engineer Dev Experts Team
€70 EUR in 40 days
5.0 (125 reviews)
8.8
8.8
User Avatar
Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within the agreed deadline. Check out my profile and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Best regards, Miljan
€30 EUR in 40 days
4.9 (141 reviews)
7.6
7.6
User Avatar
‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHON (Scrapy, Selenium) based web scraper as well as WINDOWS BASED web scraping software through which I have crawled many sites such as Craigslist, Amazon, Yelp and many others. I have also worked on complex site to bypass CAPTCHA with the use of PROXY IP bouncing techniques.. Let's work together :) Have a great day! I am glad to see your WORK HISTORY and positive reviews of other freelancers. I am really excited to work with you and would love to have a long-term business association for any of your data related needs less  ,,,,,,,,,,,,. ,
€20 EUR in 3 days
4.9 (98 reviews)
5.8
5.8
User Avatar
---- Pro Python, Software Architecture, Software Development, Web Scraping, Website Analytics Expert -------------- Dear client, Once I saw a your project, it was attracted my mind because I am very interested in your project and also I have rich experiences and high skills on these fields as you can see my profile. If given a chance, I am highly confident in my ability to deliver the highest quality. I am confident that my involvement in your project will bring it to a successful launch, on time and within budget. I look forward to hearing from you. Kind Regards
€18 EUR in 40 days
5.0 (21 reviews)
5.8
5.8
User Avatar
Hello Sir/Madam, Greetings of the day..! Hope you are doing well. I have gone through your requirement details and understood it completely. I have enough experience in web scrapping using custom scripts as well as using automation tools. I have following areas of expertize: >>Php5, PHP7, Python2, Python3 >>Mysql5.5, Mysql5.7, Mongo3, Sphinx >>Webcrawling, Webscraping, Data extraction >>Unix, Apache, Nginx, SSL, Postfix SMTP >>Aws EC2, SES, Lambda, RDS, S3. >>Selenium Automation
€20 EUR in 15 days
5.0 (25 reviews)
4.9
4.9
User Avatar
Hi, This project seems like a simple script which can be called on a weekly CRON request on the server. The data about the sites can be stored and compared at later dates the project is a quick task and will not take no more than a day or two. Contact me so we can discuss details i am available to start. Thanks
€30 EUR in 40 days
5.0 (9 reviews)
5.0
5.0
User Avatar
I have vast experience of making web crawling and scraping scripts in python, I work as a freelance web scraping expert for multiple MNCs, Let me know about your requirements. Do you want to sell it? That means a fully armed saas. Is UI a concern?
€18 EUR in 40 days
4.9 (18 reviews)
4.1
4.1
User Avatar
Hi. I can provide you with crawling, checking and testing functionalities including dashboard if required. But it is not as simple as it look on the surface and there are some questions which will have to be answered to make sure it works just the way you want. It is not a critical problem as I can provide solutions in any case, the difference is in the time it will take to develop. For the starters will crawl the whole website in search of all the publicly seen pages. All this will be added to the database and timestamped for later tests. Later when the spider will do additional crawls it will be able to test against all data to see the changes. If want to have screenshot for before and after the modification then each crawled page will have to be stored as a bitmap for every scroll. We will have to decide on how deeply you want to store the changes. If it is only previous and current then there is not many problems. But if you want to store a deep log of how page modified over some long period of time then it might require some clever optimizations. For the content changes it is not very clear on what do you consider a change in content. Is it only visible text/image changes on the page or some inner changes like page metadata and maybe changes is in the scripts are also valuable to you? For the page image difference I will suggest comparing maybe by only pixels and if needed highlight with semitransparent shapes around those changes. Write me for more details.
€28 EUR in 40 days
5.0 (2 reviews)
2.8
2.8
User Avatar
hi I have more than 4 years of work experience in c#. So far I developed more than 50 software which scraps different websites using selenium framework.
€30 EUR in 40 days
5.0 (1 review)
1.5
1.5
User Avatar
Hourly rate negotiable, same with hours per week, very flexible hours.... WILL GET THE WORK DONE!! That’s a promise
€22 EUR in 40 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of GERMANY
Garching bei Munchen, Germany
0.0
0
Member since Jun 14, 2019

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.