Scrape pages from search engine results

Completed Posted Jan 15, 2016 Paid on delivery
Completed Paid on delivery

We have 100K queries we want to pass by (1) a search engine and (2) Wikipedia and scape the results. The key part of this project is knowing how to perform this many searches and avoid getting banned from the source.

The data we want is not in the search results, but structed information on the side. Not every page results has the same type of results. Example: some examples have \"address\" and some have \"phone\", some have \"website\" (as a button). The task is to get all variations of the types of data.

We want this project run as a service. Write the code and give us the data for the 100K queries. At the end of the project, give us an application and show us how we can run it on other data sets.

example sources of data:

[url removed, login to view]

[url removed, login to view]

Input provided will be Excel file with Unique ID, Target, City, State. Search query should be combination of Target + City+ State

Expected output should be CSV file with all fields extracted from both sources (many will be blank).

If you are a scraping expert, this is a very simple procedure. To show that you have read the entire posting, but your favorite color in your bid to show that you read all of this posting.

The attached image shows the section we want to capture.

Software Architecture Web Scraping

Project ID: #9336732

About the project

18 proposals Remote project Active Jan 19, 2016

Awarded to:

e3d

my favorite color is GREEN :) I can scrape search engines, however I can't provide "At the end of the project, give us an application and show us how we can run it on other data sets" since this application will use my More

$800 USD in 10 days
(63 Reviews)
7.5

18 freelancers are bidding on average $461 for this job

mantislin

Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi

$529 USD in 6 days
(127 Reviews)
6.8
fhasanbd

A proposal has not yet been provided

$555 USD in 10 days
(119 Reviews)
6.5
Maariyaa

Hello Dear, HERE Ready to start!!! I can perfectly do it. I have lot of experience related and different types of projects. I am waiting to your quick positive reply. Feel free ask me question if any. I Look for More

$750 USD in 10 days
(102 Reviews)
6.1
johnpo

A proposal has not yet been provided

$526 USD in 10 days
(28 Reviews)
5.7
tomky

color: Blue ================== Hi Currently my bid is only about Wikipedia part but with another angle. Actually I can download data from Wikipedia (which way they recommend for massive questions) and search titl More

$444 USD in 20 days
(89 Reviews)
5.8
AwaisChaudhry

A proposal has not yet been provided

$412 USD in 10 days
(7 Reviews)
5.2
sajidmahmud

Hello I am sajidmahmud I see your project and i do it as your acceptation.I have 3 years experience to work in a any kind data process Web Scraping, Web Search, Excel,SEO,Internet Marketing,Social Media,Lead,Telemarket More

$400 USD in 7 days
(119 Reviews)
5.5
ExpertScraper1

I have already scraping scripts to scrape any websites. I can provide web scraping, data scraping, web data extraction, data mining services from online web resources. I can scrape secured and protected sites (http o More

$250 USD in 10 days
(33 Reviews)
4.6
marketiplace

I got 7+years work experience in Data Collection,Bulk Email Campaign,Excel VBA and Internet Research in IT companies here.I can do create script and scrap datas from site Directory using C++,Python and Perl coding as p More

$333 USD in 3 days
(3 Reviews)
3.8
TeamWorks1inc

Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, teamworks1inc

$333 USD in 3 days
(13 Reviews)
4.0
SarikaSabrin

Hi I'm Specialist in Microsoft technologies ( .NET , SQL Server , vba for applications) and The Master of listed bidders. I Look forward to hearing from you. Regards Sarika

$526 USD in 10 days
(19 Reviews)
4.1
mkjtunna

hi i read your project description and i think i can help you out with it. i have made similar kind of project for a client already. only difference is he wanted to scrap all facebook results from google and then he More

$300 USD in 2 days
(6 Reviews)
3.3
sadikonline

Dear Sir, I am very interested to your project. Please give me a chance to work with you. I have good knowledge about Data Entry, Web Scraping, Excel. Dear Sir, I am very interested to your project. Please give me More

$250 USD in 3 days
(21 Reviews)
3.4
saqibmunir55

ready to start the task ready to start the task ready to start the task ready to start the task ready to start the task

$555 USD in 5 days
(7 Reviews)
3.0
sinogermany

Hi there, if your 100K queries are only for wikipedia and you are just grabbing the right part of information, yes I can do this for you. And here's my suggestion: instead of grabbing just one query or grabbing all 100 More

$300 USD in 10 days
(4 Reviews)
1.8
dhwanikaneria

Hi , I am glad to bid on my first project. Notably, It's my first project on this site. But, I have worked on this type project before and I have knowledge about Search engine and little bit of things that you mention More

$555 USD in 10 days
(0 Reviews)
0.0
waheni

Let me help you ---------------------------------------------------------------------------------------------------

$555 USD in 15 days
(0 Reviews)
0.0
Churchi11

A proposal has not yet been provided

$461 USD in 8 days
(1 Review)
0.0
dhavalIT

Will be able to it what is required from your side. Multiple searches as required from your side. Let me know

$500 USD in 7 days
(0 Reviews)
0.0