Web scraper using Python + Scrapy (+ XPath) + PostgreSQL

€30-250 EUR

Completed

Posted

over 4 years ago

€30-250 EUR

Paid on delivery

Simple web scraper using Python + Scrapy (+ XPath) + PostgreSQL (current versions of products). Data will be scraped from one particular site, it contains three types of pages (based on page is different scrap logic). It will works as follows: - Reading input data for page from DB input table (based on selection criteria in script), sending to server - Reading the response, parsing (based on page type) and saving to DB output table From every page will be created 1-N rows in database – exact rows count is dependent on one from scraped values. Database structure of input/output table + data and detailed decrciption including tracing of scraped data to table fields will be provided. Functions for avoiding to be blocked also requested (using proxies, changing headers, etc.)

Project ID: 22798217

About the project

25 proposals

Remote project

Active 4 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

Awarded to:

@evgeny1002

Hi I've experience in development with python for several years. Especially I've majored in scrapping by Selenium & Spider & BeautifulSoup, and building web sites by Django & Flask. These are my recent scrapping jobs. [login to view URL] There are scrapping scripts for extract data from [login to view URL], [login to view URL], and one for table from pdf. I'll be glad to help you. Thanks, Evgeny

€50 EUR in 7 days

5.0

(29 reviews)

6.4

25 freelancers are bidding on average €143 EUR for this job

@mantislin

Hi there, I am scraping expert, I have did more than 350+ scraping project, please check my feedback then you will know. Can we discuss more details about this project? then I will provide example data/script for you. Thanks, Lin

€132 EUR in 3 days

5.0

(355 reviews)

7.9

@gangabass

I can provide you a Scrapy spider that will read input data from DB, visit website, parse HTML and next save output into database. It will use proxy / useragent rotation. I can complete it in less than 36 hours for just 140 EUR. All I need to start is some samples from you (input / output). Thanks. Roman

€140 EUR in 2 days

4.9

(461 reviews)

7.8

@zekovicm

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within the agreed deadline. Check out my profile and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Best regards, Miljan

€140 EUR in 3 days

4.9

(160 reviews)

7.8

@osmanaydin

Hello sir, Please take a look at my reviews and portfolio. I have completed web/data scraping jobs many times. I am interested in your project as well. I would like to discuss the details via pm. I look forward to hearing from you soon. Best Regards,

€100 EUR in 1 day

5.0

(51 reviews)

5.9

@marcnealer

Hi I'm a python web app developer with scraping skills. I will have no issue interacting with your DB to read and write back results. I am also familiar with various scraping techniques. On the proxy side, I have code that scraped free proxies from various sites and tests them against the required target to ensure they work, that can avoid being blocked. Thanks Marc Nealer

€250 EUR in 5 days

4.9

(13 reviews)

6.0

@NomiHD

I have been doing web-scraping since more than 4 years using python scrapy which yields data very quickly and yet in a reliable fashion. I wrote more than 1000+ scripts including: ebay yellowpages yelp zoopla rightmove meetup bestbuy (ecommerce) barnesandnoble (ecommerce) macys (ecommerce) targets (ecommerce) and many more. Looking forward to hear you soon. thanks.

€100 EUR in 2 days

5.0

(53 reviews)

5.7

@smsaurabhv

Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHON (Scrapy, Selenium) based web scraper as well as WINDOWS BASED web scraping software through which I have crawled many sites such as Craigslist, Amazon, Yelp and many others. I have also worked on complex site to bypass CAPTCHA with the use of PROXY IP bouncing techniques.. Let's work together :)

€100 EUR in 3 days

4.9

(107 reviews)

5.9

@Guptapuru304

Hi! I'm senior python developer, working for 3 years now. I am expert in python automation using selenium for chrome and web scraping which I have done for sites like amazon, facebook, ebay etc. I can complete the project in 2-3 days. I have done various automation projects recently similar to this, and can do it in a cost efficient way. Kindly contact me for more details regarding the project and see my previous works. Thanks, Puru

€125 EUR in 3 days

5.0

(24 reviews)

5.5

@unix515

Dear, sir. I am so happy to see your scraping project. I am so familiar with Scrapping and Entry. I will use Python and PostgreSQL. I can send good result you want. Hoping reply from you. Best Regards.

€120 EUR in 2 days

4.4

(15 reviews)

5.0

@Prakhark19

Hello there, Myself Prakhar, i am working in python for last 3 years. I have read your description thoroughly and i am confident that i can do this easily. I have experience in scraping and have created various s scraper like insta scraper LinkedIn scraper Let's discuss further in personal chat. Regards Prakhar.

€39 EUR in 1 day

4.9

(25 reviews)

4.2

@mathewtmathews

I had been doing scrapping from last 3-4 years. Already done with around 15000 websites so far with scrappy, celenium , browser automation in many real estate,webshop, news sites .I am much sure that I will be the best fit for your task.I will do a sample and if u feel that is fine then we can have a deal.

€140 EUR in 3 days

5.0

(5 reviews)

3.3

@adam01025

Hello. I am the data entry/processing/mining data scraping/web scraping/web crawling and automation expert and I've solid experience in similar works. I am a experienced programmer, so I use various automation tools that I specially made. I am ready to work with you with high speed and quality. I am waiting your quick positive reply. Thanks.

€140 EUR in 7 days

5.0

(1 review)

2.0

@OrionTexh

⭐ Good Day Friend. ⭐ :) I am sure you will get the perfect result if we can work together. I wanna talk to you about more detail via the chat. Thanks for viewing my proposal. Regards.

€250 EUR in 7 days

5.0

(1 review)

0.8

@kkinfotech099

Thanks for your posting I checked your requirements and I have experience in similar work and can do demo WORK FREE If you like my trial work then we have deal or you dont have to pay anything For trial work please contact me I have 10 years of expert team in different apps, website and software development waiting for your reply Thanks and Regards,

€140 EUR in 7 days

0.0

(0 reviews)

0.0

@henrikki

Hi, Having read your complete requirements, I would like to assure you that I will produce the most notable results for you, having had a myriad of previous experience with similar projects. I would like to discuss the thorough details with you to fully grasp the requirements. Can you send me the website address? thanks

€140 EUR in 7 days

0.0

(0 reviews)

0.0

@med91didi

Hi. I have experience in scraping with python . I will be pleased to work for you and deliver a good work at a good price.

€80 EUR in 3 days

0.0

(0 reviews)

0.0

@k1mnicol

Hello I have just reviewed your job description carefully. I have 5+ years of experience in web scrapping and web development with Python, C#, Php, Javascript, … BeautifulSoap Expert I can start right now and will finish this project asap. Looking forward to hearing from you. Thanks in advance for your time and consideration. Best Regards

€140 EUR in 7 days

0.0

(0 reviews)

0.0

@arshadin

HI, I have 4 years of experience in python and webscrapy, have crawled various e-commerce sites. Have crawling experience of android application also. Thanks

€80 EUR in 7 days