Automatically scrap data from our suppliers portals and load into our AWS server

Completed Posted 7 years ago Paid on delivery
Completed Paid on delivery

We're a firearms accessories online retailer and we sell a huge range (10,000+) of products. To get our quantities and prices up to date we have to scrape data from our suppliers online portals as APIs aren't available.

To date we've been using executable files based on java to scrape the data we need, but going forward, we want this to happen automatically from our AWS instance.

This project is to transition one of our scrapers to AWS. If successful, we will continue the work for our other suppliers (4 in total).

Notes

- I have attached the current executable we use as well as the output that it produces.

- The scraper should automatically run at a pre-determined frequency (daily) and append time stamped data to a table which can be queried in SQL

- If the scraper fails there needs to be an alert

- The scraper needs to be well documented so that it can be troubleshot

- An example of the type of fields that we required scraped is attached. This does not include images.

Amazon Web Services SQL Web Scraping

Project ID: #11957025

About the project

19 proposals Remote project Active 7 years ago

Awarded to:

SuiGenSolutions

Hello Sir, I have gone through your requirements. Basically you would like to place your one scraper on AWS instance and run it automatically by any scheduler. I have one question - currently your scraper requires More

$147 AUD in 1 day
(33 Reviews)
6.2

19 freelancers are bidding on average $232 for this job

mantislin

Dear sir, I am scraping expert, I have did too many scraping projects, please check my reviews then you will know. Can you tell me more details? then I will provide example data/script for you. Thanks, More

$262 AUD in 6 days
(430 Reviews)
8.2
mmadi

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. please ch More

$150 AUD in 4 days
(11 Reviews)
6.4
masud0012

I am ready to start right now.I am a full time dedicated freelancer. Dear Hiring Manager, I have read your job description carefully and I’m very considering your job such as these capabilities. I have fantastic expe More

$155 AUD in 3 days
(52 Reviews)
5.6
contactvidura

Hi, Ready for the task. Please see my reviews and previous projects. Can we discuss the details further? Waiting for your reply!

$200 AUD in 3 days
(116 Reviews)
5.4
Iamtuheedakram

Hey i can do this job since i am expert in scrapping and automation with more than 3 years of work experience. I have scrapped and automated many sites already such as Instagram, kajiji, fsbo, forsalybyowner and many More

$250 AUD in 3 days
(18 Reviews)
4.9
SkyDevelopers

Hello there! i am a web developer and a designer with over 5 years of solid industrial experience. i have excellent experience working with PHP, javascript, jquery, mysql, ajax, ect... I have very good experiance w More

$250 AUD in 14 days
(9 Reviews)
4.9
markandrewsvw

Hi, I can do this for you. I excel where other web scrapers can't. My recent reviews demonstrate this. If it is visible or clickable on the web then I can scrape it. My primary language is Python - I use the scra More

$175 AUD in 3 days
(13 Reviews)
5.0
csanadfarkas90

Hi! I done a lot of data extraction from websites, with JAVA selenium. I can make a demo with demo data if you want (FREE), before you award me and if you like it we can keep working together Just contact me and i More

$160 AUD in 3 days
(7 Reviews)
3.8
danut1

hello, I am one of the best scrapers developer you will find here. just finished a scraper that downloaded more than 10 milions rows from a website into a database. feedback will appear soon. contact me for more detai More

$250 AUD in 5 days
(7 Reviews)
2.9
rrgindore

Dear Sir i am please to offer you web scrapping solution which can automatically scrap the desired data Please initiate further conversation for better understanding Regards AMit

$155 AUD in 3 days
(5 Reviews)
2.9
MohsanEijaz

Hello there It is my great honor to apply for this vacant job. I am very hardworking and can work effectively as fast as I can. I have excellent skills in JAVA, C++, C#, OPPs, HTML5, CSS3, JS, AJAX, JQYERY, React.js, More

$222 AUD in 3 days
(1 Review)
1.9
Dilipjaipur

Hello, I have ranking in top 3500 on freelancer.com and I hope it will give you an idea about my work quality and dedication to work. You will be safe while working with me. I have dedicated in house team for Wo More

$277 AUD in 10 days
(3 Reviews)
1.6
covernal

Hi, I am glad I have a chance to show you my professional skill. I have completed all projects as successful on the Freelancer.com I will provide you my professional skills for your project. Please check my work his More

$444 AUD in 5 days
(0 Reviews)
0.0