Connect and edit existing PHP web scraper/scraping script to connect to MySQL database in Magento and CentoOS/Apache
$30-250 AUD
Closed
Posted about 10 years ago
$30-250 AUD
Paid on delivery
Hi all,
We have 2 websites:
[login to view URL] << Custom PHP website
[login to view URL] << Magento CE 1.8.1 based
This project will update products on both of our websites.
What you need to do:
1) Connect an existing PHP script (product scraper, JSON based) to our MySQL databases. At present, the script is meant to scrape all product data in full.
2) Update the script to be run only as an update (i.e. do not scrape all product info but only scrape price changes and stock level changes or only add products to our database that do not exist yet)
3) Create an ADMIN page on [login to view URL] and on [login to view URL] where:
a) we can run the script to scrape all product details in full (like what it is designed to do now)
b) we can run the script only to perform product updates (i.e. scrape only products that are not in the database, update prices and stock levels for existing products in our database + anything else you might suggest). The idea here is to run the script a lot faster.
4) Enable multi threading so that the script runs a lot faster. Use of proxies is preferred. There are over 100,000 products that we are trying to scrape so we need to bring the running time of the scraper down to under 2 days.
5) Allow us to archive products that are no longer in the data feed being scraped (i.e. the item has been removed from the supplier website). So we need to archive it in our database (not remove it as we still need a record of it), so that clients do not buy the item anymore.
6) Install any additional PHP modules on our server if need be to make the above work.
7) Set up crons to do automatic updates
8) Set a 'date stamp' to identify when a product has last been updated
9) + other ideas (this will determine the good versus the bad developer - we are after someone who has plenty of experience)
What we have:
[login to view URL]
1) Magento online store, running CE 1.8.1
2) MySQL database
3) CentOS dedicated server will full root access (i.e. cPanel, FTP, MySQL, phpmyadmin etc)
4) PHP script that scrapes products from a 3rd party website
[login to view URL]
1) Custom PHP online store
2) MySQL database
3) PHP script that scrapes products from a 3rd party website
4) Shared server with most access (i.e. FTP, phpmyadmin, MySQL)
Who you are:
1> Good feedback on Freelancer
2> Good project completion on Freelancer
3> Know web scraping, PHP, MySQL, Apache, CentOS and Magento.
Please ask any questions before bidding. If you bid a certain amount, do not try to revise it later on.
Good luck!
First the bid budget is very less. to do everything you listed like for example proxy scraping is major project. Also to change something on magento will take time and also its complex to do where the price will affect.
My offer for you Price - $950 for All these but i will do all these only on custom coded PHP website . if both websites reside on 1 server then we can update both with 1 script. let me know if ur ok with price then we can work and discuss more.
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi
Dear Sir, We claim to get it done perfectly for you EXACTLY in the way you want it - Kindly give we a chance and we will prove myself - Ready to prove our words,
let's get it done right away and I mean RIGHT AWAY !! Looking forward to hear from you soon - GOD Bless You.
Hi,
I hope my message finds you well, I am sherif, system administrator and software
developer with experience in working on LAMP/XAMPP, i have applied configuration for reverse proxy
caching (varnish) and byte caching (APC) for drupal & wordpress before, I worked with different
control panels like CPanel & ISPCpnfig
I have great experience working with shell commands, setting up cron, writing perl and shell/bash
scripts
I have existing experience working with Linux with different favors as well as unix systems
And most important, I have good communication skills and great understanding for expectations
Sherif
Hello, We have excellent team of programmers and designers to work on your project efficiently and complete job in time. We have read your deepest requirement at our best and will surely give better results. thanks
Hi Dear Hiring Manager, I ensure to bring all your requirements successful in time and you will see the
difference of my work. I always try to provide more then 100% quality work as established the long term
relationship. Thank You, Yogesh Kumar.