Build a ruby scraper to supplement a MySQL product database
$250-750 AUD
Paid on delivery
Stagecoach Software Pty Ltd is developing a website to contain statistical profiles of a particular industry in Australia. Our stack is Ruby on Rails 4.2, Ruby 2.2, Passenger 4 and MySQL 5.5. We run a 3 server configuration on Engine Yard with dedicated instances for our db, web app and utilities. Our database receives about 30,000 new rows per day through scraping operations (22 different scrapers using nokogiri). Presently our app instance only provides admin operations; the consumer front end is under development.
Currently our team comprises 3 backend ruby developers and 1 front end developer.
We have about 16 M products in our database. Each product has a distinct id number and name. Each product has 4 types of characteristics which we wish to record. Our database is incomplete. We need a ruby scraper built to look up a particular website and record these characteristics to our database for about 11 M products where they are missing.
The code will probably take several days to develop and may take several weeks to run. The running of the code must be monitored during that time. Multiple instances must be run to minimize the completion time.
All code must be developed locally and periodically uploaded to a github branch before being approved by the senior developer for merge and deployment.
Bidders are invited to set out a summary of their methodology.
This is one of a pipeline of projects in this enterprise. Further ruby projects will be available if the above tasks are carried out well. Applications from a freelancer experienced with Engine Yard and chef are particularly welcomed.
Project ID: #9862978
About the project
Awarded to:
3 freelancers are bidding on average $673 for this job
Dear, When I read this job I feel very excited to be a part of your project and share my great experience in software development. I worked as developer with team members in big projects developers. I know th More