Build a Python Based Web Scraping Application
$750-1500 USD
Paid on delivery
I'm looking for an expert in Python and Web Scraping to take on a very specific project. The ultimate project will actually NOT be Django, but rather using a microframework called Flask (If you have Django experience, Flask is about a one day learning experience).
For Python, we prefer people with Scrapy experience, and would like to stick with Scrapy and Flask.
Essentially, you'll be building a spider that will hit a specific website, and your job is to crawl approximately 20K pages. The first time, we do a full scrape, if the page exists in the Database then you'll check for changes and updates. Accuracy is 100% vital.
All data should be collected using JSON, the spider will of course run server-side, but there needs to be a corresponding web interface for manually performing CRUD operations. You should also note, that we expect the collected data to be stored in a MySQL database (which should be appropriately indexed and normalized).
Finally, it is an absolute must that you have GIT experience. Not just doing code checkins 1x per day, but routinely doing pushes to our repository. If you don't understand GIT and how team development works with a DVCS, please don't waste the time bidding. Also, if you don't get anything herein, again don't waste our time.
If you send me 20 useless website URLS I'll passover you entirely. I'm a little jaded by people that keep submitting the same "blanket response" to every project. You have enough information to accurately bid within reason, but ask questions if you don't.
Project ID: #7108070
About the project
25 freelancers are bidding on average $1269 for this job
Hi, I can do this for you. Please send a massage in the PMB for details.......Best Regards flashsaiful
hello i have good experience in writing crawlers in python . And also with scrapy and crawlera . i can work with django,flask and git . thanks
Hello Sir, I can develop your web scraper, I have already developed this type of work using BeautifulSoup. It's nota problem for me to use Scrapy. For example, I get the entire database of citizens in France usin More
I have substantial experience developing services and applications in Python, Django and Flask which would be ideal for this project. I've around 5 years commercial experience with Python, and over the past year have u More
This is a expert python developer with experience in Django and Scrapy. Hope to establish cooperation relationship. Regards.