I need a Python Scrapy library based web scraper with the following features:\\r\\n1. Mysql and SQLite connection and schema creation (no schema was created, here you can create your own... should be easier)\\r\\n2. Proxy integration, test and update in the DB for proxy that will connect to the tested website\\r\\n3. Configurable to work for several websites instead of creating several scrapers (Optional)\\r\\n4. Multithreaded\\r\\n5. Logging into file of progress as wel as on screen\\r\\n6. HTML output of the scraped data as well as DB insertion of the data\\r\\n\\r\\nThe websites to be scraped are: http:\\\\\\\\\\\\\\\\\/\\\\\\\\\\\\\\\\\/[login to view URL]\\\\\\\\\\\\\\\\\/ , http:\\\\\\\\\\\\\\\\\/\\\\\\\\\\\\\\\\\/[login to view URL]\\\\\\\\\\\\\\\\\/ , http:\\\\\\\\\\\\\\\\\/\\\\\\\\\\\\\\\\\/[login to view URL]\\\\\\\\\\\\\\\\\/ , http:\\\\\\\\\\\\\\\\\/\\\\\\\\\\\\\\\\\/[login to view URL]\\\\\\\\\\\\\\\\\/
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi