Education and Experience : Masters in Computer Applications with 10 years of strong experience in all phases Software development.
I love challenging works, if you have one - Consider me for same!
— Mobile Applications: Android / iOS / ionic / WPF / Windows Desktop / Cordova
— API integrations: Facebook / Google / Instagram / Linkedin / Stripe / Twitter
— Browser Extensions: Chrome / Firefox / Edge / Office 365 Addins
— Back-end: NodeJS / PHP / Laravel / Yii2 / Python / JAVA / Wordpress
— Front-end: WPF / Angular / CSS3 / D3 / JavaScript / HTML5 / JQuery
— Front-end: Angular (1-5) / CSS3 / D3 / JavaScript / HTML5 / JQuery / React;
— Databases: MongoDB / MySQL / Oracle / PostgreSQL
— Cloud Hostings: AWS / DigitalOcean / Google / Shared;
— Others: Client-Server / Sockets / Multi-threading / Customize apps / P2P / Wireless / USB.
I have vast experiences on scraping websites using Node.js.
Node.js are asynchronous so it can handle multiple request at the same time. As a reference on my latest scrape projects, scrapping 8,000,000 email from Facebook Business Page and scrapping 9,000,000 repository data from Github User Page are finished in under one week.
As a general rule I can scrape up to 120 pages / minute / machine (assuming the target site has small latency). When using multiple machines the speed will be multiplied. When dealing with large dataset, connection definitely will be a factor. That is why, after developed it locally, I used to use my own private server(s)