Scrapy crawler for expired domains

$30-250 CAD

Closed

Posted

almost 8 years ago

$30-250 CAD

Paid on delivery

Hello, My client wants to make a Crawler based on Scrapy to crawl expired domains. With what technology you want to develop the project? For this project, I will wish: - Crawler an url via filled a GUI - Spider will follow all the links, for links with no possible DNS resolution, the areas will be tested through an API to know if the domain is available. - If the domain is available, it will be stored in a MySQL DB and query will be performed on the Majestic API to see the TF, CF and Referal Domains, this information will be stored in MySQL. - A GUI will display a table of expired domains with their associated values. Budget is limited to 100 CAD. Please respond me on every item in your offer, thank you. Thank you,

Project ID: 10909855

About the project

14 proposals

Remote project

Active 8 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

14 freelancers are bidding on average $139 CAD for this job

@phpXpertbd

Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, PhantomJS, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database or excel or csv or xml file. I worked on many similar projects, I have big experience in data mining projects. I have written hundreds of web scrapers which scrape millions of pages each day. I'm ready to fulfill your requirement. I can finish this task in short time, with the best quality. I can assure 100% accuracy. Please give me the opportunity to do the work. With Kind Regards, Debdulal Roy Proshanta

$250 CAD in 3 days

4.9

(46 reviews)

6.6

@sstevan

Hello, I think you don't need web crawler for this specific purpose. I already have similar script and I'm willing to send you the working demo which will print expired domain - and check it using Majestic API. Can you send credentials for Majestic API and I'll try to deliver working demo tomorrow? Thanks, Stevan

$155 CAD in 6 days

5.0

(7 reviews)

5.2

@qeeqez

Hello, I'm Junior Java Developer. I have 3-year java experience, including 10 months of commercial development. I understand your task and ready to work.

$100 CAD in 5 days

5.0

(4 reviews)

3.0

@oatechnology

hi , hope you are doing well we are expert with web scraping , we will use java language in this project we can provide demo or sample for your requirements before awarding we are new in freelancer.com but we have 5 years experience in software development Thanks OAtechnology

$100 CAD in 3 days

5.0

(2 reviews)

2.2

@orpivirshad

Hello, My name is Irshad I am expert in web scrapping, copy writing, data crawling and data mining, I have 5 years+ experience in data mining, website scraping, screen scraping and I am certified web scrapper expertData Entry,Excell,. I have completed 100+ projects on web scraping -- i would love to create such a SCRAPPER for you My key Skills are: Web scraping, Data mining, Data crawling, Ms Office(FULL) JavaScript, Go, PHP, SQL, HTML, Python, Perl, CSS. Platforms: Linux, Amazon Web Services (AWS), Google App Engine, Windows, Mac OS X I have completed more than 100+ projects in Web Scraping. You can test the quality of my work if needed, and provided that I can also show you my sample works. Regards Irshad

$100 CAD in 4 days

3.2

(1 review)

0.6

@SbStudios

Hello, I can do this project. I am 8 years experiencied in development and SQL databases and I have knowledge of DNS protocol. If you hire me, the project will be done in 3 or 4 days.

$100 CAD in 3 days