Find Jobs
Hire Freelancers

crawling plus yahoo search API

$100-500 USD

Closed
Posted over 15 years ago

$100-500 USD

Paid on delivery
Looking to get this done in php or perl.? Requirements are as follows. Given a top? level domain, get all the urls of that website. So The list of urls should be comprehensive and unique. So if a website has 2000 pages, we should have all those 2000 urls. I should be able to go to? web page to fire off the crawling of that website. Yes, the crawling could take a few minutes to even a few hours if the website has lots of pages. It should then dsiplay the pages on the browser. for each page, i should be able to see which other pages on the site link to this page and using what anchor text. So if page A is linked from page B,C,D,E. I should be able to see that page B&C link to A using anchor text "blah1" and pages D&E link to page A using anchor text "blah2". The above is true for internal links within the website. Also, for each page on the website, would like to query the yahoo siteexplorer api <[login to view URL]> and get the external backward links for every url on that site.
Project ID: 3019509

About the project

4 proposals
Remote project
Active 16 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
4 freelancers are bidding on average $191 USD for this job
User Avatar
See private message.
$85 USD in 14 days
4.9 (16 reviews)
3.7
3.7
User Avatar
See private message.
$85 USD in 14 days
5.0 (7 reviews)
3.5
3.5
User Avatar
See private message.
$170 USD in 14 days
5.0 (2 reviews)
2.6
2.6
User Avatar
See private message.
$425 USD in 14 days
0.0 (1 review)
0.0
0.0

About the client

Flag of UNITED STATES
United States
0.0
0
Member since Jun 28, 2008

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.