Scraping and Assembling Data from website

$250-750 USD

Closed

Posted

about 9 years ago

$250-750 USD

Paid on delivery

Important notes on project: (a) please do not bid on this unless you have experience doing a similar web scraping and assembling project. (b) the older versions of the pages may be formatted in a different way and be harder to extract information from. Please be sure to look at pages at the end of the list (see (B)) before bidding. (A) Background -------------------- The links below contain a series of short text stories. For each link I would like a separate file in which the following information is extracted from each story and is listed on a separate line (each field below should be in quotes and the fields should be comma delimited): - story url - story name - date - film/tv show name - filming location 1 (state) - filming location 2 (street + city) Note that if more than one show or film location is listed in the story, then each show-location should be listed on a separate row. Please see the attached in (C) for an example for one of these links. (B) Links to scrape (note the set of sub-links for each case) ---------------------------------------------------------------------- (1) [login to view URL] Currently 468 pages (this will increase as the site adds entries) [login to view URL] [login to view URL] [login to view URL] ... [login to view URL] NOTE: many stories have a "[click to continue…]" link at the bottom. (2) [login to view URL] Currently 22 pages (this will increase as the site adds entries) [login to view URL] [login to view URL] ... [login to view URL] NOTE 1: many stories have a "[click to continue…]" link at the bottom. NOTE 2: some stories in this section will not involve filming locations and can be omitted from the file (if it is easier, the row could be included and entries for film/tv show name and film location may be left blank. (3) [login to view URL] Currently 537 pages (this will increase as the site adds entries) [login to view URL] [login to view URL] ... [login to view URL] NOTE 1: many stories have a "[click to continue…]" link at the bottom. NOTE 2: some stories in this section will not involve filming locations and can be omitted from the file (if it is easier, the row could be included and entries for film/tv show name and film location may be left blank). (C) SAMPLE FILE: ----------------------- see attached [login to view URL] based on [login to view URL] - [login to view URL]: example output file - OLV_Daily Filming [login to view URL]: main raw file - Friday, Feb 20 Filming Locations for Extant, Agents of S.H.I.E.L.D., Glee, Joy, .pdf: sub-link 1 - Thursday, Feb. 19 Filming Locations for Once Upon A Time, The Good Wife, Castle,.pdf: sub-link 2 NOTE 1: The sub-links files are based on clicking in the story name in the raw file (note that main file does not contain the full story) NOTE 2: I have only done the first two stories from the original page (D) FINAL PRODUCT --------------------------- Either of the following would be acceptable: (i) the three comma delimited files described in Sections (A) and (B) (ii) a program which can be installed an run on a Windows 7 x64 computer which will extract and create the files

Data Mining

Web Scraping

Project ID: 7173782

About the project

15 proposals

Remote project

Active 9 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

15 freelancers are bidding on average $491 USD for this job

@e3d

Hello, I can do this and deliver as per "(i) the three comma delimited files described in Sections (A) and (B)" but not (ii) as I won't make it under windows. Let me know if you interested. Thanks!

$750 USD in 10 days

5.0

(263 reviews)

8.7

@mhmhz

Hi I check your requirements. I can build a desktop application that fulfilled the task Let me know if you want to see a demo. Thanks

$1,052 USD in 5 days

5.0

(97 reviews)

7.6

@PigtailXL

Hello Sr. Firstly, thanks for invited me to bid on this project. I guess you have seem my Profile here and my jobs,so I will go directly to my proposal. I'm thinking to develop an application, a desktop one, for Windows compatible platforms (not Linux, not Mac). The app should need to be installed in your computer (Windows 7 64 bits) as a first and simple process. After that you will get an .EXE standard file which you will execute each time you need to run this process. The app could take, as a first step, some configuration parameters from a Config file, previously configured there. That will permits you to run the process as you'd like. Example, you can chose where to save the final Excel file with data or, another idea, how many pages you wish to process (From page and To Page). And then of that, the app will simulate a navigation (automated task without your intervention) over each page, extracting and parsing the information required by you, filling an Excel file and saving it to your chosen location. Finally, the app should notice you when the process has been concluded. I would like to get your attention on this, and if you have some time, discuss your ideas and try to develop a little sample to show you how this app will work on your computer. After that, we can chat about the price and timeline, could be? Send me a message over here, I will help you. Thanks again for this invitation. Jorge.

$250 USD in 9 days

5.0

(206 reviews)

7.4

@uumairkhalid

Hi.. Expert Web Scraper & Data Minor here. I have done too many similar project in past. Having best scraping tools and experience i assure you 100% accurate and good quality work. I have too too scraping experience. you can see my experience here: https://www.freelancer.com/u/uumairkhalid.html looking for your reply. Thanks

$333 USD in 10 days

4.9

(168 reviews)

7.0

@uumarkhalid31

I specialize in web scraping jobs like this one. You can see on my profile that I have completed many similar jobs. Don't make the mistake of hiring an amateur, who doesn't have the skill set to properly complete this task and will only waste your time. Hire me, a professional with a rock solid reputation and over 8 years of experience specifically in writing scraping code like this. I know what I can and cannot do, and I won't waste your time by saying I can do something that I can't. because I exclusively write scraping code I know what to expect and I will always live up to what I tell you, and will also deliver a higher quality final product than my competitors.

$277 USD in 10 days

4.8

(251 reviews)

7.2

@diamond247

Hello Sir, We are a team (19 operator and 2 Quality checker)here from last 4 year giving all research service world wide with best quality output , I have gone through your project description, It is really a interesting job, and our operator are experienced enough in research skill so they easily can collect the data from several source, from a deep investigation, but its bit time consuming job not a copy paste. We would like to talk in details and give the total structure about how we ll do this job if you need. Waiting for your reply. Thanks Dg

$350 USD in 10 days

4.5

(134 reviews)

6.5

@Harun1986

Dear Sir, very first within short time .Experts and experiments team here all kind making coding for web scraping also Manually Collect Data Entry Work. Please check my profile. I have done similar projects. I know some beater knowledge how to scraping website companies, Email, address, Person name, What you want So I can do the work acquired perfect in time. Please see first my work sample and if you like my sample then award me. Waiting for your reply. Thanks

$555 USD in 0 day

4.7

(76 reviews)

5.9

@Sunita0610

i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.i can do your project please discus your project.

$555 USD in 10 days

4.7

(54 reviews)

5.6

@edb38

Hello, I am a professional Java programmer. I am new to this site but have written several tools for web scraping so I am comfortable with the technical requirements. I have recently written a web scraping tool for a previous client that scrapes over 1000 pages from a website and downloads to an Excel spreadsheet. I have analysed the links below. I can build you a robust tool which allows you to specify the sites to scrape and then extracts the required information and generates a separate file for each page as you request. My higher fee reflects the cost of providing you with good quality software which will easily scale as the site grows and more pages are added. Look forward to hearing from you. James

$500 USD in 4 days

5.0

(1 review)

3.1

@zhuying1022

Hello. I am zhuying I am very glad to meet this job because it's just suitable for my ability. I have some good experiences in webscraping. As you can see, I have some estimate of web scraping on freelancer and working now. I will do your job with my team exactly. I have a confidence to do your job. Your requirement is perfect. Myskp ID is lzy_assassin. I wanna talk with you on via skp or live chat on here. We are ready to start right now. Thanks for your job post. Warm Regards.

$555 USD in 10 days

0.0

(0 reviews)

2.7

@mobileappdevin

The Scrape approach as follows: Technology: * nodejs (using cheerio plugin) * datastore mysql * scrape manager: Django (python) Approach: Initially when scraping the Data Table with pagination, the script would get data details url. While visiting data details url we can get various info about the song. Next steps would be visiting details for those entities to get the details info to be scrapped. It would be great if we get the name of the website to create a more detailed and specific approach, as the above one is very general. Waiting for your response. We have done similar jobs as well on products & image domains. If you permit we can show you as well. Thanks & Regards.

$627 USD in 20 days