Selenium/BeautifulSoup script to crawl/fetch/output a website's data from URLs
$30-250 CAD
Closed
Posted about 1 year ago
$30-250 CAD
Paid on delivery
I need a script that will work with Store URL inputs (check example below) and output entries into a file.
Once per day (about 500 URLs)
The page/URL has a product list and we need to find & output "product name && badge color" for any lines with the following keywords: Nugz/Tribal/Orchid
The ouput needs to have these 4 informations.
City-Name | Store-Name | Product Name | Badge Color (indicates stock | green/yellow/red)
Here's an example:
Input:
[login to view URL]
[login to view URL]
Output:
Ottawa, Ontario | Tokyo Smoke (Ottawa Rideau Centre) - (50 Rideau Street, Ottawa) | Tribal - Cuban Linx | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Tribal - Terple Live Resin Full Spectrum Extract | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Tribal - Cuban Linx | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Tribal - Galactic Rntz | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Tribal - Power Sherb | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Tribal - Terple | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Tribal - Power Sherb Pre Roll | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Nugz - Early Lemon Berry | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Nugz - Slapz | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | Nugz - Smalls | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | NUGZ FARM - Hybrid | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | NUGZ FARM - Old School [2g] | Green
Ottawa, Ontario | Canna Cabana - Ottawa - Rideau St - (111 Rideau Street, Ottawa) | NUGZ FARM - Fresh Frozen | Green
Please let me know if you're confident you can have this done within a week.
Hello! I'm George, and I am an experienced web crawler. I believe I can provide the service you need, and that I can have it done within a week.
I've been working with web crawling and web scraping for the past five years, building custom scripts and using Selenium/BeautifulSoup. I'm familiar with the field, and believe I'm the perfect fit for this job. I think it's a great opportunity, and I'm confident that I can help you find the data you want.
I already have a custom web scrapping tool I built which I feel will work great for your needs. I'd love to hear more about the project if possible; I have a few questions that should help us move forward.
1. How often will we have to update the database?
2. Do you have a specific format required for the output file?
3. Do you have any statistical metrics needed to be output with the data?
4. What target website(s) will we be working with?
5. Are there any other details I should be aware of?
Thank you for considering me for the job! I look forward to hearing from you soon.
Top 1% in Freelancer.com
Hi, Greetings!
✅checked your project details:
✅Completed Time: In project deadline
We have worked on 850 + Projects. I have 6 + years of the experience in same kind of projects. I am available almost 24-7 and am very responsive.
My Portfolio Items: https://www.freelancer.com/u/schoudhary1553
Kindly contact me via chat for further details and discussion.
Thank you
Sandeep
I am Python developer and have excellent experience in Selenium/BeautifulSoup and i can make Selenium/BeautifulSoup script to crawl/fetch/output a website's data from URLs and I am pretty sure that I can complete every bit of your requirements. Further details and cost can be discussed
Hi,
I have experience scraping content using python scripts making use of Beatutiful Soup and Selenium. I can write a python script that reads the urls from a csv file, then fetches the content, parse them to extract the four required fields and write them to another csv file.
Regards,
Sreekumar
Hi There,
I am Python script developer with 5+ years of experience. I can develop web scraper to scrape your required website by Python script/Bot with your instructions .
Can we discuss please?
Thanks & Regards
Pooja Bohra
"Hi, I am a scraping expert. Can scrap any website.
Scraped over 1200+ Websites/Sources. Skills: PHP, Python, Node
Please message me for further discussions.
Thanks!"
Hello OakandRocks,
I will work on this project until I get good results and complete all the tasks.
I am prepared to handle projects requiring Data Mining, Python, Web Scraping skills.
Lets connect in chat so that we discuss further.
Regards
Hi,
Iam data analyst and web scraping specialist.
I have excellent experience in webscraping using python requests, beautifulsoup, selenium.
I can provide the script working within 2 days.
Please contact me to discuss further details.
Thanks
Hi there!
I am a proficient python developer. I've a lot of experience with scraping data.
I can make a script like that for you!
Message me and we'll discuss your project!
Sincerely,
Andrii
Hello
I am a professional python developer. My main specializations are automation, web scrapers and bots development.
I have already developed over 200 scrapers. From the simplest (for example, a competitor's price collector) to complex parsers (with authorization, bypassing captcha, rotating ips and others) which can collect millions of products from amazon.
I have done web scrapers for:
- Amazon
- Instagram
- Facebook
- Google
- Twitter
- LinkedIn
- Pinterest
- Walmart
- And many others
For scraping I use:
- Python
- Requests
- BeautifulSoup
- Selenium
- Scrapy
- Pyautogui
- Undetected Chromedriver
- Rotating ips
I can bypass:
- CloudFlare
- IP blocking
- Captcha
- Authorization required
- Other limitations
Django / PostgreSQL
For big scraping projects I usually use Django with PostgreSQL. This allows us to store information in a database for further processing and use. I also set up an administration area which allows us to check the data and set up scraper configs.
If you need a professional solution in this area - I am ready to cooperate.
I am ready to make a sample script before we start
Regards, Oleg
Hi I can make you a python script to scrape City-Name | Store-Name | Product Name | Badge Color (indicates stock | green/yellow/red) into tx file. Please let's chat and get started. Best regards!
Hello,
I'm experienced in web scrpaing using Python's Beautifulsoup and Selenium libraries.
Pls chat to discuss all the details.
I look forward to working with you.
Best regards,
Mohamed
Hi, I have read your project carefully. I think I can finish it in seven days. Just need to load the URL list file and crawl each page, I can write it in Python very quickly and with high quality.
Hello,
I have a lot of experience scraping stores and your project looks fairly simple. I could get it done easily on the weekend. The URLs you provided and any future ones will all follow 1 pattern luckily so even though it is 500 URLs, it will actually be a very simple process. I will provide you a Python script that will scrape all the URLs and output to csv. Please let me know if you have any questions via private message.
Thanks,
Grzegorz
Hola, Gracias por compartir el requisito de su proyecto. Leemos su requerimiento y lo entendemos. Para entender más y aclarar nuestras dudas, queremos hablar contigo. Por eso, envíanos un mensaje para que lo analicemos en detalle y te demos un presupuesto exacto y empecemos a trabajar.
un gusto, tengo 4 año de expeciencia en python y tengo alta expeciencia en raspado web con python, http client, selenium y requests usando BS4-Beutifullsoup, me gustaria ser parte de este proyecto y poder ser la persona que se lo desarrolle.
Hello. I am a experienced web developer and I have solid experience in various kinds of web scraping.
I'm interested in your project, please let me know your details, we can discuss more closely.
Thanks.
Hi there,
I have read your request and accept it. You can get it from me for a very low cost!
I will put all of my knowledge and expertise to use to assist you in finishing this job in two days (or a maximum of four days if you have any additional requirements, adjustments, or concerns about it).
So, let me help you!
