Forgive me I’m a beginner. I’m building a playbook that will go to a Realtor Broker website to get a list of agents. I use a URL that Bardeen will use to get a list of agents . Each agent has a profile that you need to click on to scrape the data I need i.e. name, phone, email and etc. However, my problem is how do I go back to the first page (the list of agents) to click on the next profile to continue automation? The agent profile webpage doesn’t have a back button on the page itself.
My playbook is getting the the first agent on the list and scraping multiple times.
Welcome to the community and I’m so glad to hear your first use case is getting straight into our deep scrapers.
Here are the steps you’ll need to follow:
Assuming the agent list is in the active tab you’ll need a Scrape data on active tab action and then you’ll have to create a scraping template to scrape all the links to all the agent profiles. In this step, Bardeen will scrape all the links to the agent profiles
Next, you’ll need to add a scrape data in the background action to scrape each individual agent’s page. The input for this action will be the link to the agent’s profile you scraped in the first action. You’ll also need to create another scraper template for this action to scrape the specific information you want for that agent’s profile (name, phone, email…).
Finally, you’ll need somewhere to output the data. Add an ‘Add rows to Google Sheets’ action to and map the results of the two scraping actions into the google sheet columns.
Check out our Ultimate Scraping Tutorial that covers how to scrape in the active tab and how to scrape in the background (deep scraping)
Cheers,
Vin
Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community
To understand the credits on this. Assuming the steps below work for my use case, if I had a list of 500 records to scrape in Step 1. Would the cost of this be 1000 credits? Since the second scraper in step 2 needs to be run on the same 500 records?
Thank you
On Friday, December 8th, 2023 at 3:51 AM, Vinohar Kumar (Support) support@bardeen.ai wrote:
It will likely cost you only 500 credits. Scraping in the active tab (which is the first step) is not a premium action and won’t cost you any credits. Scraping in the background (second step) is a premium action and will cost you 1 credit per record. Do note that all the actions I described above will need to be in the same playbook.
Here is a breakdown of actions that cost credits - https://www.bardeen.ai/pricing (FAQ → What’s a credit ? What can i do with credits?)
Before running the playbook, if you hover over the playbook once built, you’ll see the number of credits to be used per record.
Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community
