So I’d like to create a deep scraper with Bardeen that’d click each email link, and then scrap the relevant metrics.
However when I try to create a scrapper template in Bardeen and I enter the url, it loads the screen “before” the timeperiod have been selected and the campaigns displayed, so I can’t select the right selector since they don’t appear.
I recommend after this action to add a special field of the “Delay” action of like “.4” as the input to allow the page to load the results and then start scraping the email links.
Okay so you are using a “Table or List” scraper (which is what you’ll need to scrape this page), but the wizard step you’ve screenshotted above is just asking you to locate an element from the list.
There is no list of results shown at the moment, so just click on any two elements.
Later, we’ll need to update the css selector of the container to grab the right list. (If this isn’t a public page, I cannot help you with this and it might be better to schedule a call with someone at Bardeen to help you further)
After you click on any two elements, you’ll be able to use the “click” action on the button labeled “afficher”.
And even if I click on 2 random element first, then select “click” on afficher through Bardeen, once the page loads the campaigns, Bardeen is no longer active.
It’s a shame Bardeen doesn’t give you the option to scrape the active tab, that’d be faster and would avoid these kind of issues.
Bardeen does have this ability through the “Scrape on active tab” action in a playbook. (Sorry, I didn’t realize this way was what you were looking for)
Hope I’m not being too annoying but I have another issue down the line :
I was looking at Bardeen’s video on youtube on how to create a “deep scrapper”.
In the video, there’s a first scrapper that scrap infos out of a list (on linkedin in the video), including the URL of each profile .
Then you create a second scrapper that’s gonna open each URL scrapped and scrap even more informations from each profile.
My problem is that in my case, I can’t extract a list of url with the first scrapper, I have a “javascript: dopostback […]” link instead like this :
One way around that could be to ask Bardeen to click on each links inside the column “Mailing” and scrap infos on the next page, but is it possible ? I haven’t been able to do it so far, when I select each element inside mailing and ask for click, bardeen tonly consider as a unique element instead of a list of element…
To scrape lists or tables, you need to use a “List or Table” scraper template.
When you create a scraper template, Bardeen asks you to choose which type of scraper - either a “single page” or “list or table”. The page looks like this:
