Can I scrap a page when the URL doesn't display directly the elements I want to scrap

Hi, I have an issue with Bardeen.

I’m trying to scrap the metrics of my email campaigns.

I have a screen where I can load a list of these past campaigns (on a selected time period).

So I first have to select the begin date, end date, and then click on “afficher” (display) to display the campaigns I want to scrap.

I should look like this :

So I’d like to create a deep scraper with Bardeen that’d click each email link, and then scrap the relevant metrics.

However when I try to create a scrapper template in Bardeen and I enter the url, it loads the screen “before” the timeperiod have been selected and the campaigns displayed, so I can’t select the right selector since they don’t appear.

Here’s what it shows :

I should click on “afficher” to display the campaigns but I can’t do that on the bardeen selector screen, so I’m stuck.

Is there any way to display (and scrap) those campaigns if they don’t display directly from the URL ?

Hi @pierre.lacouture

You can do this on Bardeen’s Create Template screen by selecting the “Click” option after selecting “afficher”:

I recommend after this action to add a special field of the “Delay” action of like “.4” as the input to allow the page to load the results and then start scraping the email links.

I hope this helps!
Thank you,

Hi Jess,

Thanks a lot for your reply, but I just can’t find it…

I open Bardeen
Scrapper > new scrapper template
Enter the URL
Scrap table or list

Then here’s what I get :

I see no options to click anywhere, what am I doing wrong ?

Thank you

Is this a public page, could you send the link?

Okay so you are using a “Table or List” scraper (which is what you’ll need to scrape this page), but the wizard step you’ve screenshotted above is just asking you to locate an element from the list.

  • There is no list of results shown at the moment, so just click on any two elements.
  • Later, we’ll need to update the css selector of the container to grab the right list. (If this isn’t a public page, I cannot help you with this and it might be better to schedule a call with someone at Bardeen to help you further)

After you click on any two elements, you’ll be able to use the “click” action on the button labeled “afficher”.

I hope this helps!
Thank you,

Thanks Jess,

Unfortunately it’s not a public page.

And even if I click on 2 random element first, then select “click” on afficher through Bardeen, once the page loads the campaigns, Bardeen is no longer active.

It’s a shame Bardeen doesn’t give you the option to scrape the active tab, that’d be faster and would avoid these kind of issues.

I think I’ll have to find a tool that do it.

Anyway thanks for trying, I really appreciate it.

Have a great day.

Bardeen does have this ability through the “Scrape on active tab” action in a playbook. (Sorry, I didn’t realize this way was what you were looking for)

Yeah but that doesn’t fix my problem.

Sure I can scrape the active tab, but I can only scrape using a template and template can’t be made on an active tab.

So yeah, I should have said : shame Bardeen doesn’t let you create a template on the active tab.

(unless I’m missing something)

A template can be made on an active tab :+1:

Let me show you

Click Here:

Then select your active tab after you’ve filtered (hit “afficher”)

Hey Jess,

Thanks again for your help. It helps a lot.

Hope I’m not being too annoying but I have another issue down the line :

I was looking at Bardeen’s video on youtube on how to create a “deep scrapper”.
In the video, there’s a first scrapper that scrap infos out of a list (on linkedin in the video), including the URL of each profile .
Then you create a second scrapper that’s gonna open each URL scrapped and scrap even more informations from each profile.

My problem is that in my case, I can’t extract a list of url with the first scrapper, I have a “javascript: dopostback […]” link instead like this :

One way around that could be to ask Bardeen to click on each links inside the column “Mailing” and scrap infos on the next page, but is it possible ? I haven’t been able to do it so far, when I select each element inside mailing and ask for click, bardeen tonly consider as a unique element instead of a list of element…

To scrape lists or tables, you need to use a “List or Table” scraper template.

When you create a scraper template, Bardeen asks you to choose which type of scraper - either a “single page” or “list or table”. The page looks like this:

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.