Deep scraping after monitoring

Hi!

Sorry, me again with another question.

I want to monitor a list (step 1) and then once one list item is added deep scrape the added link (step 2). In step 1 I have opted to use the URL as unique identifier:

In step 2, i get offered this choice. My question is: what is the difference between these choices?

Is there a tutorial about deep scraping in combination with monitoring?

–Bob

Hi @Bob ,

From what you are sharing, I suggest selecting the entire list item for monitoring. This way you will be notified when that list changes (a link is added). Not necessarily that you’ll be able to get a link for step 2 right away, but at this point you need a trigger only.

Step 2 is an actual scrape for fetching the added link. You might need to use a page with that link shown to configure a scraper properly.

And step 3 will be another background scrape of the link from step 2.

Regarding your questions about the screenshots. To assist with that I would need to see the website and your automation. That all is fetched from the structure of a website and I don’t see it on my end unless you share a playbook.

Victoria,
Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community

okay now I get it. I thought step 1 was already scraping the website if changes occur. It makes sense now. ITs a three step process: 1 monitor change, 2 scrape URLS added to list, 3 deep scrape URL web page. I was missing step 2 and that is why my scrapes where empty. Thanks, this helps a lot.

Hi @victoria,

So sorry im going Baby Reindeer on you with all my questions. I just cant work this one out on my own. Any help will be really appreciated. :pray:t2: Would you mind taking a look at this playbook and tell where I go wrong:

What I am trying to do:
Step 1: Monitor when concerts are added to the concert agenda
Step 2: scrape the added concerts on the concert agenda
Step 3: Deep scrape the URLs of the added concerts.

I dont get how to link step 1 and step 2. Should “Links to pages to be scraped” in step 2 refer to the URL of the concert agenda or refer to step 1? Perhaps Im missing a For Each action?

Any help would be really appreciated!
–Bob