Clicking expand tab within page before scraping

I need to run the following sequences for each pages of listing per region, before retrieving deeper to the details page. Given a listings link, the steps need to grab the list links are:

  • Click “View more” to expand listings beyond a small number
  • infinite scroll to the bottom
  • scrape all the items

During setup, I can get it to populate the fields, though its kind of messy between clicking “View more” and infinite scroll. But i have not been able to get it to run, unless i remove the click functionality, which I need for the full listing.

click / scroll issue

Hi @craigdoriot, Welcome to the Bardeen Community :blush:

Instead of using the “infinite scroll” option when creating your scraper template, you need to use the “click pagination” option and then click the View more.

A friendly reminder to add a 3-4 second delay inside of your scraper action within the playbook. This is best practices and allows the page to fully load before the scraper executes.

Give this a try and let us know if you run into any questions!

Thank you,

Awesome, this worked!

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.