Hi I am trying to extract information for mergers and acquisitions from this website:
The website follows a scroll pagination, and after a point it has a click pagination. Since bardeen doesn’t allow multiple pagination types in a single playbook.
How can i create a pagination one after the other to solve this ?
Hi @akshaykriplani258, Welcome to the Bardeen Community!
Pagination on this site is actually the “Load More Content” button here:
Simply click pagination and click this button when creating your scraper template to accommodate this piece. Additionally, it’s best practices to include a delay inside of your scraper template after grabbing all necessary data per merger/acquisition to allow time for the page to load as Bardeen scrolls the page.
Give this a try and see how it works for you.
Please let us know if you have any questions.
Thank you,
Jess
Hey Jess! Thanks for the response.
I did try the click pagination as well as the infinite scroll pagination.
I kept the delay to 3 seconds but it still doesn’t work. In fact the data is not getting extracted beyond the first scroll.
The “load more content” button is way down but the data is not being captured till the article on 4th December.
I was able to pull 75 links with this method:
Make sure to put the delay inside of your scraper template like this:
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.
