Scraper stuck in an "endless" loop - Pagination

[Automan3000]

I’m trying to scrape a tables that are spread into multiple pages. The problem is that when the scraper scrapes the last page it presses “previous page” -button and scrapes the second last page again and then continues to the last page once again and it keeps repeating this loop again and again until I shut the scraper down. The first page of the tables only have “next page” -button and the middle ones both “previous page” and “next page” -buttons and the last one only “previous page” -button.

Inspect elements from the mid pages (edellinen sivu = previous page, seuraava sivu = next page):

What I understand is that scraper get’s confused at the last page because the “next page” code is missing. I tried to fix this with the advanced edit click pagination, but it’s too advanced for me I guess that having the value=“seuraava sivu” in the selectors would fix the issue.

Here are the selectors the scraper made:
TABLE#next-prev-bottom.pagination:last-of-type > TBODY:first-of-type > TR:first-of-type > TD.more:last-of-type > FORM:first-of-type > INPUT.submit:last-of-type

TABLE#next-prev-bottom.pagination TBODY TR TD.more FORM INPUT.submit

*#next-prev-bottom.pagination > * > * > *.more > * > *.submit

TABLE#next-prev-bottom.pagination > TBODY > TR > TD.more > FORM > INPUT.submit

TABLE#next-prev-bottom:last-of-type > TBODY:first-of-type > TR:first-of-type > TD:last-of-type > FORM:first-of-type > INPUT:last-of-type

TABLE#next-prev-bottom:last-of-type > TBODY:first-of-type > TR:first-of-type > TD:last-of-type > FORM:first-of-type > INPUT:last-of-type

I would appreciate any help in this matter.
Thank you!

[Jess]

Hi @Automan3000, Welcome to the Bardeen Community

Are you able to share the link for us to inspect the page on our end?

Thank you,
Jess

[Automan3000]

Hi Jess,

of course, here is an example: Asuntojen hintatiedot

Thank you!

[Jess]

Hi @Automan3000, I took a look and found that the below selector for the pagination works for me to return 248 records:

#next-prev-bottom > tbody > tr > td:nth-child(3) > form > input.submit

Could you please try updating the pagination selector on your scraper template with the above and rerunning the automation on your end to see if it works for you?

Thank you,
Jess

[vin_bardeen]

I love these complex use cases using Bardeen ! Thanks for sharing it here as it really helps our community members who might face similar situations. Please reach out if you don’t manage to get it going

Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community

[Automan3000]

Hi @Jess , this worked perfectly. Thank you very much!

I have an another question about he credits. I’m planning to upgrade to the professional plan and I’m wondering what does the premium scrapers 1 result = 1 credit actually mean? Does the 1 result mean running the scraper once, or scraping one page or what?

[Jess]

Happy it worked for you!

I believe it is the number of results returned from the scraper. I’m not positive on this so I highly recommend reaching out to support on a separate topic or via email to confirm.

———
I’m a brand ambassador for Bardeen, here’s more about me: 🪄 Ambassador: Jess Van Wyhe | Personal Productivity

Thank you,
Jess