I am trying to scrape each Business in the directory at this site: https://www.seia.org/directory
I setup an a new template scraper, selected âList or Tableâ, then selected the entire group of data for the top company to select the list and then selected the group of data for the second company for the second request to find the list. After this it prompted me to select âclick Paginationâ and I clicked the âNextâ button. I then selected each of the each type of data I wanted from the list.
After I save it, I did not add any criteria to number of lists or pages and I set a 2 second custom delay.
When I run the scraper it scrapes the 5 companies correctly on the first page, goes to the second page and gets the first company, then goes to the third pages gets the top company and finally goes to the 4th page and gets the top company. It never returns more than 8 rows though.
Any ideas on how to fix this?
Thanks.
Hi,
Here is a playbook that should do the job: Shared Playbook Template
I ran it for a bit and was able to scrape as many listings as I needed without issues:
Please let me know (artem@bardeen.ai) if youâd like us to help you with hands-on support and build custom playbooks like this one.
Artem.
Thanks for the help, but when I run your Scraper it still only comes back with 8 entries. why do you think I am being stopped after 8 entries?
Hm, are you certain you ran a playbook that I shared? I renamed it to be sure. Can you please visit the link, then Pin the playbook (see the screenshot), then run it again please?
Customer Support - bardeen.ai
Knowledge Base https://support.bardeen.ai/hc/en-us
Explore | @bardeenai | Bardeen Community
Hi Artem,
Yes, I ran your playbook the first time and it only returned 8 results. I just pinned this one and ran it and it also only returned 8 results. Any idea what the issue is?
Thanks,
Jeremy
Any update on possible issues?
