I am trying to scrape a list of links on google search. The Google search is about 14k search results long but I am only able to get the Bardeen scraper to automatically scrape the first 190 results. How can I solve this?
Hi @skye,
Sorry to hear it isn’t pulling out all the results for you. It is likely that you’ll have to set a custom delay which allows the page (and following pages) to load before the scraper starts scraping. Typically we advise to put 2/3 seconds on each scraper.
Hope that helps but if you’re still facing issues, it would be great if you could share a link to your playbook so we can look into it.
Doesnt seem to be working. Here is my playbook: Shared Playbook Template
Hey @skye , we’re checking this with the engineering team and will get back to you shortly.
Hey @skye, can you please send me a link of the page you’re running this automation on? And also please send (if possible) an HTML page (you can do it by right clicking on the mouse → Save As...), that will help us analyze the problem with the layout.
Here is the link to the search.
Here is the Html download link
Sorry not sure if those came through.
Link to search: site:boards.greenhouse.io/* "Internship" OR "Intern" - Google Search
Tried to attach the html file in the community browser and it says file type not supported. I attached it via email in the previous message but not sure if it worked
You can try to compress the file into a .zip format and send it here 
JUst tried and this is the response I got: Sorry, the file you are trying to upload is not authorized (authorized extensions: jpg, jpeg, png, gif, heic, heif, webp, avif).
Is there another way to send you this info?
@skye I’ve PMed you, if you could email it in to us, we can have a look at it.
Just emailed it to you!
1 Like
Thanks Skye. I’ve shared it with Dan and he’s looking into resolving this for you.
Hi Skye,
First and foremost, I want to extend our sincerest apologies for all the inconvenience you may have experienced due to the recent bug in our software. We understand how frustrating such issues can be, and we appreciate your patience and understanding as we worked diligently to address it.
I am pleased to inform you that our development team has successfully identified and resolved the bug that was causing the issues you reported. The fix is now available with a new version of Bardeen (2.37.2) which addresses the issue an issue with background scraping in the previous version. You can check your current version of Bardeen in Google Chrome by following the steps in my screenshot below (the version number will appear next to the Bardeen extension) as you might still be on the old one:
Here is a quick guide on how to update your version of Bardeen: How to Update Bardeen
Thank you for your patience, and if you need assistance, email me directly lucy@bardeen.ai.
Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community
Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community
I just tried this and unfortunately it didn’t work 
It looks like it was able to scrape further down the search results but still only yielded about 50 results in the sheet after I got this error:
Is there a way you know of to resolve this?
Hi Skye,
It looks like it might be an issue with your network or the scraper not being able to see any results on the page. Our engineers were unable to replicate the error you are having but suspect the fix might be for you to set a longer custom delay on the scraping action (we recommend 3-5 seconds) so the page has time to load before the scraping starts.
Please let me know if that doesn’t work for you.
Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.