How to deal with unexpected popup on webpages I am scraping?


My scraper is working fine , however, the website randomly display a popup window from time to time.
when is happen, scraper will break and display an error msg: “Scraper returned no results. Scraper hasn’t returned any data. Make sure that you are using the correct scraper template on a web page with data”

is there anyway to overcome this? my workaround is to reduce the number of pages to scrape so that it is not long enough to encounter the popup. which is frustrating as I have to keep restarting the scraper…



Hi @alandeyong sorry about this issue you’re running into!

Can you share us details on the automation you’re using so we can try to reproduce this?

Also, can you share the error message if you get any?
Share the error

Tagging @manvel so we can check this!

Hey @alandeyong that’s probably due to the page structure changes when website shows that popup, one way to overcome this will be to create a custom scraper selectors.
Here is a tutorial on basics:

But creating custom scraper model may require some knowledge around CSS.

If you can share the website with us you are trying to run scraper on, we will be able to help you building custom model that we will ensure works when the popup is visible on the page. Unless the popup is a access wall and it prevents from data on the page to be accessed.

I did attempt to clean up the selectors using custom selectors as it was not working well by default when they were auto detected by Bardeen and got rid of the erroneous ones.

The website is

The popup Google ad applied an overlay over the page I am scraping when it shows up. I need to click the “close” button to close it. However, it seems that even if I close it in time before the delays (I added) runs out, the scraper will break. I have attached a screen grab of the popup.

An idea came to mind but I have gotten the change to check whether it’s feasible:

  1. Is there a error handling trigger I could use, so that when the automation detected unsuccessful scraping (due to the error) it can be restarted from the last page?



From what I see the ad popup page is actually completely different, when you click close a new page content is being loaded. Most probably this is due to the fact that page also clears up Bardeen related content from it causing the issue. I can see 2 solutions here:

  • Use click action in Bardeen before any action you have and follow it with the delay action before scraping fields.
  • Alternatively you can use Adblockers which suppose to block ads on the page.

I will try. where do i find the click action ?

I couldn’t find the Click action…

You can find it under “More options”, here is a demo:

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.