Hi, is the expected behavior of the IG Followers/Following template scraper to only scrape what is visible in the viewfinder of the /followers URL? I’ve seen several people post about this limitation, and I’m wondering if this is a bug or just how it is.
I attended a Bardeen scraping workshop where we were taught to add a 0.2 Delay to the list gathering elements to force it to load. I built my own IG Followers scraper with the 0.2 delay, and even with that, it still didn’t load/scrape my full list of followers. I increased it to 0.3 and had better results, but not all, and then with 0.4, it seemed to time out and not do it.
Is there something more we can be doing? The goal here is to scrape my full list of followers from the /followers url. It seems like the only workaround/solution right now would be to scroll-scroll-scroll until the full pop-up box of followers is loaded, and THEN run the scraper. Is that correct?
I’ve tested the playbook above asking it to scrape about 300 followers and it works for me. You definitely shouldn’t have to scroll down for more results as the scraper can do that.
Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community
It seems to work, but the amount of data scraped is inaccurate. The person I was testing it on had 593 followers but it ended up with 600+. For the following, it ended up with 40/341. Any way to fix this?
Hi Vin, yes I was using the prebuilt scraper you provided. I just ran it again (using the link you (provided) and it scraped only 131 results. My follower lists is over 2k that I want to scrape. It doesn’t successfully scrape this full number.
This is why I tried my own scraper with a delay built in, because it seems the prebuilt template is not capable of loading and scrolling the full IG followers pop-up box. I have seen 3-4 other posts on this forum of others expressing the same frustration (that the prebuilt scraper doesn’t scrape the full list, only a subset: what is loaded in the view window).
So what I’m trying to understand is will this be fixed with the pre built scraper template (IG Followers) so it is capable of loading and scraping my full list of 2k followers ? Or is it only built to scrape what is viewable in the window? ( today’s results of only getting 131 followers )
To me, this scraper template is only useful if it can scrape my full list, and currently it cannot.