[v2.0] Problem with scraper not pulling all Tweets

Hello. I’m new to Bardeen and have been utilizing the “Extract Tweets from page” playbook toward collecting data about tweets in specific date ranges. Such as: [July 1st, 2024 – to July 2nd, 2024];
It works rather well, which I’m enthused for;

I’m running into the problem highlighted here:

Word for word, I’m in the exact same scenario;
I’ve tried precisely what the original poster attempted, and am having the exact same problem, where it lists maybe 60-70% of the total tweets, as it pertains to collecting data from specific date-ranges.
( particularly if the results contain photos/if the tweets for the specific day surpass 20 in number );

• Luckily, a Bardeen staff member acknowledged the issue, and made a video on how to create a playbook which solves the problem, and seemingly scrapes 100% of the tweets from the Twitter advanced search queries. ( in the above highlighted question )

• Unluckily, the video is now gone from the staff member’s post :expressionless:. The Bardeen staff member hasn’t been active for ½ year.

Is there a way to recoup said video-guide on how to build a playbook that scrapes the totality of the Twitter Advanced Search -results? ( per 1-day timeframe, for example )
Is the staff member’s playbook perhaps available?

Apology for the weekend post. Thank you.

Hi Patricia,

Unfortunately there’s no way to restore that video or a playbook. But I must say that our scraper is improved so the video must be outdated.

I would like to take a look at the problem again and find a new solution for you. Would you be able to share the playbook you tried and an example link for scraping?

Looking forward to your reply.
Victoria

Customer Support - bardeen.ai
Knowledge Base https://support.bardeen.ai/hc/en-us
Explore | @bardeenai | Bardeen Community