I have an Instagram Reels scraper that works fine, but for some reason my Autobook into Notion duplicates a lot of rows. The duplicates do not happen in the scraping step, when looking at results - so it might be Notion. Also, the duplicates seem to happen more on profiles with more posts - 20 posts works fine usually, but 80 posts duplicates around 30%.
Because of this, I want to create an Autobook that “finds” the Notion database and then either:
Checks if it doesn’t exist, and adds it
OR checks if it does exist, and skips it
I’ve tried lots of different things the last few hours, including a condition - however the condition functionality doesn’t seem to work at all. I tried running it while only swapping the “Yes” and “No” and not changing anything else, and everything still went through the filter both times.
I’ve tried using the AI to assist in creating different flows, but nothing actually works.
Normally a combination of “Find Notion pages” and a condition should work well for this case. I think there may be a misconfiguration in your playbook, would you mind sharing it with me?
On the other hand, I am thinking what is a duplicate in this case, are those identical reels with the same author and description or maybe the same video posted by many people?
For the definition of the duplicates: it’s simply the URL to the reel.
In this case, in the Notion database, it’s the Title/Heading column I’ve named “Reel”, which is this URL (first column in the attached image). For example:https://www.instagram.com/reel/C7ZibJ-NO-0/
(I have another “Reel URL” property in the Notion database, but it’s only been used for testing)
I’m trying to match the “Reel” Notion property (which is the URL I explained above) with the “Post URL” from the Bardeen scraper for the duplicates.
Both of these work well in isolation and produce the correct results.
I think/hope it’s set up correctly like yours, but it still adds duplicates to the final database. 87 reels became 134 entries, and 95 reels from another profile became 214 entries
Also, I’m not sure if I really need the automation, because the Notion databases get so large with hundreds of reels and they start being slow to respond, etc.
I can do what I need to do without the automation anyway, and I already have and use a lot of other Bardeen automations
What if you try with a new database to make sure there’s no existing duplicates? I checked your playbook and can confirm that it should work the same as mine. Also I tested mine once again and did not get any duplicates.
Also, could you try restarting Bardeen in case if something got cached?
