[Urgent] Top posts data scraping - issues

Hi,
I’d really appreciate an expert’s help here.
I’m trying to automate scraping of data of top posts from instagram, tiktok, and Twitter (yes, all 3 platforms).

There are two steps to this.

  1. organize all posts/reels by most viral (most views) on top, then in descending order
  2. scrape the following data for the top 20 posts
    *** If sorting the posts by view count is not automatically possible, then my alternative wish is to provide a list of URLs (in airtable), then have Bardeen scrape the data for each of the posts provided in that list of URLs.
    I wish to collect the “view count” of each post, which seems to not be part of the Bardeen-built scraper for posts. I also couldn’t locate this ‘view count’ number visible on website, so it’d have to be somehow worked on.

other list of data I’d like to scrape for each post:
URL, view count, like count, comment count, caption/description of the posts, reels, tiktoks, or tweets.

If this has to be done in multiple steps, please advise as such!

Thank you.

Hi there,

Thanks for reaching out. Here are some bits of feedback in terms of what is and isn’t possible with reference to your asks:

  1. Sorting posts by views is not possible as the respective platforms don’t have this as an option. If you’re able to get the links in an airtable, that would be better.
  2. With instagram and tiktok, you won’t be able to scrape the view count as this isn’t made visible when you open up the reel/post. You can however scrape any other metrics that are made visible (number of comments, likes etc.). I’ve included a short demo below of how you would build the playbook with an output to either google sheets or airtable. You will also likely need three separate (but very similar) playbooks depending on the platform you want to scrape
  3. If you would like to build on one of our existing scraper templates, you can always click on the little pen next to the scraper template name and it will open up an editor.

Demo:
2024-02-02_12h34_54

Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.