How to scrape a page within a page within a page

Hey,

I’m trying to scrape a FB group for its members, but I want to get additional data for each member.
for this I was able to get the list, do some manual manipulation to get the actual personal profile links (I did it manual, but is there a way to manipulate the strings in bardeen? A.K.A. parse, edit and rebuid?) but where things get messy, is where I tried scraping the profiles from the list.

What I’m trying to do is:

  1. get the link to a profile from the list (google spreadsheet)
  2. for each link then:
    2.1. get the about page link
    2.2. inside the about page get links to 2 other pages (contact info and places lived)
    2.3. scrape data from the 2 inner links
    2.4. sum the extracted data from all the pages (the profile and both inner pages) to an additional google sheet.

Is there a way to do this?
thanks in advance.

1 Like

Hey @avinoamkam! Welcome to Bardeen :cat_jam:

Congrats on your creative use-case for scraping a Facebook group and gathering additional data for each member.

In Bardeen, it is indeed possible to scrape pages within pages using the same logic as “Deep scraping” that we teach in our tutorials.

You can start with a single page scraper to obtain the links to the about page, and then create a “scrape data in the background” action to extract data from the about page and the two inner pages (contact info and places lived).

You can use multiple scrapers in a single flow and combine the extracted data into a single spreadsheet row.

Feel free to check out our tutorials at https://www.youtube.com/watch?v=26Gt_9kFVok for more guidance.

Hope this helps! You can share your progress and automation or screenshots here so the community can advise further.