Scraping gamejobs list, how to open the urls in the list and scrape them?

❓Help and questions
1

So! For example if I want to scrape this page - Game Developer Jobs - GameJobs.co - and create records for each, how do I also do this: open each item’s URL, extract another piece of data from each separate page, and then sync it back with the main list? For example:

" Narrative Director

Ubisoft · Montréal, Quebec · 6 days ago ·"

I’d want to get that info, but also open up the first link - Narrative Director at Ubisoft - and get the ‘Apply button’ url link on that page - Ubisoft Narrative Director | SmartRecruiters, and I’d want that URL to be added to my original scraped lis from the main jobs page.

I started creating this in the below but I was stuck on how to scrape the URLs pulled from the first list as part of my flow, and also am not sure how to connect it all up once the scraping is done.

(as a bonus question - is there a way to make it so our flows only add new/unique entries, and don’t add entries if they find duplicates?)

2

Hey @author

Welcome to Bardeen :jammies:

This use-case seems like the standard case for “deep scraping”.

  1. You get the links on a list scraper on the first page
  2. Then you scrape data from all the jobs on the background

Here’s the same use-case for scraping jobs out of a search on linkedin:

2023-09-06_13h01_00
2023-09-06_13h01_001215×547 70.1 KB

3

Yep, using the “Update commands”, like this one “Update google sheet rows” :smile:

4

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.