How to scrape a text that doesn't have a selector (Scraping location from LinkedIn jobs)

[daniel]

Hi,
I’ve been using Bardeen for a while now and have come across a little issue I hope you can help with. I am trying to scrape a specific element on a page, but it doesn’t seem to have a selector, so won’t allow me to scrape that one element, I have to scrape the whole block. I’ve tried to use xpath, but that doesn’t seem to work either. The video below will hopefully better explain what I’m trying to convey.

[ivan]

Hi Daniel

Sorry about the delay, the team was offsite and some conversations slipped thought the cracks!

Seems like you’re trying to scrape the location from Jobs.

Tried to get the selector myself and prooved to be tricky!

This selector gets you to the section:
“[class*=‘jobs-unified-top-card__primary-description’] div”

But I’m blocked on how to capture the text

Involving our engineers @dan @manvel so they can teach us how we can approach this case.

Example:

Sign Up | LinkedIn

[daniel]

Hi @ivan

Thanks for coming back to me. I have tried a number of times, in a number of different ways, but it seems like it doesn’t have a selector if you look at my video.

I am able to copy the XPath and full XPath which provides the below, however when you paste that in the advanced options section it doesn’t recognise it. If you could help at all @manvel or @dan it would be very much appreciated. Hope to hear from you soon.

//*[@id=“main”]/div/div[2]/div/div[2]/div[1]/div/div[1]/div/div[1]/div[1]/div[2]/div/span[2]

/html/body/div[7]/div[3]/div[4]/div/div/main/div/div[2]/div/div[2]/div[1]/div/div[1]/div/div[1]/div[1]/div[2]/div/text()

[ivan]

Hey @daniel

This is a very custom case, where the scraper is working fine, but you want a specific field.

So unfortnately, I’m unsure how long it can take them to respond back

Consider these alternatives:

1. Getting the location from the original text.

We have regex commands “find values using regular expresion”.

I’ve tried prompting ChatGPT to give you the regex you need and has worked quite well!

There might also be ways to extract the location with Google Sheets formulas, might be worth a try!

2. Refining the scraper yourself

We recently did 2 workshops teaching about these topics if you want to learn:

Sept 14 - Bardeen Workout! Mastering Bardeen’s Scraping - Class 1: The Overview

Sept 14 - Bardeen Workout! Mastering Bardeen's Scraping - Class 1: The Overview

Unlock the Power of Web Scraping: Bardeen Workshop Date: Thursday, September 14th Time: 11:00 AM (EST Time) | 8M PST | 5PM CET Ready to take your automation skills to the next level? Join our Bardeen Masterclass and dive deep into the world of web scraping alongside our experts, Ivan Escobar, @manvel and @danmelk, Bardeen’s scraper leading engineers. What you’ll learn: Inside Bardeen Scrapers: Explore the inner workings of Bardeen’s scraper. Learn how to s…

Oct 6 - Bardeen Workout! Mastering Bardeen’s Scraping - Class 2: The Overview

Unlock the Power of Web Scraping: Masterclass 2 - Deep Diving into Scraping! Date: Thursday, October 5th Time: 11:00 AM (EST Time) | 8 AM PST | 5 PM CET Ready to take your automation skills to the next level? Join our Bardeen Masterclass and dive deep into the world of web scraping alongside our experts, Manvel Saroyan (@manvel) and Danil Melkozerov (@danmelk), Bardeen’s scraper experts. This is not a begginers class, we WILL get into the scraper’s tech…

3. Finding a user who can give custom advise at Jobs and Gigs

[manvel]

Hey @daniel this is an amazing use-case and we added support for it, lately. Now you can select text Nodes using XPath, should be available in the upcoming release.

[daniel]

Hey @manvel That’s awesome news! When is the next release? This is something I’ll definitely be implementing. Would you be doing a webinar or some sort of video explaining how to implement it once it’s ready to be used?