Scraping wellfound.com

❓Help and questions
1

I am trying to scrape www.wellfound.com/jobs for sales executives and account managers. It looks like it is scraping the information but not adding it to Google Sheets. I have Sheets set up correctly.

I have tried several times with different setups, and it is not working. I am unsure if I got my IP blocked or if they make scraping difficult.

2

Hi @chris1, could you please share your automation?

If you show results on your scrape action, you’ll be able to determine if the scrape is pulling data or not.

I hope this helps!

3

Screenshot 2023-09-18 at 12.31.58 PM
Screenshot 2023-09-18 at 12.31.58 PM2148×382 26.3 KB

I have it set up to ask me every time how many job posts I want to scrape. 100 is the number I gave it on the last scrape attempt.

4

Thank you for that extra information. Are you able to share the automation?

5

Thank you for the help.

6

Thanks @chris1 - your automation is not setup correctly for the following:

  • 100 is being input into cell A:1 because the “ask for data” is input on the “Add text to Google Sheet” action:
    image
    image634×780 34.7 KB
  • the “ask for data” should be inside of the “Scrape data on active tab” on one of the two fields here:
    image
    image649×781 42.6 KB

    – these tell you the number of items to scrape OR the number of pages to scrape, only choose one. It sounds like in your case you would choose the first one - items.

You also then would need to update the “Add text to Google Sheet” action to pull in the scraped data from the previous action like so:

image
image648×763 34.8 KB

Here’s your updated automation:

I hope this helps!

7

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.