Issues with scraper being consistent

❓Help and questions
1

I have been trying to scrape this site in the background. I can get some of the items to scrape properly, but there is still quite a few that are not.

This is an example of one of the pages:
https://www.courts.mo.gov/cnet/cases/newHeader.do?inputVO.caseNumber=2316-FC05450&inputVO.courtId=CT16#party

I’m trying to Scrape the Case# and Parties at the top, Petitioner Address and the Respondent Address.

Because some of the pages are a little bit different it’s not scraping them properly. Sometimes one party or the other will have an address, sometimes one won’t. Sometimes there is an attorney on the right side, sometimes, there is two and sometimes there are none.

I’m not sure if the scraper is designed to find items in the same exact spot on the page each time or if it’s finding the information based on the code of the website and trying to find it based on the label for where that information is placed.

Any help is greatly appreciated as I have made a handful of different scrapers to scrape this site and I can not get any of them to get through my list of URLs to scrape and not have about half of them wrong.

1 Like
2

Hi @thisshirtoffmyback (lol) - could you please provide a few links you are trying to scrape from? I’ll see if we can get you the correct selectors to use for this use case and that should solve the issues you’re running into. :slight_smile:

1 Like
3

https://www.courts.mo.gov/cnet/cases/newHeader.do?inputVO.caseNumber=2316-FC05599&inputVO.courtId=CT16#party

https://www.courts.mo.gov/cnet/cases/newHeader.do?inputVO.caseNumber=2316-FC05371&inputVO.courtId=CT16#party

4

What I have figured out is, for any record that looks like the ones with parties on the left and no attorneys on the right. I have a scraper that will pull them perfectly. However, when I have a list of links that has both types of pages, I don’t know which one is pulling in the info correctly unless I manually look at everything. I’m trying to figure out a way that I can run the scrapper for all links I have and have it pull all the correct info on every one or close to every one.
Hope that makes sense.

5

Okay, I’ll trial/error on my end aiming to get the following data in a GSheet:

image
image1071×252 19.3 KB

1 Like
6

I’m not having any luck on my end, could just be user error. I’m not exactly sure, but it looks like these pages are built differently so I’m not certain it’s possible?

Tagging @Deyan_Petrov (Advanced Scraping GURU) for further assistance. Deyan, are you able to find the correct selectors for these elements that @thisshirtoffmyback is looking to scrape?

Thank you for your assistance!

2 Likes
8

Yes, if the page markup is different you might need to create different scraper models to get data from the page with different Markup, but you can also try building model with custom selectors.

I will recommend checking current video: https://www.youtube.com/watch?v=26Gt_9kFVok
Also please check advanced section in here: Tutorial: How to use Bardeen scraper | Bardeen.ai

But in general yes, if the pages are different then different model is created, unless it’s similar/same page layout-wise.

In this specific case the pages below does use different layout:

10

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.