How to scrape only new listings that weren't there the last scrape

šŸ Getting started.
,
1

Hi,
I am scraping Zillow listings to google sheets, and every day there is new listings that came up for my search.
How (and what is the best method) to scrape only listings that I didnā€™t scrape yet?

2

I have a similar question: Questions about adding/updating pages on Notion

I canā€™t seem to figure out a way to check if something is scraped or notā€¦

3

Hi Chaim,

If you would like to only scrape new listings, I would suggest you consider configuring an autobook: https://support.bardeen.ai/hc/en-us/articles/23925118331277-Creating-Autobooks

Basically, you can configure it for watching a search results page for updates. Please note that such a setup will charge you 1 credit per hour of watching.

Alternatively you can scrape a first page of listings and configure a conditional logic for adding only new records to a sheet. Please note that such a setup may be not cons effective because you will need to pull the entire google sheet when adding a record (get table from google sheet is charging 1 credit per row).

Let me know if you have any other questions!

Victoria

Customer Support - bardeen.ai
Knowledge Base https://support.bardeen.ai/hc/en-us
Explore | @bardeenai | Bardeen Community

4

Thanks for the reply!
Can I configure it to check for changes once a day and then it will be only 1 credit?

5

Follow up Question.
My current plan is 500 credits for 15$ a month, and when I look at the pricing now for another account the cheapest option I see is 99$ a month for 15k credits!
My issue is, one, I donā€™t think its reasonable to pay so much money, two, I dont need more then 5k -10k credits a month for the autobook iā€™m createing.
All I need is to check once a day for new listings on Zillow and add them to google sheets, why should it be so complicated and cost so much???

1 Like
6

Can I configure it to check for changes once a day and then it will be only 1 credit?

Iā€™m afraid that thereā€™s no way to change the intervals for scans.

Follow up Question.
My current plan is 500 credits for 15$ a month, and when I look at the pricing now for another account the cheapest option I see is 99$ a month for 15k credits!
My issue is, one, I donā€™t think its reasonable to pay so much money, two, I dont need more then 5k -10k credits a month for the autobook iā€™m createing.
All I need is to check once a day for new listings on Zillow and add them to google sheets, why should it be so complicated and cost so much???

You are right, recently there was a change in pricing. Just to clarify, the $99/month plan youā€™re seeing is actually part of our annual billing option, which provides 15,000 credits upfront for the entire year. While itā€™s displayed as $99/month for reference, the full payment is collected at the start of the subscription. This setup offers flexibilityā€”you can use more credits in busy months and fewer in others without losing any.

Unfortunately, Iā€™m afraid we donā€™t currently offer a smaller or custom plan in between the tiers. I truly wish I could offer more flexibility here, but this is the structure we have at the moment.

Let me know if youā€™d like help optimizing your automation to use fewer credits, or if thereā€™s anything else I can support you with!

Best,
Victoria

Customer Support - bardeen.ai
Knowledge Base https://support.bardeen.ai/hc/en-us
Explore | @bardeenai | Bardeen Community