Duplicates on a recurring automatic scrapping

I set up a automatic flow where functions as follows:
1- Trigger every week run 10 times.
2.- Action Get some urls to scrappe from GoogleSheets
3.- Premium Scrapper gets Urls from GoogleSheets and return back 10 items per run per url.
4. Post results in GoogleSheets.

I was expecting to get new items every time it runs, but on the contrary Im getting the same 10 items for each URL every time that it runs.

What Am I doing wrong?

Hi ,

I’d like to clarify your statement ‘return back 10 items per run per url’. Everytime you run the playbook, it will get the first 10 items on the list you are trying to scrape. So unless that list has a new list of 10 items, it will always return the same 10 items.

Hope that answers your question. If it doesn’t, it would be good to have more context on why you are only scraping 10 items each time.

Regards,
Vin

Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community

I don’t want to overload the sales team withthousand contacts at the same tine, I want the to receive a certain amount per week

Can I achieve this behavior some how?kind of what Phanyom Bustet is doing

On 6 Dec 2023, at 23:25, Vinohar Kumar (Support) support@bardeen.ai wrote:

Hi,

Unfortunately, this isn’t possible right now as we can only start scraping from the top of the list each time, which will likely mean the same 10 results will show each time

Could I recommend that you scrape the full list once, but then use some Google Sheets formulas to only show 10 at a time to the team ? The QUERY formula is probably a good one where you can limit the number of results.

Cheers,
Vin

Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community

Could you elaborate a little bit more on the QUERY Google sheet part? Or point me to a video to understand what you are describing?

On 8 Dec 2023, at 11:12, Vinohar Kumar (Support) support@bardeen.ai wrote:

Hi,

Here’s a guide I found on YouTube describing how to use the Query formula: https://youtu.be/oCKvAcXTLZo?si=3j0nEVKb5MzH837L&t=1

I think I have a better solution for you though:

  1. Using your current playbook to scrape the full list of leads into a Google Sheet and add a column at the end with a checkbox so you can tick it whenever the lead is sent to the Sales team

  2. Schedule a weekly Bardeen playbook that contains the following actions:
    a. Get table from Google Sheet - to get the list of leads from the sheet with the results from Step 1
    b. Add a new condition to search if the last column of the table is ticked (so you only pick out new leads that haven’t been sent to the sales team
    c. After the condition, use the get slice of array to get 10 rows from the table
    d. You can update another tab or slack that the Sales team has access to with the 10 leads
    e. Tick the last column of the original sheet with the lead list so that the next time this playbook runs, it’ll take a new set of 10 leads

I can appreciate that the full solution might not be clear over email but if you have a list of all the leads, I could look into guiding you further on the 2nd playbook that pulls 10 leads each week.

Cheers,
Vin

Customer Support - bardeen.ai
Explore | @bardeenai | Bardeen Community

Vin actually is quite clear to me let me try this out and I will come back to you to tell you the outcome so far I have been busy but I will try it out

Thanks :pray:

On 10 Dec 2023, at 20:16, Vinohar Kumar (Support) support@bardeen.ai wrote:

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.