First of all, I would like to thank you for providing your tool.
However, I am currently facing a problem and hope for your support. I am encountering difficulties while performing deep scraping on the following page (ADAC Fahrzeugwelt Auto Abo Angebote).
I do scraping in the background and get the prices without any problems. However, the rates in Google Sheets are not reflected when I perform the “Click” actions to include the changed price with the selected fields. Link: Abfrage ADAC - Google Sheets
Could you please help me by letting me know if I am making a mistake in my approach or if there may be any limitations on the website that are hindering scraping? Your help in resolving this issue would be extremely valuable to me.
Indeed this particular website is not willing to cooperate with a scrapper. But I think there may be still a solution for this. Scraping highly relies on fetching links, so I suggest you to perform a deeper scraping:
First step: scraping the links on the list of cars
Second step: doing a series of clicks and getting a link of that page instead of the calculated price:
Step 3: Scraping prices by loading the links from step 2 on background
Please let me know how it goes for you. So far my tests of this scraper return an updated price while the price is not being updated when configuring the scraper due to site limitations.
Thank you very much for your solution approach. I didn’t quite get it right with your solution, but I was able to find another solution. I used the “delay” button and gave the page a few seconds (4 sec) to load and then scraped the price. That worked.
