I have been struggling with deep scraper for sometime now. I am trying to scrape news articles from an Arabic website for a research. I followed one of the videos instructions and created an automation for the results lists then another one for the news page. I run the automation several times and each time I get no results at all or one or two results. I need to scrape about 660 news articles from this website for my research and it is now time consuming. I tried to increase the time before scraping each page but with no luck. The results page link: https://www.youm7.com/home/Search?Drpcallist=1&Drpseclist=&allwords=حرب%20غزة%20&page=5
the news articles page: https://www.youm7.com/6593247
Hi @nancy.adel.nh,
Could you please share the automation you’ve made to allow us to assist further in troubleshooting?
Thank you,
Jess
Here is my automation: Shared Playbook Template
I run it several times and sometimes it only got me the links of the list without any page content. In some other trials is got me only 15 pages out of 663. I tried to lower the number of pages I scrape to 5 each time and it just gave me one result or nothing at all. I can’t complete my project by scraping 5 pages each time as this is time consuming. Is here anything I can do to scrape all the data at once?