The ‘click' and 'hover' problem

Hi there,
I want to scrape data from next page to gather more information about the personal information in the social platforms like twitter. However, it doesn’t work when I click ‘click’ or ‘hover’. Therefore, I wonder how could I fix the problem. Or, is there any ways for me to grasp the data in the next personal page? For example, the autobook way? Could you show me the tutorial? Thanks!

Hi there,

In order to scrape websites like you mentioned, you would need to refer to deep scraping. This means you will create 2 scrapers, one to scrape a list with search results and links to a details page of every item, the second scraper will scrape every item precisely.

Please review the next links to get an idea.

General guide on scraping:
Deep scraping tutorial with live examples: (long video)

Tutorial to deep scraping:

This way for twitter case you’ll need 2 actions:

  1. Scrape on active tab as list of profile links
  2. Scrape in the background using “Twitter profile by Bardeen” scraper connected to links of action 1.

Let us know how it goes or if you have any other questions.

Customer Support -
Knowledge Base
Explore | @bardeenai | Bardeen Community