Scraping Tips


Interesting tips and articles about Web Scraping. How to successfully use automation to gather data from websites. Data extraction techniques and code are available in our tutorials

An API for every site using web scraping

An API for every site using web scraping

There is a lot of content available on the millions of websites on the Internet, and all of them involve some amount of programming to get them there, however, to get to all this content using a programmatic API isn’t really possible. If you need data scraped from a website in a specific format in […]

XPaths and their relevance in Web Scraping

XPaths and their relevance in Web Scraping

XPath (XML Path Language) is a syntax for defining parts of an XML document. We will explain the relevance of Xpath in web scraping. XPath is a query language for identifying and selecting nodes or elements in an XML document using a tree like representation of the document. XPath was defined by the World Wide […]

Why *not* scrape yourself

Why *not* scrape yourself

Before you get all kinds of ideas about what the topic of this article means – please look at the context – We are talking about Web Scraping here ! This post will talk about reason why not to do this yourself and why to call in a professional (wink wink – use ScrapeHero) You […]

Webscraping using Python without using large frameworks like Scrapy

Webscraping using Python without using large frameworks like Scrapy

Scrapy is a well-established framework for scraping, but it is also a very heavy framework. For smaller jobs, it may be overkill and for extremely large jobs it is very slow. If you would like to roll up your sleeves and perform web scraping in Python.¬†continue reading. If you need publicly available data from scraping […]

5 tips for scraping big websites

5 tips for scraping big websites

Scraping bigger websites can be a challenge if done the wrong way. Bigger websites would have more data, more security and more pages. We’ve learned a lot from our years of crawling such large complex websites, and these web scraping tips could help solve some of your challenges Web Scraping Tips Here are 5 web […]

Turn the Internet into meaningful, structured and usable data