All Articles

Did you read this interview text?

Did you read this interview text?

Recently scraping.pro (one of our favorite blogs) interviewed our very own Tony Paul and asked him about many current and future trends on Web Scraping. Our favorite quote from this interview about Tesla and open source You can’t steal what’s free and freely available – pure genius in our view You can read the whole […]

Chart: Count of products in Amazon US for major categories

Chart: Count of products in Amazon US for major categories

EDIT: Amazon sells about 368,876,590 products as of Dec 1, 2016. Updated post available here – How many products does Amazon.com sell – December 2016 Total Products 323,187,951 Books 52,039,819 Electronics 42,426,029 Home and Kitchen 39,708,640 Digital Music Songs 32,977,000 Cell Phones and Accessories 24,039,269 Sports and Outdoors 19,665,787 Clothing Shoes and Jewelry 17,599,928 Collectibles and Fine Art […]

How many books does Amazon sell?

How many books does Amazon sell?

We all know Amazon.com started as an online seller of books and now they sell everything. This post tells you how many total products Amazon has and out of those 52 million are books ! So they still sell a good percentage of their total products – nearly 16% as books.

How many products does Amazon Store sell?

How many products does Amazon Store sell?

A question that comes up often and it is a nice data point. As of now Amazon US store has 323,187,951 products for sale. That is a staggering number – 323 million products !

Connected Cars – an infographic

Connected Cars – an infographic

Why *not* scrape yourself

Why *not* scrape yourself

Before you get all kinds of ideas about what the topic of this article means – please look at the context – We are talking about Web Scraping here ! This post will talk about reason why not to do this yourself and why to call in a professional (wink wink – use ScrapeHero) You […]

Surprise – Games are big on Apple iOS Store !

Surprise – Games are big on Apple iOS Store !

Not a real surprise – but Games are big (so much for productivity though – coming it at less than 3%!)   Find more statistics at Statista

Google Play Store Application growth

Google Play Store Application growth

Find more statistics at Statista

Webscraping using Python without using large frameworks like Scrapy

Webscraping using Python without using large frameworks like Scrapy

If you need publicly available data from scraping the Internet, before creating a web scraper, it is best to check if this data is already available from public data sources or APIs. Check the site’s FAQ section or Google for their API endpoints and public data. Even if their API endpoints are available you have […]

Why pay for a web scraping service rather than get it for free

Why pay for a web scraping service rather than get it for free

Free has a price We have all heard of the saying “There is no free lunch” and no matter how many times businesses come around giving us something for free, there is always a price. So you use Google for free or Gmail for free or Facebook or Twitter or any of the companies that […]

How to prevent getting blacklisted while scraping

How to prevent getting blacklisted while scraping

Web scraping is a task that has to be performed responsibly so that it is does not have a detrimental effect on the sites being scraped. Web Crawlers can retrieve data much quicker, in greater depth than humans, so bad scraping practices can have some impact on the performance of the site. If a crawler […]

5 tips for scraping big websites

5 tips for scraping big websites

Scraping bigger websites can be a challenge if done the wrong way. Bigger websites would have more data, more security and more pages. We’ve learned a lot from our years of crawling such large complex websites, and these tips could help solve some of your challenges 1. Cache pages visited for scraping When scraping big […]

Turn the Internet into meaningful, structured and usable data