This tutorial is a follow-up to Tutorial: How To Scrape Amazon Product Details and Pricing using Python, by extending the Amazon price data to also cover product reviews. The scope of this tutorial is limited…
In this article, we will show you how to scrape all product reviews for a particular product or products, by using just a browser and a Chrome extension called Web Scraper. You will not need any special software, programming or other skills to monitor reviews about a handful of your own products on Amazon.
What data are we extracting from Amazon?
For this tutorial we will only extract the following fields from the products review page:
Below is the annotated screenshot of the fields of one review:
- Google Chrome Browser – You will need to download the Chrome browser. The extension requires Chrome 49+.
- Web Scraper Chrome Extension – The Web Scraper extension can be downloaded from the Chrome Web Store. After downloading the extension you will see a spider icon in your browser toolbar.
The Amazon Review Scraper
Using Web Scraper extension, you can create a sitemap that shows how the website should be traversed and what data should be extracted. The setup process is fairly simple, you can follow some of our other Web Scraper Extension tutorials or Documentation if you need to know more.
Here is the step by step video to extract reviews from Amazon:
After you have installed the extension right-click anywhere on a page, go to ‘Inspect’ and the Developer Tools console will pop up. Click on the tab ‘Web Scraper’ and go on to the ‘Create new sitemap’ button and click on the ‘Import sitemap’ option. Now paste the JSON below into the Sitemap JSON box.
Copy the JSON above and import into Web Scraper Extension, as shown below. If you have trouble copying, please copy from Github.
We will extract reviews from the product review page – https://www.amazon.com/Screen-Protector-SPARIN-Tempered-Glass/product-reviews/B013JZCAZK/ref=cm_cr_arp_d_viewopt_sr?ie=UTF8&reviewerType=all_reviews&pageNumber=1&filterByStar=one_star
You can scrape other URLs by editing the metadata. In the Web Scraper toolbar, click on the Sitemap button (which would have changed to sitemap ‘your sitemap name’ now) and select the “Edit metadata’ option and paste the new URL as the Start URL.
Here is how
Run the Scraper
To start scraping, go to the Sitemap and click ‘Scrape’ from the drop down. A new instance of Chrome will launch, enabling the extension to scroll and grab the data. Once the scrape is complete, the browser will close automatically and send a notification.
Download the Data as CSV
To download the scraped data as a CSV file that you can open in Microsoft Excel or Google Sheets, go to the Sitemap drop down > Export as CSV > Download Now
We can help with your data or automation needs
Turn the Internet into meaningful, structured and usable data
Disclaimer: Any code provided in our tutorials is for illustration and learning purposes only. We are not responsible for how it is used and assume no liability for any detrimental usage of the source code. The mere presence of this code on our site does not imply that we encourage scraping or scrape the websites referenced in the code and accompanying tutorial. The tutorials only help illustrate the technique of programming web scrapers for popular internet websites. We are not obligated to provide any support for the code, however, if you add your questions in the comments section, we may periodically address them.