Product Monitoring – Essentials

Most requests for “Product Monitoring” or “Price Monitoring” that we receive do not have crucial information that is needed for us to provide any meaningful pricing back.

This page should help anyone requesting such monitoring to refine their own ideas, identify gaps and understand the nuances for a seemingly “simple” request.

This will also help the you to respond back to us clearly to expedite the process.

Common Product Monitoring Scenarios

Price Monitoring

Monitor product price or third party reseller prices

Review Monitoring

Monitor Product Reviews, all reviews or the latest daily reviews

Monitor other Data

Accuracy, Inventory, Search Prominence, Native Advertising etc

The Essential Questions

The essential questions you need to think about that are also essential for us to provide you a quote are:

Source Identification

Where is the data that you need?
Do you know which websites you want to monitor?
OR
Do you just want to get a general idea of price monitoring on the "Internet" without specifically scraping all known and unknown competitor websites?

Product Identification

How do you identify a product?
We have to know how to find the products that you want to monitor (sounds logical?).

Here are some common inputs that our customers provide:

Potential inputs that you could provide usExample of the inputAccess Method *Page count for each product*
Product URLhttps://www.amazon.com/dp/B07392JXFRDirect Access1x the number of product URLs
ASIN
(specific to Amazon)
B07392JXFRDirect Access1x the number of ASINs
Product CategoryAll products in “Toys & Games : Baby & Toddler Toys : Bath Toys” category on Amazon.comBrowse based1.2x the number of products in the category (on average)
Seller StoreAll products sold by a vendor or third party seller on Amazon or eBay virtual storeBrowse based1.2x the number of products in the category (on average)
Product Name“Amazon Echo”, “Crayola 12 ct Markers”Search based2x
Model Number or Manufacturer part number – MPN“Crayola 52-3024”Search based2x
SKU843278Search based2x
UPC or GTIN071662000240Search based2x
Product Keywords“iPhoneX cover” on ebay.comSearch based2x

* has a direct impact on the cost

The access method and the page count have a direct impact on the pricing. 

To understand the impact of these choices, please click on each of the links below to read more about the impact.

Frequency

How often does the data need to be gathered?
The simple rule for pricing is "more costs more".

Is the data needed weekly, monthly, daily etc?

While most people would like to get "real time" changes to data, that process become very expensive and unsustainable both from a processing perspective for us and consumption perspective for you.
The data we gather eventually has to be actionable for you.

e.g if you ask for price updates every minute, what can you do with this data once you have it every minute? Will you be able to adjust your prices globally every minute?

If you cannot do anything actionable, it does not make any sense to increase your spend for such non-actionable update of data.

Important Limitations

Number of Products

A lot of websites restrict the number of products that they show.

e.g. If you wanted the pricing for ALL products in Toys (or any category) on Amazon.com, it is not possible to get that data realistically because Amazon restricts the number of products it shows for any category, sub-category or filter or even a search.
The last time we checked the limit was less than 10,000 products.

Searching for Products

The ability to find products using the “search terms” such as SKUs or MPNs, UPCs, GTINs at each website depends on whether the website

a) stores the SKU, MPN, UPC, GTIN etc 

AND 

b) allows searching for the SKU, MPN, UPC, GTIN using the site's search function.

Contrary to popular belief, not every website stores or allows for searching of these numbers

Please factor this limitation into your product monitoring strategy and requests to us.

Product Matching

“Apples to Apples” comparison of products across various websites is not as easy as it sounds. 

A seemingly simple process of comparing products one website with another website for various business needs such as “Price Monitoring” or MAP violations is not so simple.

This related article highlights some of the basic challenges associated with such projects. It is best to be aware of the complexities as you embark on these business critical projects. Read more about the challenges associated with Product Matching 

Turn the Internet into meaningful, structured and usable data   

ScrapeHero Logo

Can we help you get some data?