As location specific data gets more prevalent, the impact on using web scraping for data gathering is significant.
On the positive front, the cost savings are significant because brands do not have to send “mystery shoppers” to physical stores to check product availability or pricing. This data can now be gathered with increasing accuracy online.
The data can now cover the whole country or all stores of a brand and all brands. The savings versus the traditional approaches are huge and the data coverage is much wider and frequent.
On the negative front, the cost of gathering this data increases because of the need to get the data from multiple locations as opposed to a single location.
The cost of getting IPs that are specific to a city or location are also higher due to the availability of local IPs in every location.
The number of pages that need to be scraped also increases because the location has to be set for each product and that sometimes takes 2 to 3 times the number of pages compared to a location agnostic data gathering project.