Free has a price We have all heard of the saying "There is no free lunch" and no matter how many times businesses come around giving us something for free, there is always a price.…
Before you get all kinds of ideas about what the topic of this article means – please look at the context – We are talking about Web Scraping here !
This post will talk about reason why not to do this yourself and why to call in a professional (wink wink – use ScrapeHero)
You see some data that is publicly available and you start thinking about the possibilities of what you could do with it.
A brand new use, maybe the idea for a new startup, maybe a way to improve your existing business, reduce costs, get a leg up in your competition – the ideas just keep pouring in.
The possibilities are really endless.
Now you are excited and start to wonder how you can get this data in a structured format and actually use it.
Google to the rescue – you type in a few keywords around website data and start to see something about scraping – yes that is what I want.
You start to see a whole new world open up, tools, programmers, software, programming languages, do it yourself websites.
Now your head is spinning. You take a deep breath and revisit this whole thing the next day.
You wake up clear-headed the next day and feel like you are up for the challenge – you read up some more.
Start to spend money on getting cloud based servers, calculate bandwidth costs, figure out that you need a lot of servers to get any decent scraping done in a reasonable amount of time.
You add up the costs and they do add up – add up to a lot.
Then you wonder if you will be able to wake up every morning raring to go and scrape away – is that what you really want?
Is that what you really want to do day in and day out every day?
The simple answer is “scraping it yourself” is more like a DIY home project for people who have the skills AND the time and consider it as a relaxing hobby (excludes the majority of this blog’s readers).