Startups and Data

Everybody’s gotta start somewhere – so goes the saying.

Startups especially start from nothing.

In today’s world full of data, the question is where do startups get their data?

Let’s look at the sources of the data in the various stages of a startup.

Development Phase

At this stage a startup usually works with manually created data or in some cases, fake machine generated data. If the data needs of a startup relies on real world data, there are no real “data banks” that you can buy data from.

In this case you can “copy and paste” or use Web Scraping to get this data. The scraping can be done using homegrown scripts, commercial tools and in some case Macros in Microsoft Excel or even Google Sheets can be useful.

Testing Phase

At this stage, the startup really needs to test their application, platform or whatever they are building with the “typical usage scenario” level of data both from a quality and quantity perspective. This quantity of data cannot be manually generated – it has to be machine generated or procured from the real world.

Again, if you can use fake machine generated data and can also generate it effectively, this approach might work.

However, as before, if your startup is all about real world data, you will most likely resort to Web Scraping to get this data.

Beta Phase

The ever increasingly popular phase – the BETA phase. Here startups invite or open up the registration of their application or platform to users who either want to get things for free or just like the thrill of getting to be the first users of a system.

These users will enter a lot of user related data, but again, depending upon the type of application or platform, you will need to provide the “framework related data” – geographic, store locations, phone numbers, company names, business details etc to these Beta users so that they don’t feel like they are part of a ghost town (wink at Google+ – the social network most gmail users belonged but never really used – we are also guilty as charged)

Go live

In this stage, the startup will most likely use the Beta phase data, but to keep your application usable and in certain cases “fresh” you will need to augment your data from external sources – and you guessed it –  Web Scraping is one of the best approaches to do all that.

Buying data

In some instances a startup may be able to just buy data from a data vendor – email lists is one popular source. But all this “data for sale” is usually “data that is stale“.

This data was probably scraped at one point and now has turned stale, outdated and overused. The same old email leads have been used up and these leads have no desire to answer another unsolicited pitch from another startup.

Scraping the web for fresh leads is a great option to build something that is specific to your needs and fresh.

We are always ready to help your startup, just like we help so many others.

In case you find that gathering data through scraping isn’t your “core competency” and you would rather leave it to the pros, reach out to us.