In writing my keynote speech for the Data Summit SF, I did a lot of thinking about how to best explain my beliefs about data, and how it will change the world. As always, it’s handy to have a nice metaphor to fall back on. And there are a lot to chose from in the data space: Data is the new oil, soil, currency, economy, bacon – you can fill in practically anything here. For my talk I chose to equate data with gold, partly because it’s a good metaphor, but mostly because I was speaking in San Francisco which owes a large part of it’s success to the Gold Rush of the 1850s.
In speaking to people about it afterwards though, I don’t think the gold metaphor quite did the idea justice…
I’m not going to waste your time telling you how much data is out there and how much more of it there will be soon. Suffice it to say, it’s a lot. The simple existence of data isn’t really that relevant. If you can’t access it, don’t what to do with it, don’t know how to use it – or probably a combination of all of them – then it’s not really doing you much good.
When I give speeches or write about data I talk a lot about founding import to democratize data, but that’s not really accurate. I can no more make data publicly available than you can. What we’re really trying to do is democratize the access to that data.
That’s a bit of a confusing concept. How can you democratize the access to something? Which is why I like to use a metaphor to help explain it:
When the gold rush started in 1848, these were the tools you had to use to get it.
As you can imagine this process was long and arduous; and not all that many people were able to do it effectively or at a large enough scale that it could make real profit from it. This is similar to manually accessing information from the web; anyone with internet access can do it (usually with copy and paste) but it takes a very long time and is pretty impossible to scale.
Fast forward a century and a half, and this is how we mine for gold today…
These machines have automated the process of mining for gold, making it far more cost efficient and scalable. These machines are also very expensive, meaning that there are only a handful of companies in the world that can afford to own an operate one. This is a lot like the advent of web scraping technology; it allowed those that were highly skilled and could afford the infrastructure to automate the process of accessing data from the web, making it far more efficient to use.
Now imagine if I could simply give you one of those machines for free (and imagine you could operate it by youself). That’s what we are trying to do with import.io. We want to give everyone access to the tools and infrastructure they need to automate the process of extracting data from the web. Democratize the access to data through better tooling.
Of course there are many steps between mining gold and turning it into a fancy necklace. The same is true of your data. Once you’ve got it, you need to refine, visualize and analyze it to get the real value from it. Again, there are many companies in this space that are endeavoring to make this process simpler and more affordable for everyone – many of whom we partnered with at the Data Summit.
As you can see, my gold analogy is quite a good one – though you could use practically any commodity. The old process was hard and time consuming, the new process is faster but expensive, we want to make the new process free. Job done.
The problem, is that these metaphors lack magnitude. Making gold more accessible is only good if you’re interested in gold. It doesn’t accurately represent how all encompassing the data revolution is going to be.
To really understand the impact that data can have we need to relate it to something like the industrial revolution. The industrial revolution changed the way people did things, it made the mass production of products possible, created 1000s of jobs, boosted the economy, lowered prices and lead to staggering amount of innovation. It didn’t matter whether you worked in an office or a factory, the industrial revolution changed the way everyone interacted with the world.
That is the effect data is going to have. Very soon, every single one of us will have our lives irreversibly altered by data. It’s why I believe that access to this new, powerful resource should be available to everyone – not just those that can afford it. At import.io we’re working to create a suite of tools that will allow anyone to access data from the web. And it’s working. Using our newest tool, Magic, users created more APIs in its first week than exist on the whole of programmableweb, long thought to be the definitive source of APIs.
This kind of progress, and seeing all the amazing uses our users find for web data is what inspires us to continue evolving and innovating as a company. Events like the Data Summit are our chance to showcase this for you guys and to inspire you with the latest advances in data. We’ve had so much fun doing them, that we’ve decided to host two a year, one in London and one in San Francisco!