L-R: Mark Buchanan, Nate Silver, Jessica Bland and Greg Mead
In a big data world we have been promised that it will be possible to predict certain future outcomes and solve the world’s problems. Essentially, More data = more power. But according to a panel of statistical experts at last Friday’s “How to Think About the Future” panel discussion, having the data is only the first step in understanding current phenomenon and predicting future outcomes.
According to Nate Silver, author of The Signal and the Noise, we are currently quite bad at predicting the future and, to date, we are not getting any better. Why? Because our subjective point of view often clashes with the massive volumes of data we are seeing. With so much data it is possible to make the numbers say almost anything, and often as humans, we are looking for what we want or expect to find.
Fellow panelist Mark Buchanan, physicist and author of What Physics, Meteorology, and the Natural Sciences Can Teach Us About Economics, added that models will never be able to predict everything no matter how good we get at interpreting data. Nothing can ever be perfectly efficient, and even if it could, it would stagnate invention because there would be no incentive for innovation. He also noted that the speed of decision making has made predictive modeling quite risky. Often it is machines who are trading data with each other and making decisions based on predetermined algorithms. By taking all of the human logic out of decision making we put ourselves in the same type of risky position which led to the financial crisis a few years ago.
It’s not all doom and gloom however, big data is already being used in unusual ways to help industries to predict where risks should and should not be taken. Greg Mead, CEO of MusicMetric, spoke about how his company uses the massive amounts of music data now available to build predictive models which can tell them how popular a new artist is likely to be or how well a tour will do. These models allow record labels to minimise the risk of signing a new artist by predicting the demand for that artist ahead of time.
He cautioned, however, that there is a danger behind only listening to the algorithm. There needs to be human input as well, otherwise we run the risk of becoming too averaged. If that happened, we could end up with a million Justin Biebers – and nobody wants that!
The overall takeaway from the night is that models can be used to do amazing things, but only in the right conditions. There are tradeoffs to the simplification needed to build a model and sometimes these outweigh the benefits. It is important for people to realise that more data doesn’t necessarily mean better analysis. More data might equal more power, but not until we learn how to accurately harness that power.
by Jen Methvin, Maketing Associate