Lessons from numbers guru Nate Silver about working with big data

Says it's not about having the data – it's about knowing what to do with it

There's a big misconception about big data, renowned statistician and blogger Nate Silver says. It's not enough to just have access to massive amounts of data: You need to know how to use it and what to do with it.

It may seem obvious, but it’s a problem many organizations struggle with, Silver said during an interview with Silicon Angle’s “The Cube” after his keynote speech at data management company Tableau’s user conference.

[FIRST LOOK: iPhone 5c and 5s debut]

“None of this big data stuff is going to be a solution for companies who have poor corporate cultures where you have trouble communicating ideas, where you don’t have everyone on the same page,” he says. “You need buy-in from all levels of the organization, you need C-lever people who understand the value of analytics, you need analysts who understand what business problems are trying to be solved and what the organizational goals are.”

For the last four years Silver has worked as a blogger for the New York Times in which he focused primarily on political statistics, including election predictions and the 2012 presidential race, which he predicted with remarkable accuracy. Recently, he signed on to work for ESPN and take his popular FiveThirtyEight blog there.

For businesses to truly extract value from data, you need a high volume of data, but you also need high quality data, he says. “A lot of success in business and certainly when it comes to the use in analytics is stripping away the things that get in the way from understanding and distracting. It’s not like you can wave a magic wand and have some formula that uncovers all the secrets in the world, it’s more like if u can strip away the noise you’ll have a much clear understanding of what’s really there.”

There are no shortcuts to working with data, you have to put in the time and resources to extract value from the data. There are very few shortcuts in life, he says. Even in sports when professional athletes use performance-enhancing drugs, it does not automatically make them elite athletes. “The guys who really get the benefits from the steroids, they have to work their butts off,” he says, creating a synergy of hard work plus performance enhancing drugs.

People in the political world, which he has frequented for the past few years, he says are “operating to manipulate public opinion and they don’t place a lot of value on the truth,” which he considers immoral. Using data and science to paint an objective view of the world is a moral obligation given all of the spin, he says.

Because he’s focused on politics primarily for the last few years, Silver says he has a lot of “pent up ideas” about how to look at sports through a statistical angle. For example, evaluating which of the five players on a basketball team are the best to take a certain shot, handicapping international soccer teams, and using the plethora of statistics from baseball to tell interesting stories.

Senior Writer Brandon Butler covers cloud computing for Network World and NetworkWorld.com. He can be reached at BButler@nww.com and found on Twitter at @BButlerNWW. Read his Cloud Chronicles here.

Join the discussion
Be the first to comment on this article. Our Commenting Policies