Human Capital Briefs

Nate Silver on Finding a Mentor, Teaching Yourself Statistics, and Not Settling in Your Career

October 28, 2013

Perhaps no one has done more for the cause of data-driven decision-making in the minds of the public than Nate Silver. His book, The Signal and the Noise, explains the power of statistical modeling to improve our predictions about everything from the weather to sports to the stock market. Data science is the hottest field to be in right now, and Silver is its poster child.

But for most people, the gulf between recognizing the importance of data and actually beginning to analyze it is massive. How do those without extensive training in statistics equip themselves with the skills necessary to thrive (or even just survive) in our age of “big data”?

Last month I had the chance to put that question to Silver, and his answers may surprise you. Far from counseling that everyone must major in statistics, in the edited conversation below he advises students and executives alike to roll up their sleeves — no matter their statistical literacy — and get their hands dirty with data.

HBR: If I’m an average professional or an executive, I’ve read your book, I know this stuff matters and I also know it’s complicated and I can only expect so much. Is there such a thing as kind of a level of statistical literacy that I need to get to? What kind of education do I have to go back and make sure that I have?

Silver:  I think the best training is almost always going to be hands on training. In some ways the book is fairly abstract, partly because you’re trying to look at a lot of different fields. You’re trying not to make crazy generalizations across too many spheres.

But my experience is all working with baseball data, or learning game theory because you want to be better at poker, right? Or [you] want to build better election models because you’re curious and you think the current products out there aren’t as strong as they could be.  So, getting your hands dirty with the data set is, I think, far and away better than spending too much time doing reading and so forth.

HBR: What about if I’ve read your book and I’m just starting college or a little younger and I’m trying to think actually maybe this statistician/data scientist role is something that I’m interested in? What do I study? How much education do I need? What’s that base for plugging into some of these jobs?

Silver:  Again, I think the applied experience is a lot more important than the academic experience. It probably can’t hurt to take a stats class in college.

But it really is something that requires a lot of different parts of your brain. I mean the thing that’s toughest to teach is the intuition for what are big questions to ask. That intellectual curiosity. That bullshit detector for lack of a better term, where you see a data set and you have at least a first approach on how much signal there is there. That can help to make you a lot more efficient.

That stuff is kind of hard to teach through book learning. So it’s by experience. I would be an advocate if you’re going to have an education, then have it be a pretty diverse education so you’re flexing lots of different muscles.

You can learn the technical skills later on, and you’ll be more motivated to learn more of the technical skills when you have some problem you’re trying to solve or some financial incentive to do so. So, I think not specializing too early is important.

HBR:  Say you’re at the point where you started playing around with some data. You’re interested, you’re motivated, and now it’s time to actually learn some of those skills just like you talked about. Am I just going and picking up a textbook? Am I trying an online course?

Silver: I mean my path has been kind of sui generis in some ways, right? Probably an online course could work, but I think actually when people are self-taught with occasional guidance, with occasional pushes here and there, that could work well.

An ideal situation is when you’re studying on your own and maybe you have some type of mentor who you talk to now and then. You should be alert that you’re going to make some dumb mistakes at first. And some will take a one-time correction. Others will take a lifetime to learn. But yes, people who are motivated on their own, I think, are always going to do better than people who are fed a diet of things.

HBR: Say an organization brings in a bunch of ‘stat heads’ to use your terminology. Do you silo them in their own department that serves the rest of the company? Or is it important to make sure that every team has someone who has the analytic toolkit to pair with expertise?

Silver: I think you want to integrate it as much as possible. That means that they’re going to have some business skills, too, right? And learn that presenting their work is important. But you need it to be integrated into the fabric of the organization.

You’ve seen this shift in baseball teams, for example, where it used to be that you’d hire an analyst to check that box and have them compartmentalize. That doesn’t accomplish much at all.

HBR: You’ve had obviously some very public experience with the fact that even when the data is good and the model is good, people can push back a lot for various reasons, legitimate and otherwise. Any advice for once you’re in that position, you have a seat at the table, but the other people around the table are really just not buying what you’re selling?

Silver: If you can’t present your ideas to at least a modestly larger audience, then it’s not going to do you very much good. Einstein supposedly said that I don’t trust any physics theory that can’t be explained to a 10-year-old. A lot of times the intuitions behind things aren’t really all that complicated. In Moneyball that on-base percentage is better than batting average looks like ‘OK, well, the goal is to score runs. The first step in scoring runs is getting on base, so let’s have a statistic that measures getting on base instead of just one type of getting on base.’ Not that hard a battle to fight.

Now, if you feel like you’re expressing yourself and getting the gist of something and you’re still not being listened to, then maybe it’s time to change careers. It is the case [that] people who have analytic talent are very much in demand right now across a lot of fields so people can afford to be picky to an extent.

Don’t take a job where you feel bored. If it’s challenging, you feel like you’re growing, you have good internal debates, that’s fine. Some friction can be healthy. But if you feel like you’re not being listened to, then you’re going just want to slit your wrists after too much longer. It’s time to move on.

HBR: What about, from the perspective of an organization or a business, knowing those areas where data is really going to be the key to making good predictions and good decisions versus those areas where it isn’t? Speaking to a lot of start-ups and tech companies, you hear ‘Data can’t tell us anything. The future is so different than the past and we really can’t rely on it at all, so it’s really an intuition game.’

Silver: A lot of times when data isn’t very reliable, intuition isn’t very reliable either. The problem is people see it as an either/or, when it sometimes is both or neither, as well. The question should be how good is a model relative to our spitball, gut-feel approach. And also how much do we know about this problem. There are some issues where you just don’t have a good answer and you have to hedge your risks as a business and not pretend that you’re more certain than you really are.

A lot of private businesses are very reluctant to deal with uncertainty in their outlook. The manager doesn’t want to seem like he’s not sure what he’s doing. And the consultant or the analyst wants to provide information to make the manager feel more confident. That’s quite problematic because a lot of problems that are on the frontier of business, on the frontier of science, [are] by definition fairly challenging ones that no one else has solved.

That’s where having a more humble attitude about what you can accomplish and what you can’t is important. Just because a model is not going to be very precise or accurate doesn’t mean that therefore you should trust your gut instinct after a couple of whiskeys and assume it’s going to be very much better.

 

This blog first appeared on Harvard Business Review on 09/24/2013.

View our complete listing of Career Development blogs.