Nate Silver and the One-Eyed King in the Land of the Blind

Amidst all of the hubbub over Nate Silver’s departure from the NY Times, Kevin Drum notices something disturbing (boldface mine):

At the same time, this tidbit from Politico about Silver’s decisionmaking process while he was weighing competing offers from the Times and ESPN worries me just a little:

Silver had told The Times that he wanted to expand to weather, economics and anyplace else at The Times that had statistics and numbers he could bring to life….Nate will appear on the air on ESPN and ABC, and will get “verticals,” or web hubs, devoted to a variety of new topics. He’s very interested in education, so there’s been a lot of conversation about that.

Silver has done a great job with sports and campaign polling. But these are fairly unique areas. In the case of sports, Silver has a lot of subject matter expertise to go along with his number crunching. In the case of campaign polling, you don’t really need that much. You can get by with a pretty pure data-driven approach to the whole thing.

But weather, economics, and education? I’m skeptical that you can just parachute into those fields and add a lot of value. They’re far more complex, are already heavily populated with sophisticated statistical modeling, and generally require some serious subject matter expertise in addition to raw number-crunching skill.

When Silver entered the political prediction field, it was relatively wide open. Most popular prognosticators did no statistical analysis whatsoever–recognizing that polls have confidence intervals was a mark of sophistication (such as it was). But political polling is relatively straightforward compared to educational data. ‘Political’ data are relatively simple and uniform, and there are many opportunities for internal validation (e.g., daily tracking polls). But educational data are very different. States administer distinct tests and score them differently (cutoffs, raw scores, adjusted scores, value-added, MGP, etc.). On top of that, the stupid fucking natural history facts matter. Charter schools, for instance, do a couple of things to fudge the numbers, including transferring out students who perform poorly (and are then counted as non-charter students, even if they were in the charter for most of the year) and having a much less challenged student body (fewer English-as-a-second-language and special education students). It’s just not that simple; you have to understand the limitations of the data. Unfortunately, given Silver’s fame, he could potentially do a lot of damage here–and we’re not dealing with sports or political horse races, but childhood development. You have to get it right.

I’m also confused as to what Silver thinks he’ll offer in the weather department. It’s not like there aren’t entire departments of meterology, not to mention multiple government agencies and professional private forecasting firms trying to figure this stuff out. Maybe he’s got some kind of revolutionary special sauce that can be run on a laptop, but, as I understand it, he’s going to need a lot of computing power. Like in the many millions of dollars variety. As Drum puts it, the field is a little crowded.

Anyway, it will be interesting.

This entry was posted in Education, Experts, Polling, Statistics. Bookmark the permalink.

4 Responses to Nate Silver and the One-Eyed King in the Land of the Blind

  1. Bayesian Bouffant, FCD says:

    But these are fairly unique areas.

    Grrrr. Language pedantry needed.

  2. Bashir says:

    Interesting indeed. There’s certainly something to taking a tool (stats) and looking for areas with the biggest difference between potential use and actual use. Politics being the big one that he hit on. Weather seems like an odd choice to try to move into. Maybe he just wants to be as broad as possible.

  3. kaleberg says:

    I’m looking forward to some clear statistical thinking with regards to economics, weather and education.

    Typical reporting on economics is all horse race garbage where tiny variations are turned into the story. I’d rather see more focus on the general trends, the context, revisions, problems with measurements and relations among the various statistics. Different types of statistics are often contradictory, but this is rarely noted by the press. The frenzy of various announcements is ludicrous, and, as in politics, makes for bad business and economic analysis.

    Weather could also use some better thinking. There are often systematic biases in weather forecasts that make them much less useful than they should be. Mainstream weather reporting is also. You’ll often get blather about ongoing cooling trends in which the cooling miraculously is not associated with lower temperatures.

    You’ve often noted the problems with the usual interpretations of education statistics on this very blog. I’m sure Nate Silver and his team could do a much better job reporting educational statistics and what they tell us about our educational system than is usually done.

    It seems the only place anyone takes statistics seriously anymore is in sports. Everywhere else, they just try to impose their own narrative rather than letting the numbers speak. (Cecelia Holland used this contrast in her book set in Byzantium, Belt of Gold.)

  4. Pingback: Senseless Tragedy – Bridget Magnus and the World as Seen from 4'11"

Comments are closed.