Consider this part of our ongoing series in how not to use polling data. This head-explodey tweet came across the transom recently:
The tweet claims, “For the third week in a row, The Economist/YouGov polling shows that the share of Democratic primary voters who say they are considering voting for Elizabeth Warren is higher than the share saying they’re considering Bernie Sanders.” We’ll try to ignore that Morris is apparently a data journalist at the Economist*.
This statement is based on a sample of 537 Democratic voters (the poll samples 1,492 voters, 36% of whom said they were eligible to vote in the Democratic primary). According the poll (pdf), 41% said they would consider voting for Warren, while 40% percent said they would consider voting for Sanders. Here’s what it looks like in a table (I did some rounding; it doesn’t make a difference):
Remember I’m not arguing their ‘special sauce’–how they adjust the numbers to have a representative sample–is broken. Nor am I arguing that multiple corrections need to be made (the more comparisons you examine, the more likely it is you’ll find something). I’m simply asking if these two distributions are different from each other. If you used an ‘ocular statistical’ method (looking at the damn table), you might doubt there’s anything there–and you’re correct! It doesn’t matter what method you use, whether any number of 2×2 comparison methods, resampling, a binomial distribution based method, it’s just not significant. There is no difference here.
Mind you, I’m a Warren supporter, though I would have no problem voting for Sanders if it comes to that. But this is bad framing, based on overreach. While Yglesias might not know enough stats to puzzle this out (though it’s not that hard if you’re paying attention), someone whose Twitter bio mentions teaching RStats should know this.
Of course, if there’s a narrative to plug, then rigor goes out the window I suppose.
This is going to be a long primary season.
*I don’t subscribe to the Economist, so I’m not pissing away my money.