This week, House Republican Majority Whip lost to David Brat, who is quite the wingnut. While the real story is that there’s a new political infrastructure undergirding the Republican Party–and Brat was hooked into that–much of the shock resulted from the Republican pollster McLaughlin & Associates missing the outcome by twenty two points.
As you might imagine, Republican pollster Frank Luntz has something disingenuous to say about that (boldface mine);
The simple truth remains that one in 20 polls — by the simple rules of math — misses the mark. That’s why there is that small but seemingly invisible “health warning” at the end of every poll, about the 95 percent confidence level. Even if every scientific approach is applied perfectly, 5 percent of all polls will end up outside the margin of error. They are electoral exercises in Russian roulette. Live by the poll; die by the poll.
There’s winding up outside the margin, and then there’s winding up half way across the country. While this could be really bad luck–there’s no way to rule that out conclusively–it’s probably a screwed up polling design. If we assume that the margin of error (i.e., the standard error) was ± 4 points, we’re talking about a 5.5 sigma shift here. In English, that’s somewhere in the neighborhood of 1 out of 52 million.
Either these were the unluckiest pollsters ever, or they screwed up their polling sample design.
Which do you think it is?