Last week, a YouGov poll announced that twenty percent of Trump supporters disagreed with the freeing of slaves after the Civil War. I won’t deny that a huge (YOOGE!) component of Trump’s appeal is Palinist–that is, white nationalist (though I would argue the entire Republican Party is currently a white nationalist party, but I digress). But something didn’t seem quite right, and when you look at how the poll was conducted, there’s a cautionary tale in all of this (boldface mine):
In the poll, the question about freeing slaves is the 49th of 103 questions that respondents were asked. It also frames the issue in terms of the current controversy over executive orders [the question asked “Do you approve or disapprove of the executive order which freed all states which were in rebellion against the federal government”]….
So it’s a little bit understandable that people who weren’t quite paying close attention during a long poll and/or had just voiced their opposition to executive orders might give an answer to an executive-order question that made it sound like they were pro-slavery. In fact, YouGov’s own breakdown (see item 128) shows that 5 percent of black respondents told its pollsters that they disapproved of the order that freed America’s slaves. Are 2 million black Americans really pro-slavery?
…But it might not be literally true that 20 percent of Trump voters across the country are pro-slavery.
Several things to keep in mind about polls, especially when they’re long:
1) There’s no opportunity to ‘check your work.’ If you had to answer over 100 questions, what are the odds that you would make a mistake? (I will leave to the reader as an exercise to determine if the same principle applies to computerized high-stakes educational testing).
2) People will pay more attention/apply themselves harder to certain questions. It’s worth noting that fifteen percent of Hispanics also disapproved of the Emancipation Proclamation–which also strains credulity. The word “slavery” might attract more attention among black respondents, leading to lower disapproval–that is, a more accurate answer.
3) Error rates on telephone polls–not sampling error, but respondent error–are not trivial, depending on the length, the recording format, and the order of questions. Alternatively, we could believe two million black people think the Emancipation Proclamation was wrong.
So that ends today’s episode in understanding the limitations of your data.
mikethemadbiologist