What Do Focus Groups Actually Tell Us?

One of the tools that people interested in framing use is the focus group. I’ve always been wary of them because I think researchers and the participants can strongly bias your results. Joe Klein described a focus group involving Colorado independents. Klein describes the group:

There was some doubt among commenters that these independents were actually…independent, especially since a majority of them voted for Bush last time–but that’s how Independents broke in Colorado in 2004. Four years ago, I attended a Hart focus group of undecideds in Kansas City and it soon became apparent that they weren’t really–they had decided against John Kerry, even if they weren’t admitting it to themselves. By contrast, these folks seemed truly undecided–downright resentful when Luntz tried to push them for a final choice. They were adamant–and I should have included this above–that the debates will be crucial when it comes to making a decision….
this was a focus group for undecided voters…so it was a snapshot of a sliver–a crucial sliver–of the electorate. I thought the results were credible, if a bit depressing…but you go to the polls with the electorate you have, not the electorate you’d like to have.

First, the group wasn’t happy with Obama’s rhetoric–they wanted specifics:

“Change” as a theme is over. Too vague. And Obama’s rhetoric has begun to seriously cut against him. “No more oratory,” one woman said. “Give us details.” (There may be a racial component to this, by the way, as some white people associate soaring oratory with African-American leaders like Martin Luther King Jr. and Jesse Jackson.)

Fair enough. I like specifics; it’s how you make a decision. But when asked what matters, issues… weren’t…important (boldface mine):

What do they want? Given a list of 31 personal attributes the next President might have and asked to pick the eight most important, “Accountability” finished highest with 13 votes, next was “Someone I can trust” with 12, “honest and ethical” was third with 11. “Agrees with me on the issues” got one vote. They didn’t care if the candidate was a Washington insider or outsider. “A dynamic and charismatic leader” got two votes…(Add: When Luntz asked them which was more important, “accountability” or “change,” the vote was 17 to 4 in favor of accountability.)

How does that square with wanting “details”? I really have to wonder what this means. To me, it sounds like a lot of rationalization, combined with the phenomenon of a bunch of people trapped in a room who: 1) don’t really know what they’re talking about; and 2) are encouraged to complain. There has to be a better way of figuring out what voters want than this.
More importantly, what is staggering, and ultimately depressing is the criteria that these swing voters use–issues don’t matter. They’re not using heuristic shortcuts to determine possible positions on issues, they fundamentally do not think that policy is what politics is about. There is one caveat that provides a glimmer of hope, however:

In this case, I had one gripe with Luntz: he didn’t show Obama’s or McCain’s solutions to the issues he focused on–health care and social security (the group was sponsored by AARP, hence the issue focus). When I asked the group later how they’d feel about raising taxes on people with incomes over $250,000–Obama’s solution to both problems–they split down the middle.

I do wonder how different our politics would be if these swing voters were ‘forced’ to decide how to vote based on policy and not personality traits. And that brings me to something Ezra Klein discussed regarding the media (boldface mine; italics original):

It’s sort of like a TV show: If Friends had had an episode where Ross and Rachel hooked up, but never mentioned it again, that would’ve been weird, but their tryst wouldn’t have been a big part of the story. Since they mentioned it all the time, and came back to it, and fit future events into that context, it was a big story. Similarly, if the press reports something and never mentions it again, the public knows to forget it. It’s not important. If they mention it constantly — “I voted for the $87 billion before I voted against it” — they know it is important. The job of the media, in other words, is now to also emphasize the right parts of the story.
This requires deciding what matters.

Too bad our political discourse is ruled by the Gang of 500 Mediocrities. As Maha put it:

The future of humanity, and possibly of our planet, may hinge on the results of the November election. And this being America, the campaign has devolved into adolescent accusations that one candidate called another candidate a “pig.”
I can pretty much guarantee that many hours of television programming today will be dedicated to serious discussion of whether Barack Obama intentionally called Sarah Palin a “pig” — a phony controversy generated by the McCain campaign that could be dismissed in a few seconds with a simple review of what Barack Obama actually said.
I can pretty much guarantee that at no time today will any of the major cable news networks dedicate even a few seconds to substantive discussion of the candidates’ positions on health care, even though Americans place health care very high on their list of concerns.
The McCain campaign consists mostly of frantically throwing red herrings in all directions, hoping no one notices that John McCain and his moose-shootin’ sidekick have no idea how they might govern. And this is working very well for them, it seems. The American public has gotten so used to content-free campaigns they think this is normal.
Over the years Americans have been conditioned to respect utter nonsense, because they see our national leaders and the “pundits” in mass media respecting utter nonsense. If by some miracle we woke up tomorrow morning in a world where our leaders were engaged in sincere, factual, and substantive discussion of issues, most Americans would be dumbfounded.

Sigh.

This entry was posted in Framing. Bookmark the permalink.

7 Responses to What Do Focus Groups Actually Tell Us?

  1. Joel says:

    Here’s another way to look at it.
    If you as a politician connect with voters on a gut level, they will follow you anywhere and not fret about the details. If you donít connect with them on a gut level, you canít show them enough details. Obama early on, and particularly with young people, connected on a gut level like no other politician since Ronald Reagan.
    http://www.nytimes.com/2008/09/10/opinion/10friedman.html?_r=1&ref=opinion&oref=slogin

  2. Most good research initiatives will start with focus groups or in depth interviews and then follow with a series of surveys. Other research on framing has been carried out in experimental settings. Each method had drawbacks and advantages, like anything in science, you are looking for a triangulation of methods and results.

  3. SimonG says:

    Is picking a president on personalities rather than issues such a bad idea? He’s going to have to make decisions about all manner of events which haven’t happened yet. His character might tell you more about that than his party’s policies.
    Perhaps even more important are the types of advisors that surround him. These are the people who provide him with the information to make decisions: he’s unlikely to come up with something original himself, just decide between the options they suggest.

  4. QrazyQat says:

    Is picking a president on personalities rather than issues such a bad idea? He’s going to have to make decisions about all manner of events which haven’t happened yet. His character might tell you more about that than his party’s policies.
    How do you find out about a candidate’s character? The process you describe is exactly what got us GW Bush twice, as the media narrative told people that GW was just dandy and Gore and Kerry were dull and earnest liars. If you think the last 8 years of presidency in the USA worked out great, then do it that way. I think that approach led to disaster, and we’re now trying to replicate that disaster via the media coverage of McCain and Palin.

  5. QrazyQat says:

    And BTW, your second paragraph was exactly, and I mean practically word for word, the reasoning given to me by a guy who voted for Bush in 2000 despite misgivings about the candidate.

  6. Eric Lund says:

    Is picking a president on personalities rather than issues such a bad idea? He’s going to have to make decisions about all manner of events which haven’t happened yet. His character might tell you more about that than his party’s policies.
    What QrazyQat said.
    The flaw is that we don’t necessarily see the candidates’ true character, but rather the character the spinmeisters want us to see. Those who take the trouble to look behind what the MSM was telling us could see GWB for what he was. But most of the electorate got the view pushed by the spinmeisters that GWB was a trustworthy sort that you’d want to have a beer with and Gore was an aloof liar. Never mind that this view had no basis in reality.
    Ditto the advisors. The true character of GWB’s advisors was not on display during the campaign (neither was Gore’s, but it was obvious that Bush was relying more heavily on advisors than Gore).

  7. QrazyQat says:

    Actually, in the case of my “friend” (I like the guy in some ways but his political dimness requires the quotes) he thought that GW’s experience in business and that of his known advisors and likely administration fillers would be great. Now it was easily known that GW failed at every business he tried except for the baseball team where they got the government to pry people’s houses and a family farm away from them, and that the others had similar government-dependent business experience, but that didn’t sink in his brain. Cheney, for instance, simply made business out of government deals and getting around US laws about dealing with Iran; Rumsfeld, for instance, worked around laws to sell nuclear tech to the North Koreans. This kind of business experience shouldn’t insprie confidence.
    For that matter business experience isn’t a good indicator of governing ability; government and business work very differently, and should. In fact government has to. In government you shouldn’t, for example, serve only the easiest to serve customers like business typically does, cutting the others loose and leaving them to be served by some other entity which specialises in niche markets, or simply not be served at all. That’s far from the only way businesses work in ways government shouldn’t, but it’s a major one. It doesn’t mean that a good business person can’t be a good government person, but it’s just not a good indicator, and in fact many aspects of their business experience teach them methods and habits that have to be overcome to be good at governing.

Comments are closed.