 Ruy Teixeira |
(covering polls and related articles from the week of August 30–September 5)
In this edition of Public Opinion Watch:
• Bush’s Convention Bounce Worst Ever for Incumbent President
• Adventures in Likely Voter Land
Bush’s Convention Bounce Worst Ever for Incumbent President
SRBI poll of 1,128 registered voters for Time magazine, released September 3 (conducted August 31–September 2)
Princeton Survey Research poll of 1,008 adults for Newsweek, released September 4 (conducted September 2–3)
Gallup poll of 1,018 adults for CNN/USA Today, released September 6 (conducted September 3–5)
The new Gallup poll, conducted entirely after the GOP convention and therefore the first poll that truly measures Bush’s bounce, shows Bush with a very small bounce indeed: 2 points, whether you look at registered voters (RVs) or likely voters (LVs). His support among RVs has risen from 47 percent before to 49 percent after the convention, so that he now leads Kerry by a single point (49-48) rather than trailing by a point.
Note also that Bush’s 2-point bounce from his convention (which is defined as the change in a candidate’s level of support, not in margin) is the worst ever received by an incumbent president, regardless of party, and the worst ever received by a Republican candidate, whether incumbent or not (see this Gallup analysis for all the relevant historical data). In 2000, Bush received an 8-point bounce. And even his hapless father received a 5-point bounce in 1992.
The poll contains other results that suggest the relative ineffectiveness of the GOP convention.
Bush’s acceptance speech, which the media fawned over so ostentatiously, was not rated any better by the public than was Kerry’s—in fact, it received slightly worse ratings. Kerry’s acceptance speech was rated excellent by 25 percent and good by 27 percent; Bush’s was rated excellent by 22 percent and good by 27 percent.
In terms of whether the Republican convention made voters more or less likely to vote for Bush—the real point of the convention, after all—there were almost as many saying the convention made them less likely to vote for Bush (38 percent) as said it made them more likely (41 percent).
This is actually quite a poor performance. The Democratic convention this year had a substantially better 44 percent more likely/30 percent less likely split. In fact, looking back to 1984, which is as far back as Gallup supplies data, no candidate has ever had a more likely to vote for/less likely to vote for split even close to as bad as Bush’s this year.
Well, what about the tone of the convention? Do voters think the Republicans got that one right? Nope. Just 39 percent think the GOP maintained the right balance between criticizing the Democrats and saying positive things about themselves, compared to 50 percent who think they spent too much time criticizing the Democrats. By contrast, in 2000, 45 percent thought the GOP maintained the right balance in their convention, compared to 38 percent who thought they spent too much time criticizing.
But this unfavorable judgment on the GOP’s tone this year is not without precedent. In 1992, just 26 percent thought the Republicans maintained the right balance in their convention, compared to 56 percent who thought they spent too much time criticizing. And we know what happened to the Republican candidate that year.
The internals of the poll provide further indicators that the effectiveness of the GOP convention was vastly overestimated by the media. For example, Kerry’s lead in the battleground states actually widened over the course of the convention. Prior to the Republican convention, Kerry had a 1-point lead among RVs (47 percent to 46 percent) in the battleground states. After the Republican convention, once the battleground voters had a chance to take a closer look at what Bush and his party stand for, Kerry now leads by 5 percent in these same states (50 percent to 45 percent)! Note that Kerry gained 3 points among battleground voters, while Bush actually got a negative 1-point bounce.
And wait—there’s more! The Gallup poll’s internals also show that Kerry continues to lead among independents (49 percent to 46 percent) and that both parties’ supporters are equally polarized for their respective candidates (90 percent to 7 percent). Note that these findings directly contradict the results of the recent Newsweek poll (see below), which showed Bush doing much better among Republican partisans than Kerry was doing among Democratic partisans. Note also that, given the equal polarization of partisans and Kerry’s lead among independents, the only possible reason Bush has any lead at all among Gallup’s RVs must be because their sample has a GOP advantage on party identification (my guess is 5 points) that is inconsistent with almost all other polling data from this campaign season (see below for more discussion of this issue).
Indeed, if equal polarization of partisans continues and Kerry carries a 3-point lead on independents into the election, he’ll win fairly easily, since the Democratic proportion of voters in presidential elections is always higher, not lower, than the Republican proportion. In 2000, after all, Bush carried independents by 2 points and received stronger support from his partisans than Gore did from his—but still lost the popular vote by half a point.
In light of these findings, the big story from the convention seems clear, right? Bush got a disappointingly small bounce from his convention—indeed, the smallest bounce ever by some important historical standards—and it was a relatively ineffective convention. But not if you’re writing stories at USA Today. Apparently, you dare not contravene the current conventional wisdom about the campaign (Bush surges ahead!) no matter what your own data says.
That’s why we get a story like “Bush Leads Kerry by 7 Points,” which prominently features the LV results (where Bush does have a 7-point lead) and resolutely refuses to dwell on Bush’s historically poor result from his convention or on his almost nonexistent lead among RVs.
Where did this conventional wisdom that the media is now in thrall to come from? It came from two news magazine polls—Time and Newsweek—that were conducted in whole or in part during, not after, the GOP convention, but were nevertheless seized on by the media as evidence that Bush had received a large bounce from his convention and now led Kerry by a wide margin.
I consider both of these polls and their many problems below. First, let’s look at the Newsweek poll, which was conducted September 2–3 and had Bush ahead 54 percent to 43 percent among RVs. There were two big problems with this poll, neither of which were covered (or probably even understood) by the mainstream media.
First, it was not a true bounce poll; only one night of the two covered by the poll actually took place after the GOP convention was over. That night was highly likely to be Bush’s best post-convention night, since it was right after his big speech and the huge media splash the next day. And, in fact, Newsweek’s data show that Bush led by 16 points in its poll on this night and by only six the night before. Kerry also did very well in polls the night after his acceptance speech, but then fell off rapidly in the next few days.
So why did Newsweek (and Time—see below) insist on doing their bounce polls wrong so they were almost guaranteed to get misleading results? Simple: their publication schedule. They had to have data in time to dump it into their print publication. If they had waited to do it right, the poll would not have made it into their post-convention issue.
Second, aside from the timing, there were other reasons to be skeptical of the Newsweek poll. For example, the partisan distribution of the RVs in the Newsweek poll was quite startling: 38 percent Republican, 31 percent Democratic, and 31 percent independent. This 7-point lead for the GOP on party identification did not comport well with other data on partisan distribution this campaign season—which have consistently shown the Democrats leading by at least several points—and can’t be blamed on a likely voter screen since there was none.
As Chris Bowers of MyDD shows, if you assumed a more reasonable distribution of party identification, Bush’s lead in this poll was about cut in half. Moreover, if you assumed that the differential in partisan support rates in the poll—94 percent to 4 percent for Bush and only 82 percent to 14 percent for Kerry—was highly likely to converge toward parity in the near future (indeed, the new Gallup data show complete parity in this respect; Republicans and Democrats support their candidates with equal intensity) even a Bush lead of 5 to 6 points looked very unstable.
So how did Newsweek manage to pull a sample with a 7-point GOP lead on party identification? It is certainly possible that there was a sudden, large shift in party identification to the Republicans; the distribution of party identification is not completely stable and does indeed change over time. But a shift of this magnitude so suddenly and so off-trend (which has been toward the Democrats) was quite unlikely. I find it more plausible that there was differential interest in being interviewed by Democratic and Republican voters over the time period and that produced a skewed distribution of partisan identifiers in their RV sample.
Does that mean I favor polls like this weighting their samples by party identification? No, I don’t, because the distribution of party identification does shift some over time and polls should be able to capture this. What I do favor is release and prominent display of sample compositions by party identification, as well as basic demographics, whenever a poll comes out. Consumers of poll data should not have to ferret out this information from obscure places—it should be given up front by the polling organizations or sponsors themselves. Then people can use this information to make judgments about whether and to what extent they find the results of the poll plausible.
The Time poll was conducted August 31 to September 2—that is, entirely during the GOP convention—and had Bush ahead by 11 points, 52 percent to 41 percent, in a three-way LV match up (the most widely publicized finding), as well as 10 points ahead in a two-way LV match up and 8 points in an RV match up. It was these findings that really got the ball rolling on the idea that Bush was getting a huge convention boost and was now far ahead of Kerry.
How plausible were these results?
Well, it’s certainly possible that Bush was as far ahead during the convention as this poll suggested. But all other available polls taken during the convention contradicted that result.
In an attempt to compare apples to apples, here are Bush-Kerry results from contemporaneous three-way LV match ups (except Rasmussen, where only a two-way LV result is available), with Bush’s margin in parentheses:
Zogby, August 30 to September 2: 46 percent Bush to 43 percent Kerry (+3)
ARG, August 30 to September 1: 47 percent Bush to 47 percent Kerry (tie)
Rasmussen, August 31 to September 2: 49 percent Bush to 45 percent Kerry (+4)
In this company, 52 percent Bush to 41 percent Kerry (+11) certainly sticks out. Could it have anything to do with the different dates included in these surveys, even though they were very close? Well, the Rasmussen data were from exactly the same period as the Time data (August 31 to September 2).
But if you are skeptical of the Rasmussen data, consider the Zogby data. The Zogby data only included an additional day (August 30) when compared to the Time data. But perhaps August 30 was a very pro-Kerry day since the Republican convention had just started. However, for Zogby and Time to match up (have Bush leading by 11) for the three days they share, Kerry would have had to be leading by about 21 points in Zogby on the day (August 30) they didn’t share. This seems highly implausible.
The simplest hypothesis, then, is that the Time poll, for that period, was exceptionally pro-Bush. It should have been viewed with skepticism by the media but, of course, was viewed with anything but skepticism by reporters hungry for the next big story. They should have paid more attention to what Gallup had to say in an analysis released during the Republican convention:
“Based solely on history, the Bush–Cheney ticket could expect to gain five to six points among registered voters after this week’s convention. That would result in a 52% to 53% support level for Bush among registered voters, up from 47% in the pre-convention poll.
However, the results from Gallup’s post-Democratic convention poll showed that history might not apply in 2004, a year in which the electorate was activated long before the conventions (usually the conventions serve to activate voters), and a year in which relatively small proportions of undecided and swing voters are available to the two presidential tickets. Also, the post-Democratic convention poll suggested that the Democratic convention might have helped energize Republican voters. It is unclear whether the Republican convention could have a similar paradoxical effect on Democrats, or if Republicans will be activated, as is typically the case.”
Of course, it now turns out that Gallup’s cautions were fully justified. Bush failed to get the anticipated bounce from his convention and the race is now about tied among voters nationwide. But you’d never guess that from current media coverage of the campaign—they’re still consumed with last week’s storyline about Bush’s big convention bounce and wide lead.
And they’re likely to remain so until the polls start moving in Kerry’s direction, at which point they’ll forget everything they’re currently saying and act like Kerry’s got it locked up.
Too bad the media don’t pay more attention to the details of polling data and the various ways in which these data can be mis- and over-interpreted. It would be fairer to the candidates and much fairer to the voters, who are continually told the race is over when it’s not.
Adventures in Likely Voter Land
It has not escaped my notice that many people are puzzled as to how exactly polls go about determining likely voters (LVs). There’s a good reason for this: polling firms or sponsors rarely put much effort into explaining, clearly and precisely, the mechanics of how they select these LVs.
So, as a public service, here’s how they do it. Let’s start with Gallup since, as noted above, it is currently showing a wide disparity between its RV and LV results. According to David Moore of Gallup:
“Gallup asks each [RV] respondent seven LV screening questions, and gives each person an LV score of 0 to 7. [Assuming a turnout of 55 percent], the top 55% are classified as likely voters.”
Here are the seven LV screening questions:
1. SALIENCE: How much thought have you given to the upcoming election for president?— quite a lot, or only a little? (“Quite a lot” or “Some” as a volunteered response score 1 point.)
2. KNOWLEDGE: Do you happen to know where people who live in your neighborhood go to vote? (“Yes” scores one point.)
3. BEHAVIOR: Have you ever voted in your precinct or election district? (“Yes” scores one point.)
4. BEHAVIOR: How often would you say you vote—always, nearly always, part of the time, or seldom? (“Always” or “Nearly always” scores one point.)
5. INTENTION: Do you, yourself, plan to vote in the presidential election on November (*), or not? (“Yes” scores one point.)
6. BEHAVIOR: In the [last] presidential election, did you vote for (*) or (*), or did things come up to keep you from voting?
7. INTENTION: I’d like you to rate your chances of voting in the upcoming election for president on a scale of 1 to 10. If “1” represents someone who definitely will not vote, and “10” represents someone who definitely will vote, where on this scale of 1 to 10 would you place yourself?
If respondents answer each of these questions the “right” way, they get a 7; miss one and they get a 6, and so on. In practice that typically means all of the 7s—given full weight—plus some proportion of those with lower scores (usually the 6s), who are weighted down so that the size of the likely voter sample matches the projected turnout for the year (apparently 55 percent this year), are included in the poll. All other voters are discarded from the sample.
That’s how Gallup does it. What about other organizations—do they select likely voters in the same way? Nope, they don’t. CBS News doesn’t use a cut-off model, where low-scoring respondents are thrown out altogether, but instead includes everyone in their RV sample, in some form, in their LV sample. They do this by asking respondents a series of voting-related questions and then assigning each respondent a weight based on their score on these questions, from very high weights for high-scoring respondents to very low weights for low-scoring respondents.
Finally, by far the most common way is simply to ask a few screening questions and then terminate the interview with those respondents who give the “wrong” answers. Or only one question: some likely voter screens are as simple as asking an RV how likely they are to vote in the upcoming election; if they don’t say “almost certain” or “probably,” out they go.
So that’s how they get the likely voters in the polls you read about. How do they know that likely voters, months before the election, are actually the voters who will show up on election day? They don’t.
Here’s David Moore from Gallup again: “We simply do not know, nor can we know, which model is better during the campaign itself.” Exactly. So why does he think the Gallup LV model works so well months and months before the election? Because “if it is the most accurate model just before the election, it is probably the most accurate during the campaign as well.”
But that doesn’t follow at all. The Gallup LV model could work perfectly right before the election (not that it really does, but that’s another discussion) and still be quite a biased instrument earlier in the campaign. Pretty much by definition, Gallup’s LVs months before the election are not the same voters as Gallup’s LVs right before the election, since voters answer the LV questions differently at different stages of the campaign. And if there is any kind of partisan dimension to “tune-in,” so that, say, Democratic partisans or groups that lean strongly Democratic (like minorities) tend to tune in later, that means the LV model will have a systematic tendency to, on average, favor the party (the Republicans) whose partisans or groups tune in the earliest.
Of course, my hypothesis here about Gallup LV bias might be completely wrong. But to evaluate it, Gallup would have to make available the demographics and partisan breakdown of both its RV and LV samples for the polls it releases plus, ideally, the results (including demographics and partisan breakdowns) of the various screening questions it uses. I’m not holding my breath.
Ruy Teixeira is a joint fellow at the Center for American Progress and The Century Foundation.