The only poll that matters, dealers in political cliches tell us, is the one taken on Election Day. For third party candidates, the Committee on Presidential Debates has imposed another consequential metric: they must reach 15 percent in a select set of polls if they want access to the debates and the attention and credibility that are attached.
Right now, it looks like Clinton and Trump will be alone on stage come debate season. But if the polls were just conducted differently, Libertarian presidential candidate Gary Johnson has suggested he would easily meet that requirement.
"None of the polls being conducted right now have us on the top line. None of them. It's always Trump and Clinton and then second question, third question, ‘Well, what if you add Johnson-Weld?’," Johnson told CNBC on August 22.
In a couple of other cases Johnson specified how well he thinks he’d be doing if he were listed as an option in pollsters’ first horse-race question: 20 percent, more than enough to qualify for the debates.
"I think if we were included in the top line, as Johnson/Trump/Clinton, we'd be at 20 percent. A lot of that has to do with how polarizing the two of them are. But that's the issue right now. We need to be top line on the polls," Johnson said in another CNBC interview on Aug. 11.
This would be quite a swing. Theodore Roosevelt was the last third-party presidential candidate to register more than 20 percent of the vote on Election Day, running as a Bull Moose in 1912, and Ross Perot reached 18.9 percent in 1992. We decided to look at whether the polls are really being conducted the way Johnson says, and if it really has such a dramatic impact.
We looked at 25 national polls conducted in July and August, chosen off of a list compiled by FiveThiryEight.com. Based on this list, we can say that Johnson is certainly wrong that every poll asks about Clinton and Trump before adding his name in a later question. Eleven of the polls we looked at -- close to half -- included Johnson in their first or only question about the presidential horse-race.
None of these polls vaulted Gary Johnson to 20 percent. In fact, he did slightly worse, on average, when he was included in the first horse-race question. Johnson’s average in the 25 polls was 9 percent. When he was listed in a later horse-race question, his average performance was slightly higher than that average. In the polls where he was listed in the first or only horse-race question, Johnson did slightly worse on average.
Some of his best performances -- 12 percent in polls conducted for Fox News and International Business Daily and 13 percent in a poll conducted for CNN -- were in polls that listed him in a later question.
We ran Johnson’s theory by several academics who study polling. While research hasn’t been done on the specific question Johnson addresses, they told us that question order could have an effect on polling results. All of them were, however, skeptical that the effect could be as large as Johnson suggested.
"It's not a totally crazy notion to think that including him in the first round versus not might affect his support. As a researcher, I would ask how big an effect that would have to be to get the kind of outcome he said. … You would have to have an effect of about 12 percent shift, which to me is just implausible," Adam Berinsky, a professor of political science at MIT.
Even if there was some sort of bias introduced by the question order, it wouldn’t necessarily be in his favor.
"(Trump and Clinton) have the lowest favorability ratings of presidential candidates in recent memory. So I could spin a story that would suggest if you first ask about Clinton or Trump without Johnson, you kind of prime people to think ‘I don’t really like either of these folks.’ And then you include the question with him in it, and they say, ‘Oh, there’s another option," Berinsky said.
When reached for comment about this article, Johnson communications director Joe Hunter wrote in an email that, "As far as the issue of whether a third candidate is included in the first ballot test question, or added to a subsequent question, we believe it does, indeed, make a difference, and pollsters with whom we work agree."
Since Johnson’s likely to appear on the ballot in all 50 states, it's worth considering why any pollster puts forward a horse-race question without including his name. Support for third party candidates tends to flag down the stretch, with many of their supporters leaving for one of the major party options. Questions that only offer those candidates allow pollsters to get a sense of how people currently registering support for third parties might break if that happens.
It's worth noting that Johnson seems to be bucking this trend, according to a recent article by the polling analysts at 538. Based on current polling, 538’s polls-plus general election forecast, which adjusts for economic factors and historical trends, expects Johnson to earn a bigger percentage of the vote than any third-party candidate since Ross Perot in 1992.
Johnson said that polls ask about a straight Clinton/Trump race first, cueing them to ignore his name when it appears later. He said that including his name in the first questions pollsters asked about the horse-race could increase his support to 20 percent.
Many polls do include Johnson’s name in their first horse-race poll, and Johnson does not do better in these polls. Experts say the jump Johnson is talking about is implausible. We rate the claim False.