Readers say we were uninformed about Jon Stewart's claim

When we gave a False rating to a statement by Daily Show host Jon Stewart, the response from readers was swift and virtually unanimous. They said we were wrong.

We rated a statement Stewart made on the June 19, 2011, edition of Fox News Sunday during an interview  with Chris Wallace.

"Who are the most consistently misinformed media viewers?" Stewart asked Wallace. "The most consistently misinformed? Fox, Fox viewers, consistently, every poll."

One reader wrote, "I'm afraid I'm rather disappointed with this article. The methodology is fine, no question. You gather the facts, you report fairly what you find. The problem is you were analyzing the wrong facts."

"Again and again you say in this article the word ‘ill-informed,’" the reader continued. "The problem is that Stewart did not, in fact, choose this word. He actually chose misinformed, which is a bit different, at least to my mind. When I hear ‘ill-informed,’ I think of receiving little information. I might use that word to speak of someone who pays little attention to the news. When I hear the word ‘misinformed,’ I think of something else quite different. That brings to mind someone who has been informed, but with the wrong information.

In checking Stewart's comment, we found two polling organizations that have produced periodic "knowledge" surveys differentiated by the respondent’s frequent news sources. One is the Pew Research Center for the People and the Press, and the other is worldpublicopinion.org, a project managed by the Program on International Policy Attitudes at the University of Maryland.

We found three Pew studies that ranked Fox viewers low on the well-informed list, but other general-interest media outlets -- such as network news shows, network morning shows and even the other cable news networks -- often scored similarly low. Meanwhile, viewers of particular Fox shows -- such as The O’Reilly Factor and Sean Hannity’s show -- scored consistently well, occasionally even outpacing Stewart’s own audience.
   
Meanwhile, we concluded that the other set of knowledge surveys, 2003 and 2010 surveys from worldpublicopinion.org, offered mixed support for Stewart. The 2003 survey struck us as pretty solid, but we noted some methodological critiques with the 2010 survey.
   
Ultimately, we ruled that for Stewart’s comment to qualify as accurate, the data had to be "consistent" -- a term he used not once but three times. Since not "every poll" showed that result, we rated his claim False.

The striking thing about the comments we received -- almost 100 e-mails plus countless tweets and Facebook postings during the first 24 hours -- is how quickly readers zeroed in on a distinction our piece did not draw. They said we erred by failing to distinguish between a viewer being "misinformed" and a viewer being "uninformed" or "ill-informed."

Generally speaking, readers viewed the three Pew studies -- which we saw as some of the strongest evidence undercutting Stewart’s position -- as irrelevant because they were testing factual knowledge, and if someone got one of those questions wrong, it wasn’t because they had been "misinformed" but rather because they were simply "uninformed."

By contrast, many readers argued that the only legitimate way to test Stewart’s thesis was to refer to the worldpublicopinion.org studies, since they tested the extent of beliefs that were the result of misinformation, whether it’s Barack Obama being born in Kenya or the stimulus not creating or preserving a significant number of jobs.

Steven Kull, the director of WorldPublicOpinion.org, wrote us after the story appeared to say that testing for lack of knowledge is not enough. "We analyzed the effect of increased exposure to news outlets. We found that with all other outlets, increased exposure generally resulted in less misinformation. However, for Fox viewers, on nine points of information, increased exposure correlated with increased misinformation. This was true of only one point of information for public broadcasting and MSNBC viewers, and two points of information for network news. This effect was found in the 2003 study as well. Fox viewers were the only group for whom increased exposure resulted in greater misinformation."

Kull added that, "simply on face value, such issues as knowledge of who was vice president are unlikely to be related to exposure to news outlets. Information on issues that were very foreground in media reporting and analysis, which ours were, are more likely to be related to exposure" to media outlets.

Readers sound off

Jane Hamsher, who blogs at the liberal site Firedoglake.com, spoke for many readers in a blog post she published the morning after our story appeared.

Here are a few additional comments from those who wrote to us directly:

The reader quoted above, continued: "What your analysis shows is that Fox viewers are not ill-informed. They are, indeed, receiving news.  What's more in question is the level of misinformation they receive. Things like who's in charge of the House of Representatives -- well, there's not really anyone who could try to spin that. There are some levels of reality so patently obvious that even the most stringent partisans have to bow to it. This is not what Stewart was referring to. He was referring, rather, to many of the same statements that end up  here for you to have to adjudicate -- things such as ‘the government is taking over health care’ and ‘the President was born overseas.’ In matters such as this, I believe you'll find Mr. Stewart to be correct. Or at least not clearly false."

One reader drew the contrast this way: "To give you a simple, though slightly extreme, example, an ill-informed person wouldn't be able to name the president, but a misinformed person would think he was a Kenyan."

Another offered this comparison: "If I tell you, ‘Nancy Pelosi, the former Speaker of the House, is trying to pass legislation which will send troops to your house to murder you,’ I have been spreading misinformation, and if you believed me, you would have been misinformed. It doesn't matter that you know Nancy Pelosi was the Speaker of the House -- that doesn't negate your status as ‘misinformed.’"

Some readers thought we overlooked Stewart’s underlying argument.

"His point, I believe, was that when given an opportunity to push their agenda, Fox is able to spin their message and implant in their viewers minds some very distorted ideas," said one reader. "The ‘birther’ issue, for example, was perpetuated by Fox News repeatedly giving (Donald) Trump, as well as others, a platform to make ridiculous accusations."

Another reader agreed: "Asking questions such as who is the vice president and the secretary of state is an EXTREMELY low standard to use to determine how informed people are. Stewart's claim may not have been literally true, but his overall point that Fox News consistently misleads its viewers seems to be at least Half True."

A few compliments

Finally, a tiny minority of people who wrote us offered praise for our work.

"All of us—Democrat and Republican alike—can get stuck in our preconceptions," wrote one. "I think it's great that you challenge us and keep us honest. Thank you!"

And one commenter on Facebook wrote, "I think it says a lot for PolitiFact that they can deliver a message that I initially disagree with and bring me around. Not many organizations can do that."