Wednesday, October 1st, 2014

The Principles of PolitiFact, PunditFact and the Truth-O-Meter

We've published more than 7,000 Truth-O-Meter fact-checks since we launched six years ago.
We've published more than 7,000 Truth-O-Meter fact-checks since we launched six years ago.

Editor’s note: We often get questions about how we select claims to check and how we make our rulings. So a couple of times a year, we publish this overview of our procedures and the principles for Truth-O-Meter rulings.

PolitiFact is a fact-checking website that rates the accuracy of claims by elected officials and others who speak up in American politics. PolitiFact is run by editors and reporters from the Tampa Bay Times, an independent newspaper in Florida, as is PunditFact, a site devoted to fact-checking pundits. The PolitiFact state sites are run by news organizations that have partnered with the Times. The state sites and PunditFact follow the same principles as the national site.

PolitiFact staffers research statements and rate their accuracy on the Truth-O-Meter, from True to False. The most ridiculous falsehoods get the lowest rating, Pants on Fire.

PolitiFact checks claims by elected officials, candidates, leaders of political parties and political activists. We examine officials at all levels of government, from county commissioners to U.S. senators, from city council members to the president.

We also check claims by groups involved in the discourse -- political parties, advocacy groups and political action committees -- and examine claims in widely circulated chain emails.

PunditFact checks claims from pundits, columnists, bloggers, political analysts, the hosts and guests of talk shows, and other members of the media.

Choosing claims to check

Every day, PolitiFact and PunditFact staffers look for statements that can be checked. We comb through speeches, news stories, press releases, campaign brochures, TV ads, Facebook postings and transcripts of TV and radio interviews. Because we can't possibly check all claims, we select the most newsworthy and significant ones.

In deciding which statements to check, we ask ourselves these questions:

  • Is the statement rooted in a fact that is verifiable? We don’t check opinions, and we recognize that in the world of speechmaking and political rhetoric, there is license for hyperbole.

  • Is the statement leaving a particular impression that may be misleading?

  • Is the statement significant? We avoid minor "gotchas" on claims that obviously represent a slip of the tongue.

  • Is the statement likely to be passed on and repeated by others?

  • Would a typical person hear or read the statement and wonder: Is that true?

 

Transparency and on-the-record sources

PolitiFact and PunditFact rely on on-the-record interviews and publish a list of sources with every Truth-O-Meter item. When possible, the list includes links to sources that are freely available, although some sources rely on paid subscriptions. The goal is to help readers judge for themselves whether they agree with the ruling.

 

Truth-O-Meter rulings

The goal of the Truth-O-Meter is to reflect the relative accuracy of a statement.

The meter has six ratings, in decreasing level of truthfulness:

TRUE – The statement is accurate and there’s nothing significant missing.

MOSTLY TRUE – The statement is accurate but needs clarification or additional information.

HALF TRUE – The statement is partially accurate but leaves out important details or takes things out of context.

MOSTLY FALSE – The statement contains an element of truth but ignores critical facts that would give a different impression.

FALSE – The statement is not accurate.

PANTS ON FIRE – The statement is not accurate and makes a ridiculous claim.

 

Principles in Truth-O-Meter rulings  

Words matter – We pay close attention to the specific wording of a claim. Is it a precise statement? Does it contain mitigating words or phrases?

Context matters – We examine the claim in the full context, the comments made before and after it, the question that prompted it, and the point the person was trying to make.

Burden of proof – People who make factual claims are accountable for their words and should be able to provide evidence to back them up. We will try to verify their statements, but we believe the burden of proof is on the person making the statement.

Statements can be right and wrong – We sometimes rate compound statements that contain two or more factual assertions. In these cases, we rate the overall accuracy after looking at the individual pieces.

Timing – Our rulings are based on when a statement was made and on the information available at that time.

 

Process for Truth-O-Meter rulings

A writer researches the claim and writes the Truth-O-Meter article with a recommended ruling. After the article is edited, it is reviewed by a panel of at least three editors that determines the Truth-O-Meter ruling.

 

Corrections and review

We strive to make our work completely accurate. When we make a mistake, we correct it and note it on the original item. If the mistake is so significant that it requires us to change the ruling, we will do so.

Readers who see an error should contact the writer or editor. Their names are listed on the right side of every Truth-O-Meter item. Clicking on their names will take you to their bio pages, where you can find their email addresses.

When we find we've made a mistake, we correct the mistake.

  • In the case of a factual error, an editor's note will be added and labeled "CORRECTION" explaining how the article has been changed.

  • In the case of clarifications or updates, an editor's note will be added and labeled "UPDATE" explaining how the article has been changed.

  • If the mistake is significant, we will reconvene the three-editor panel. If there is a new ruling, we will rewrite the item and put the correction at the top indicating how it's been changed.

We respect that reasonable people can reach different conclusions about a claim. If you disagree with a ruling, we encourage you to email the writer or editor with your comments about our ruling. You can also post comments to our Facebook page or write a letter to the editor. We periodically publish these comments in our Mailbag feature.

PolitiFact has two other features:

  • The Flip-O-Meter, which rates whether an elected official has been consistent on an issue.

  • Promise meters, such as the Obameter and the GOP Pledge-O-Meter, that rate the status of elected officials' campaign promises.

 

The Flip-O-Meter

The Flip-O-Meter rates an official's consistency on an issue. The rating is not making a value judgment about a politician who changes positions on an issue. Indeed, voters often like politicians who are flexible and have the ability to compromise or adapt their positions to the wishes of constituents. Still, accusations of shifting positions are so common in politics that it is valuable to have us provide an analysis of a shift and rate the amount of change. The Flip-O-Meter has three ratings:

  • No Flip – No significant change in position.

  • Half Flip – A partial change in position.

  • Full Flop – A complete change in position.

The writing, editing and rating process for Flip-O-Meter items is the same as the process for Truth-O-Meter items.

The Promise meters

To create our promise meters, staffers pore through speech transcripts, TV appearances, position papers and campaign websites looking for promises.

However,  a promise is not a position statement. We define a promise as a prospective statement of an action or outcome that is verifiable. All of our promises list the source.

The promise meters have six levels. The first three provide a broad picture of whether the official is making progress; the final three indicate whether he or she kept the promise.

Not Yet Rated — Every promise begins at this level and retains this rating until we see evidence of progress — or evidence that the promise has stalled.

In the Works — This indicates the promise has been proposed or is being considered.

Stalled — There is no movement on the promise, perhaps because of limitations on money, opposition from lawmakers or a shift in priorities.

Compromise — Promises earn this rating when they accomplish substantially less than the official’s original statement but when there is still a significant accomplishment that is consistent with the goal of his original promise.

Promise Kept — Promises earn this rating when the original promise is mostly or completely fulfilled.

Promise Broken – The promise has not been fulfilled. This could occur because of inaction by the executive or lack of support from the legislative branch or other group that was critical for the promise to be fulfilled. A Promise Broken rating does not necessarily mean that the executive failed to advocate for the policy.

Promise ratings change whenever the circumstances change. For some promises, it's possible that the status could initially go to In the Works, but then move back to Stalled if we decide the proposal has hit a lull, and then go back to the In the Works.