Organization: PolitiFact
Applicant: Angie Holan
Assessor: Michael Wagner
Conclusion and recommendations
on 17-Aug-2018 (3 months ago)

Michael Wagner wrote:

Politifact is a world-class fact-checking organization that complies with IFCN principles. I recommend renewal. Given that IFCN and PolitiFact share an owner (Poynter), more care should be taken by PolitiFact to be transparent about its funding. IFCN should also note information about ownership when listing PolitiFact as an IFCN approved source.

In addition to a bit more transparency on funding, my assessment offered some ‘food for thought’ for PolitiFact in terms of its ratings system, processes for dealing with complaints and with respect to sharing the transcripts of on-the-record interviews for fact-checks. It might also better serve PolitiFact’s readers, and help PolitiFact’s own overall transparency, to develop a protocol for dealing with complaints about ratings. Some complaints will surely be ideological screeds. However, others will debate the rating on the merits. This second category of complaints deserve some kind of response, whether individually, publicly, or via some other mechanism.

None of these suggestions should be interpreted as requirements for renewal. PolitiFact is, in my view, adhering to IFCN’s principles already. These suggestions are to encourage an industry leader to continue innovating with respect to their methods, transparency and accountability to their audience.

on 17-Aug-2018 (3 months ago)

Michael Wagner recommended Accept

Section 1: Organization

Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

Regarding Requirement A, PolitiFact is an organization set up for the express purpose of fact-checking. On PolitiFact’s site, they note a change in ownership by writing, “PolitiFact had been owned by the Tampa Bay Times, but in 2018 direct ownership of PolitiFact was transferred from the Times to Poynter, which is the newspaper’s parent company. The move allows PolitiFact to function fully as not-for-profit national news organization.” With respect to PolitiFact’s organizational status, the site notes, “Control of both the the Poynter Institute and the Tampa Bay Times lie with a single executive. Upon retirement, that leader picks a successor. Poynter himself picked Eugene Patterson, who picked Andrew Barnes, who picked the Times' current chairman and CEO, Paul Tash, a lifelong journalist.” The site also offers information about donors who have given more than $1,000 to PolitiFact.

done_all 1a marked as Fully compliant by Michael Wagner.

Criterion 1b
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.

20-Mar-2018 (8 months ago) Updated: 3 months ago
Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

Regarding Requirement B, PolitiFact produces fact-checks regularly, usually publishing several fact-checks per week day. With respect to the code of ethics of IFCN, their site notes, “Each day, PolitiFact journalists look for statements to fact-check. We read transcripts, speeches, news stories, press releases, and campaign brochures. We watch TV and scan social media. Readers send us suggestions via email to truthometer@politifact.com. Because we can't feasibly check all claims, we select the most newsworthy and significant ones. About one-third of the facts we choose to check come from reader ideas.

In deciding which statements to check, we consider these questions:

• Is the statement rooted in a fact that is verifiable? We don’t check opinions, and we recognize that in the world of speechmaking and political rhetoric, there is license for hyperbole.

• Does the statement seem misleading or sound wrong?

• Is the statement significant? We avoid minor "gotchas" on claims that are obviously a slip of the tongue.

• Is the statement likely to be passed on and repeated by others?

• Would a typical person hear or read the statement and wonder: Is that true?”

It is my assessment that PolitiFact produces fact-checks that appear to genuinely attempt to follow these principles. While no organization is perfect and while here and there PolitiFact makes editorial judgments with respect to ratings that people of good faith might disagree with, PolitiFact adheres to IFCN’s organization principles.

done_all 1b marked as Fully compliant by Michael Wagner.

Section 2: Nonpartisanship and Fairness

Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

PolitiFact’s fact-checks cover a wide variety of topics, from claims made by President Trump to claims made by prominent bloggers, email chains, politicians and news organizations. A rough content analysis I conducted of a random sample of 50 fact checks suggests no systematic bias toward overcovering any one political party or ideological orientation. When there is an imbalance of who is checked, it appears to be on the side of those in the country’s or a state’s majority party in government. This is an appropriate editorial choice. 

done_all 2a marked as Fully compliant by Michael Wagner.

Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

Regarding the criterion about supporting candidates or issues, PolitiFact allows their staff members to vote. The site does not specifically describe whether staff members can vote in partisan primaries, where voters must declare a party allegiance before casting a vote. It is possible that those interested in the organization could look up the primary voting records of staff members and seek to draw conclusions about the staff members’ politics but I do not think that this fact should alter PolitiFact’s policy as it upends the role of citizen in a democratic republic. PolitiFact prohibits campaign giving, working and volunteering, has a clear social media policy and certainly takes great care to communicate an environment of fairness to its audience.

Note: Two complaints to IFCN about PolitiFact - from the same source - were brought to my attention. In both cases, the complaints rest upon disagreements about the editorial judgment PolitiFact made in rendering Truth-O-Meter ratings. In one case, the issue related to what constitutes a cut to a public policy program and in the other case the issue related to whether it is appropriate to infer credit claiming by a political elite when a precise claim of credit was not made but obviously inferred. In the first case, the complaint stands on firmer ground than the complaint in the second case. Regardless, my own content analysis of about 50 claims concluded that there is not systematic professional or ideological bias in terms of the facts checked or ratings rendered.

Even so, the person who lodged the complaint, a fact-checker himself, also claimed to have received no response from PolitiFact. I’m not sure of the volume of complaints the site receives, but the critiques leveled by the reader merited attention.

done_all 2b marked as Fully compliant by Michael Wagner.

Section 3: Transparency of Sources

Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

PolitiFact has a policy to conduct on-the-record interviews for their fact checks. PolitiFact does not provide transcripts of the interviews it conducts, which would be an additional layer of transparency to provide for audience members, though that would have to be balanced with the effect it might have on sources’ willingness to be interviewed for PolitiFact fact-checks. Additionally, claims of fact are well-sourced, with hyperlinks and narratives describing the fact-checking process. When reading fact-checks, it is easy for audience members to learn where the evidence the reporter is using came from and how the reporter is applying that evidence to ultimately rate the claim on the Truth-O-Meter. The evidence used in each fact-check is generally easily available to readers who might be interested in replicating the fact check. The slipperiness of some definitions on the “Truth-O-Meter” - such as “ridiculous claim” (a necessary condition to earn the “Pants on Fire” rating) prevent absolute precision in replication, but this quibble falls in the category of editorial judgment, not transparency.

done_all 3a marked as Fully compliant by Michael Wagner.

Section 4: Transparency of Funding & Organization

Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

PolitiFact notes that they are owned by Poynter and that they also raise money from funders and funding organizations, claiming, “we receive support from revenue generated through content partnerships, online advertising and grants. In 2017, PolitiFact launched a membership campaign called the Truth Squad to allow donations from readers and fans of fact-checking.” There are not many details about this and no mention of ways that their claims of financial support can be verified. Given that PolitiFact is owned by the same organization, Poynter, that operates the IFCN, more care should be taken in identifying the precise nature of how the site is funded - and the fact that the IFCN assessment is conducted . For example, names of people who give more than $1,000 dollars are listed on the site. That’s good, but there is not additional information given about them, such as the information provided to the FEC by political candidates. Interestingly, one of the donors shares the same name as my father, though my father is not a donor to PolitiFact. The way the $1,000 donors are listed provides no way to precisely know who the donors are and thus, what their political orientations might be. 

done 4a marked as Partially compliant by Michael Wagner.

Criterion 4b
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.

20-Mar-2018 (8 months ago) Updated: 3 months ago
Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

PolitiFact provides clear and complete biographies of their reporters. These are easy to find. The staff has considerable gender diversity and apparent (though I make this judgment hesitantly since it is only by photograph and name) modest racial and ethnic diversity. I recommend that PolitiFact also provide a page with biographies of past reporters with an update about their current whereabouts as an additional layer of transparency for audience members.

done_all 4b marked as Fully compliant by Michael Wagner.

Criterion 4c
Evidence required: Please link to the section where readers can get in touch with the organization.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

It is easy to contact PolitiFact with a comment, suggestion or complaint. Anyone with a passing familiarity to the Internet should be able to find the contact page.

done_all 4c marked as Fully compliant by Michael Wagner.

Section 5: Transparency of Methodology

Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

PolitiFact is exceptionally clear about its methodology. First, the website notes, “Our core principles are independence, transparency, fairness, thorough reporting and clear writing.” Regarding the Truth-O-Meter ratings, the site clearly describes the different rating categories and the definitions that fit each. While there is not a precise definition for what constitutes “ridiculous claim” (as noted above), PolitiFact has a process by which the ratings are vetted by the reporter and three additional editors. The process they undergo is clearly articulated as well. The site also clearly describes what kinds of claims they choose to check and what kinds of claims they will not check. One piece of food-for-thought for the PolitiFact team is to consider a category that indicates a claim is not precisely checkable. Many claims made by newsworthy sources are claims that fit the definition of what PolitiFact wants to check save for the ‘precisely checkable’ requirement. Some fact-checking organizations will provide an explanatory story about these kinds of claims while noting that the nature of the claim is not precisely checkable. It is understandable why PolitiFact would not want to go down this road and perhaps this is something staff (and/or the site’s creators) discuss(ed), but it is worth thinking about. On the other hand, it is also worth considering the consequences of altering the Truth-O-Meter as any alteration done would make precise “apples to apples” over-time comparisons of the Truth-O-Meter impossible.

I should note that other meters (the Flip-O-Meter and the campaign promise tracker) are also clearly identified and articulated.

done_all 5a marked as Fully compliant by Michael Wagner.

Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

The evidence submitted meets the IFCN requirements. 

done_all 5b marked as Fully compliant by Michael Wagner.

Section 6: Open & Honest Corrections Policy

Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.

Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

The corrections policy is public and is clearly stated. The site does an admirable job of defining different categories of correction and the policies for dealing with each of them. Laudably, there is a Corrections and Updates RSS feed as well (feed://www.politifact.com/feeds/statementsbysubject/corrections-and-updates/).

This is pretty close to a gold standard for corrections.

One suggestion for consideration and one minor quibble. The suggestion: given the prominence and quality of PolitiFact, it might be worth considering including a page that publishes complaints and questions about the work PolitiFact does. Of course, many of these complaints will be of a nature that does not merit a response, but the transparency of noting the questions the organization is asked and the comments the organization receives would serve to counter some of the more conspiracy-minded complaints made about the important work PolitiFact does. Some of this is already done in the corrections and updates tag, though readers have no way of knowing whether this is comprehensive.

The quibble: the corrections policy notes that PolitiFact will update a story if there is a reader “outpouring.” The outpouring standard is but an example of when explanatory notes regarding corrections and updates will be used, but I would suggest that an outpouring should not be the only standard (and maybe it isn’t, an exhaustive list isn’t provided...and probably doesn’t need to be provided). For example, an outpouring of reader comments may notice an error, or one individual may notice an error. If there is something in need of additional explanation, an outpouring of response need not be required to issue it.

done_all 6a marked as Fully compliant by Michael Wagner.

Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.

20-Mar-2018 (8 months ago) Updated: 3 months ago
Michael Wagner Assessor
20-Mar-2018 (8 months ago) Updated: 3 months ago

The evidence submitted meets the IFCN requirements. 

done_all 6b marked as Fully compliant by Michael Wagner.