Politifact

Organization: PolitiFact
Applicant: Angie Holan
Assessor: Michael Wagner
Conclusion and recommendations
on 17-Aug-2018 (1 year ago)

Michael Wagner wrote:

Politifact is a world-class fact-checking organization that complies with IFCN principles. I recommend renewal. Given that IFCN and PolitiFact share an owner (Poynter), more care should be taken by PolitiFact to be transparent about its funding. IFCN should also note information about ownership when listing PolitiFact as an IFCN approved source.

In addition to a bit more transparency on funding, my assessment offered some ‘food for thought’ for PolitiFact in terms of its ratings system, processes for dealing with complaints and with respect to sharing the transcripts of on-the-record interviews for fact-checks. It might also better serve PolitiFact’s readers, and help PolitiFact’s own overall transparency, to develop a protocol for dealing with complaints about ratings. Some complaints will surely be ideological screeds. However, others will debate the rating on the merits. This second category of complaints deserve some kind of response, whether individually, publicly, or via some other mechanism.

None of these suggestions should be interpreted as requirements for renewal. PolitiFact is, in my view, adhering to IFCN’s principles already. These suggestions are to encourage an industry leader to continue innovating with respect to their methods, transparency and accountability to their audience.

on 17-Aug-2018 (1 year ago)

Michael Wagner recommended Accept


Section 1: Organization

Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

Regarding Requirement A, PolitiFact is an organization set up for the express purpose of fact-checking. On PolitiFact’s site, they note a change in ownership by writing, “PolitiFact had been owned by the Tampa Bay Times, but in 2018 direct ownership of PolitiFact was transferred from the Times to Poynter, which is the newspaper’s parent company. The move allows PolitiFact to function fully as not-for-profit national news organization.” With respect to PolitiFact’s organizational status, the site notes, “Control of both the the Poynter Institute and the Tampa Bay Times lie with a single executive. Upon retirement, that leader picks a successor. Poynter himself picked Eugene Patterson, who picked Andrew Barnes, who picked the Times' current chairman and CEO, Paul Tash, a lifelong journalist.” The site also offers information about donors who have given more than $1,000 to PolitiFact.


done_all 1a marked as Fully compliant by Michael Wagner.

Criterion 1b
Archive
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.

PolitiFact
20-Mar-2018 (2 years ago) Updated: 1 year ago
Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

Regarding Requirement B, PolitiFact produces fact-checks regularly, usually publishing several fact-checks per week day. With respect to the code of ethics of IFCN, their site notes, “Each day, PolitiFact journalists look for statements to fact-check. We read transcripts, speeches, news stories, press releases, and campaign brochures. We watch TV and scan social media. Readers send us suggestions via email to truthometer@politifact.com. Because we can't feasibly check all claims, we select the most newsworthy and significant ones. About one-third of the facts we choose to check come from reader ideas.

In deciding which statements to check, we consider these questions:

• Is the statement rooted in a fact that is verifiable? We don’t check opinions, and we recognize that in the world of speechmaking and political rhetoric, there is license for hyperbole.

• Does the statement seem misleading or sound wrong?

• Is the statement significant? We avoid minor "gotchas" on claims that are obviously a slip of the tongue.

• Is the statement likely to be passed on and repeated by others?

• Would a typical person hear or read the statement and wonder: Is that true?”

It is my assessment that PolitiFact produces fact-checks that appear to genuinely attempt to follow these principles. While no organization is perfect and while here and there PolitiFact makes editorial judgments with respect to ratings that people of good faith might disagree with, PolitiFact adheres to IFCN’s organization principles.


done_all 1b marked as Fully compliant by Michael Wagner.

Section 2: Nonpartisanship and Fairness

Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

PolitiFact’s fact-checks cover a wide variety of topics, from claims made by President Trump to claims made by prominent bloggers, email chains, politicians and news organizations. A rough content analysis I conducted of a random sample of 50 fact checks suggests no systematic bias toward overcovering any one political party or ideological orientation. When there is an imbalance of who is checked, it appears to be on the side of those in the country’s or a state’s majority party in government. This is an appropriate editorial choice. 


done_all 2a marked as Fully compliant by Michael Wagner.

Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

Regarding the criterion about supporting candidates or issues, PolitiFact allows their staff members to vote. The site does not specifically describe whether staff members can vote in partisan primaries, where voters must declare a party allegiance before casting a vote. It is possible that those interested in the organization could look up the primary voting records of staff members and seek to draw conclusions about the staff members’ politics but I do not think that this fact should alter PolitiFact’s policy as it upends the role of citizen in a democratic republic. PolitiFact prohibits campaign giving, working and volunteering, has a clear social media policy and certainly takes great care to communicate an environment of fairness to its audience.

Note: Two complaints to IFCN about PolitiFact - from the same source - were brought to my attention. In both cases, the complaints rest upon disagreements about the editorial judgment PolitiFact made in rendering Truth-O-Meter ratings. In one case, the issue related to what constitutes a cut to a public policy program and in the other case the issue related to whether it is appropriate to infer credit claiming by a political elite when a precise claim of credit was not made but obviously inferred. In the first case, the complaint stands on firmer ground than the complaint in the second case. Regardless, my own content analysis of about 50 claims concluded that there is not systematic professional or ideological bias in terms of the facts checked or ratings rendered.

Even so, the person who lodged the complaint, a fact-checker himself, also claimed to have received no response from PolitiFact. I’m not sure of the volume of complaints the site receives, but the critiques leveled by the reader merited attention.


done_all 2b marked as Fully compliant by Michael Wagner.

Section 3: Transparency of Sources

Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

PolitiFact has a policy to conduct on-the-record interviews for their fact checks. PolitiFact does not provide transcripts of the interviews it conducts, which would be an additional layer of transparency to provide for audience members, though that would have to be balanced with the effect it might have on sources’ willingness to be interviewed for PolitiFact fact-checks. Additionally, claims of fact are well-sourced, with hyperlinks and narratives describing the fact-checking process. When reading fact-checks, it is easy for audience members to learn where the evidence the reporter is using came from and how the reporter is applying that evidence to ultimately rate the claim on the Truth-O-Meter. The evidence used in each fact-check is generally easily available to readers who might be interested in replicating the fact check. The slipperiness of some definitions on the “Truth-O-Meter” - such as “ridiculous claim” (a necessary condition to earn the “Pants on Fire” rating) prevent absolute precision in replication, but this quibble falls in the category of editorial judgment, not transparency.


done_all 3a marked as Fully compliant by Michael Wagner.

Section 4: Transparency of Funding & Organization

Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

PolitiFact notes that they are owned by Poynter and that they also raise money from funders and funding organizations, claiming, “we receive support from revenue generated through content partnerships, online advertising and grants. In 2017, PolitiFact launched a membership campaign called the Truth Squad to allow donations from readers and fans of fact-checking.” There are not many details about this and no mention of ways that their claims of financial support can be verified. Given that PolitiFact is owned by the same organization, Poynter, that operates the IFCN, more care should be taken in identifying the precise nature of how the site is funded - and the fact that the IFCN assessment is conducted . For example, names of people who give more than $1,000 dollars are listed on the site. That’s good, but there is not additional information given about them, such as the information provided to the FEC by political candidates. Interestingly, one of the donors shares the same name as my father, though my father is not a donor to PolitiFact. The way the $1,000 donors are listed provides no way to precisely know who the donors are and thus, what their political orientations might be. 


done 4a marked as Partially compliant by Michael Wagner.

Criterion 4b
Staff
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.

PolitiFact
20-Mar-2018 (2 years ago) Updated: 1 year ago
Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

PolitiFact provides clear and complete biographies of their reporters. These are easy to find. The staff has considerable gender diversity and apparent (though I make this judgment hesitantly since it is only by photograph and name) modest racial and ethnic diversity. I recommend that PolitiFact also provide a page with biographies of past reporters with an update about their current whereabouts as an additional layer of transparency for audience members.



done_all 4b marked as Fully compliant by Michael Wagner.

Criterion 4c
Contact
Evidence required: Please link to the section where readers can get in touch with the organization.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

It is easy to contact PolitiFact with a comment, suggestion or complaint. Anyone with a passing familiarity to the Internet should be able to find the contact page.


done_all 4c marked as Fully compliant by Michael Wagner.

Section 5: Transparency of Methodology

Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

PolitiFact is exceptionally clear about its methodology. First, the website notes, “Our core principles are independence, transparency, fairness, thorough reporting and clear writing.” Regarding the Truth-O-Meter ratings, the site clearly describes the different rating categories and the definitions that fit each. While there is not a precise definition for what constitutes “ridiculous claim” (as noted above), PolitiFact has a process by which the ratings are vetted by the reporter and three additional editors. The process they undergo is clearly articulated as well. The site also clearly describes what kinds of claims they choose to check and what kinds of claims they will not check. One piece of food-for-thought for the PolitiFact team is to consider a category that indicates a claim is not precisely checkable. Many claims made by newsworthy sources are claims that fit the definition of what PolitiFact wants to check save for the ‘precisely checkable’ requirement. Some fact-checking organizations will provide an explanatory story about these kinds of claims while noting that the nature of the claim is not precisely checkable. It is understandable why PolitiFact would not want to go down this road and perhaps this is something staff (and/or the site’s creators) discuss(ed), but it is worth thinking about. On the other hand, it is also worth considering the consequences of altering the Truth-O-Meter as any alteration done would make precise “apples to apples” over-time comparisons of the Truth-O-Meter impossible.

I should note that other meters (the Flip-O-Meter and the campaign promise tracker) are also clearly identified and articulated.


done_all 5a marked as Fully compliant by Michael Wagner.

Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

The evidence submitted meets the IFCN requirements. 


done_all 5b marked as Fully compliant by Michael Wagner.

Section 6: Open & Honest Corrections Policy

Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.

Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

The corrections policy is public and is clearly stated. The site does an admirable job of defining different categories of correction and the policies for dealing with each of them. Laudably, there is a Corrections and Updates RSS feed as well (feed://www.politifact.com/feeds/statementsbysubject/corrections-and-updates/).

This is pretty close to a gold standard for corrections.

One suggestion for consideration and one minor quibble. The suggestion: given the prominence and quality of PolitiFact, it might be worth considering including a page that publishes complaints and questions about the work PolitiFact does. Of course, many of these complaints will be of a nature that does not merit a response, but the transparency of noting the questions the organization is asked and the comments the organization receives would serve to counter some of the more conspiracy-minded complaints made about the important work PolitiFact does. Some of this is already done in the corrections and updates tag, though readers have no way of knowing whether this is comprehensive.

The quibble: the corrections policy notes that PolitiFact will update a story if there is a reader “outpouring.” The outpouring standard is but an example of when explanatory notes regarding corrections and updates will be used, but I would suggest that an outpouring should not be the only standard (and maybe it isn’t, an exhaustive list isn’t provided...and probably doesn’t need to be provided). For example, an outpouring of reader comments may notice an error, or one individual may notice an error. If there is something in need of additional explanation, an outpouring of response need not be required to issue it.


done_all 6a marked as Fully compliant by Michael Wagner.

Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.

PolitiFact
20-Mar-2018 (2 years ago) Updated: 1 year ago
Michael Wagner Assessor
20-Mar-2018 (2 years ago) Updated: 1 year ago

The evidence submitted meets the IFCN requirements. 


done_all 6b marked as Fully compliant by Michael Wagner.

Section 7: Eligibility to be a signatory

Criterion 1.1
The applicant is a legally registered organization, or a distinct team or unit within a legally registered organization, and details of this are easily found on its website.

Criterion 1.2
The team, unit or organization is set up exclusively for the purpose of fact-checking.

Criterion 1.3
The applicant has published an average of at least one fact check a week over the course of the three months prior to the date of application.

Criterion 1.4
On average, at least 75% of the applicant’s fact checks focus on claims related to issues that, in the view of the IFCN, relate to or could have an impact on the welfare or well-being of individuals, the general public or society.

Criterion 1.5
The applicant’s editorial output is not, in the view of the IFCN, controlled by the state, a political party or politician.

Criterion 1.6
If the organization receives funding from local or foreign state or political sources, it provides a statement on its site setting out to the satisfaction of the IFCN, how it ensures its funders do not influence the findings of its reports.

Section 8: A commitment to Non-partisanship and Fairness

Criterion 2.1
The applicant fact-checks using the same high standards of evidence and judgement for equivalent claims regardless of who made the claim.

Criterion 2.2
The applicant does not unduly concentrate its fact-checking on any one side, considers the reach and importance of claims it selects to check and publishes a short statement on its website to set out how it selects claims to check.

Criterion 2.3
The applicant discloses in its fact checks relevant interests of the sources it quotes where the reader might reasonably conclude those interests could influence the accuracy of the evidence provided. It also discloses in its fact checks any commercial or other such relationships it has that a member of the public might reasonably conclude could influence the findings of the fact check.

Criterion 2.4
The applicant is not as an organization affiliated with nor declares or shows support for any party, any politician or political candidate, nor does it advocate for or against any policy positions on any issues save for transparency and accuracy in public debate.

Criterion 2.5
The applicant sets out its policy on non-partisanship for staff on its site. Save for the issues of accuracy and transparency, the applicant’s staff do not get involved in advocacy or publicise their views on policy issues the organization might fact check in such a way as might lead a reasonable member of the public to see the organization’s work as biased.

Section 9: A commitment to Standards and Transparency of Sources

Criterion 3.1
The applicant identifies the source of all significant evidence used in their fact checks, providing relevant links where the source is available online, in such a way that users can replicate their work if they wish. In cases where identifying the source would compromise the source’s personal security, the applicant provides as much detail as compatible with the source’s safety.

Criterion 3.2
The applicant uses the best available primary, not secondary, sources of evidence wherever suitable primary sources are available. Where suitable primary sources are not available, the applicant explains the use of a secondary source.

Criterion 3.3
The applicant checks all key elements of claims against more than one named source of evidence save where the one source is the only source relevant on the topic.

Criterion 3.4
The applicant identifies in its fact checks the relevant interests of the sources it uses where the reader might reasonably conclude those interests could influence the accuracy of the evidence provided.

Section 10: A commitment to Transparency of Funding & Organization

Criterion 4.1
Applicants that are independent organizations have a page on their website detailing each source of funding accounting for 5% or more of total revenue for its previous financial year. This page also sets out the legal form in which the organization is registered (e.g. as a non-profit, as a company etc).

Criterion 4.2
Applicants that are the fact-checking section or unit of a media house or other parent organization make a statement on ownership.

Criterion 4.3
A statement on the applicant’s website sets out the applicant’s organizational structure and makes clear how and by whom editorial control is exercised.

Criterion 4.4
A page on the applicant’s website details the professional biography of all those who, according to the organizational structure and play a significant part in its editorial output.

Criterion 4.5
The applicant provides easy means on its website and/or via social media for users to communicate with the editorial team.

Section 11: A commitment to Standards and Transparency of Methodology

Criterion 5.1
The applicant publishes on its website a statement about the methodology it uses to select, research, write and publish its fact checks.

Criterion 5.2
The applicant selects claims to check based primarily on the reach and importance of the claims, and where possible explains the reason for choosing the claim to check.

Criterion 5.3
The applicant sets out in its fact checks relevant evidence that appears to support the claim as well as relevant evidence that appears to undermine it.

Criterion 5.4
The applicant in its fact checks assesses the merits of the evidence found using the same high standards applied to evidence on equivalent claims, regardless of who made the claim.

Criterion 5.5
The applicant seeks where possible to contact those who made the claim to seek supporting evidence, noting that (i) this is often not possible with online claims, (ii) if the person who makes the claim fails to reply in a timely way this should not impede the fact check, (iii) if a speaker adds caveats to the claim, the fact-checker should be free to continue with checking the original claim, (iv) fact-checkers may not wish to contact the person who made the claim for safety or other legitimate reasons.

Criterion 5.6
The applicant encourages users to send in claims to check, while making it clear what readers can legitimately expect will be fact-checked and what isn’t fact-checkable.

Section 12: A commitment to an Open & Honest Corrections Policy

Criterion 6.1
The applicant has a corrections or complaints policy that is easily visible and accessible on the organization’s website or frequently referenced in broadcasts.

Criterion 6.2
The policy sets out clear definitions of what it does and does not cover, how major mistakes, especially those requiring revised conclusions of a fact check, are handled, and the fact that some complaints may justify no response. This policy is adhered to scrupulously.

Criterion 6.3
Where credible evidence is provided that the applicant has made a mistake worthy of correction, the applicant makes a correction openly and transparently, seeking as far as possible to ensure that users of the original see the correction and the corrected version.

Criterion 6.4
The applicant, if an existing signatory, should either on its corrections/complaints page or on the page where it declares itself an IFCN signatory inform users that if they believe the signatory is violating the IFCN Code, they may inform the IFCN, with a link to the IFCN site.

Criterion 6.5
If the applicant is the fact-checking unit of a media company, it is a requirement of signatory status that the parent media company has and adheres to an open and honest corrections policy.