Check Your Fact

Organization: Check Your Fact
Applicant: Christian Datoc
Assessor: Michael Wagner
Edits made by the organization after this assessment

IFCN Staff wrote:

NOTE:
This signatory was approved on Feb 2 2018 but placed under review on Nov 30 2018 following a Board decision concerning the violation of criterion 4a. Check Your Fact relies on fact-checkers employed by The Daily Caller News Foundation as well as The Daily Caller, Inc. Only the latter was mentioned in the application. A new assessment will be required to evaluate compliance.



From the assessor: 


Check Your Fact has made several edits to their website in response to the IFCN review. I now deem it compliant or partially compliant with all criteria and recommend the Board verify the organization.

The site now lists a methodology for selecting and checking claims of fact, has staff biographies, discloses requirements for nonpartisanship for its reporters and lists a corrections policy.

Check Your Fact does not clearly list its funders. The biography page notes that some people not listed on the page might write fact-checks from time to time.

Overall, Check Your Fact generally appears to be in compliance with IFCN standards. While its current statement of ownership respects the requirements for “partial compliance” in the assessor guidelines under criterion 4b – and the rules that allowed other privately owned media fact-checkers to be approved under this code — Check Your Fact could increase trust in its platform by more clearly listing its funders and budget as well as adding biographical information for contributors who are not regular fact-checkers as the need arises. Additionally, it would be helpful if fact-checks specifically noted the “primary claim” of the fact(s) being checked in a story as the criteria for determining whether a claim is true or false rests on what Check Your Fact calls the “primary” claim and not a totality of truth. This way, the audience can be confident about what it is exactly that the fact-check finds is true, false or unsubstantiated.


Conclusion and recommendations
on 02-Feb-2018 (10 months ago)

Michael Wagner wrote:

CheckYourFact is a quality fact-checking organization that does not yet fully comply with the principles articulated by the IFCN. CheckYourFact partially complies with the principles articulated by IFCN. The most important areas of non/partial compliance are related to the description of the fact-checking process for the purposes of replication. In particular, the definitions of “true” and “false” and how the verdicts are reached are not precisely clear. On the one hand, CheckYourFact journalists generally clearly identify what claim is being checked, link to the sources they used to determine their verdicts and select newsworthy claims to check that do not appear to be biased to any particular political, economic or social persuasion. On the other hand, the lack of clarity about the site’s definitions for “true” and “false” make reliable replication impossible, even if the spirit of replications would likely mirror what CheckYourFact produces. Other areas of partial or noncompliance with the IFCN principles are noted here as well. The stand-alone site does not clearly and transparently list the site’s funders. The site does not provide biographical information for the journalists, there is a lack of a complete archive of fact checks on the stand alone site (checkyourfact.com) as compared to the linked site on the Daily Caller’s website (dailycaller.com/check-your-fact).

In my view, it would not be difficult for CheckYourFact to address these issues.

on 02-Feb-2018 (10 months ago)

Michael Wagner recommended Accept with edits


Section 1: Organization

Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.

Check Your Fact
02-Feb-2018 (10 months ago)
Michael Wagner Assessor
02-Feb-2018 (10 months ago)

CheckYourFact is a stand-alone website that exists for the sole purpose of fact-checking. It is owned by the news media organization, The Daily Caller (THE DC hereafter). The site has a clear Terms of Service agreement and a clear Privacy Policy. However, the site also has a url (http://dailycaller.com/check-your-fact/) that lives on THE DC’s website. I am not sure whether precisely conforms to the criteria specified in this section. There is a separate site, http://checkyourfact.com, that does appear to conform to the requirement for complete compliance. So, the “partial” ruling here should be read as “nearly complete.”


done_all 1a marked as Fully compliant by Michael Wagner.

Criterion 1b
Archive
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.

Check Your Fact
02-Feb-2018 (10 months ago)

You can view all of our fact checks on our website: http://checkyourfact.com/

They are also all housed on our mothership website, The Daily Caller: http://dailycaller.com/

Michael Wagner Assessor
02-Feb-2018 (10 months ago)

It is difficult to assess the question of weekly fact checks over a three month period as the fact checks are not dated in the body of the story. However, the URLs of the fact checks do appear to use a typical date-based naming convention. These dates suggest partial compliance with this criteria. On the http://checkyourfact.com site, the fact checks, using the “load more” option at the bottom of the screen, only provide fact checks over the past few weeks. On THE DC’s link to the page, one can search fact checks back several months. ON THE DC’s site, there is complete compliance with this criteria. On the stand alone site, there is not.


done_all 1b marked as Fully compliant by Michael Wagner.

Section 2: Nonpartisanship and Fairness

Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.

Check Your Fact
02-Feb-2018 (10 months ago)

  1. http://checkyourfact.com/2017/06/15/fact-check-did-tim-kaine-and-loretta-lynch-call-for-blood-and-death-in-the-streets/ 
  2. http://checkyourfact.com/2017/08/03/fact-check-do-you-have-to-speak-english-to-be-naturalized/ 
  3. http://checkyourfact.com/2017/09/14/fact-check-do-the-majority-of-americans-find-single-payer-health-care-a-horrible-idea/ 
  4. http://checkyourfact.com/2017/08/28/fact-check-has-hurricane-harvey-caused-once-in-500-year-flooding/ 
  5. http://checkyourfact.com/2017/08/03/fact-check-do-almost-a-fourth-of-americans-between-25-and-54-not-have-jobs/ 
  6. http://checkyourfact.com/2017/08/02/fact-check-has-china-done-nothing-about-north-korea/ 
  7. http://checkyourfact.com/2017/08/02/fact-check-did-the-stock-market-nearly-triple-under-obama/ 
  8. http://checkyourfact.com/2017/07/26/fact-check-are-15000-trans-people-serving-in-the-us-military/ 
  9. http://checkyourfact.com/2017/07/14/fact-check-massive-iceberg-raises-alarm-over-apocalyptic-climate-change/ 
  10. http://checkyourfact.com/2017/07/05/fact-check-is-russias-economy-just-a-little-bit-bigger-than-illinois-economy/ 

Check Your Fact focuses primarily on claims that are demonstrably true or false. If a claim cannot be proven false or true, the verdict “Unsubstantiated” will be applied.

Our rulings must be confirmed by multiple editors as well as the author of the fact check. The process often requires a back-and-forth exchange of evidence and research before a verdict is settled on.

After the fact check is written, two editors will again comb through the data, sources, and proof to check again whether the verdict is entirely accurate.

Check Your Fact works hard to include the necessary context for the reader to better understand the claim being made, along with our verdict and its supporting evidence.

If new information is provided after publication, it is reviewed by the editors to determine if it alters the verdict. If a verdict were to be changed, a note would be prominently affixed to alert the reader of the change and the reason. This has never happened.

Michael Wagner Assessor
02-Feb-2018 (10 months ago)

CheckYourFact chooses a variety of subjects and speakers to fact check. They list criteria that can largely be summed up with the notion that they check newsworthy claims made by public figures that are earning some kind of public attention - no matter who is making the claim or who a true or false verdict might benefit. Topics that are selected fit the bill of newsworthy topics regardless of who those topics might benefit or harm politically.

There is not a compliance category for “we follow the same process for every fact check and let the evidence dictate our conclusions” as described in the IFCN materials. If there was, I would rate the compliance as none as there is not a process that CheckYourFact provides with respect to how they go about conducting the fact check. They note that all fact check possibilities are pitched to a particular editor, but the process the fact check undergoes at that point is not specified. They do regularly link (in the body of the fact check itself) to evidence that would clearly help the audience understand how CheckYourFact reached their “verdict.” It appears that there are two possible “verdicts’ to a fact check – true and false. It is not clear how CheckYourFact determines “how much” truth is required for a claim to be true as compared to false. The decisions the site makes appear to be reasonable given their description of their investigations of the claims they check, but I cannot find or articulate a site-wide criteria.


done_all 2a marked as Fully compliant by Michael Wagner.

Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.

Check Your Fact
02-Feb-2018 (10 months ago)

As a news outlet, we mandate our employees do not advocate for individual politicians or candidates. Doing so would violate our journalistic code of ethics and make reporting the news in a non-biased way impossible.

Our writers agreement specifically addresses this point:

"Political Activities -- Writer agrees not to partake in any partisan political activity or other issue advocacy during the term of this Agreement. Writer understands that any such political activity or advocacy could cause others to question their independence as a reporter."

Files Attached
picture_as_pdf 2b - Writing _ Video... (383 KB)
Michael Wagner Assessor
02-Feb-2018 (10 months ago)

CheckYourFact claims that employees may not advocate for individuals or politicians. They do not appear to prohibit the taking of positions on any issues not strictly related to fact-checking.


done_all 2b marked as Fully compliant by Michael Wagner.

Section 3: Transparency of Sources

Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.

Check Your Fact
02-Feb-2018 (10 months ago)

Each fact check is heavily dependent on reputable, primary sources. When possible, the source is linked, along with a clear explanation of where the information comes from and what the information demonstrates.

If we are not able to link the sources, Check Your Fact tells readers where the information came from and the qualifications of the institution or individual who provided the content.

Check Your Fact strives to clearly show our work, regardless of the particular claim we are checking. Whenever evidence is provided to our readers we tell them where the information came from, what it says, and why it supports the rating.

It is essential that our sources are clearly shown in each article, because if there is a problem with our sources, we must be the first to know that. This way, our audience can easily reproduce our work and validate or refute it on their own.

Another important step in showing our work and sources is by laying out how the information was discovered by Check Your Fact. If we are using an interview that we did not personally acquire, we cite and link to the primary source.

The primary way in which we source the information, data, etc. is through hyperlinking the information in sentences where we cite the appropriate information. Check Your Fact cites the organization or individual that the information was produced by to allow our readers clarity on the source at hand.

Check Your Fact also uses graphs and charts to clearly demonstrate the applicable data. This data is not only sourced within the graph, but also in the text of the fact check. We continue to strive to make data, sources, and information easily accessible to readers, allowing them to fact check the fact checkers. Building trust and objectivity is a primary concern for Check Your Fact.

Michael Wagner Assessor
02-Feb-2018 (10 months ago) Updated: 4 months ago

*No text was included here. The spreadsheet that CheckYourFact sent along notes that evidence should be linked when possible. My assessment is that the CheckYourFact staff follow this dictate as a matter of course in their fact checks.

While it would not be difficulty to replicate the evidence trail used by CheckYourFact journalists given the quality of their source materials that they provide, it would be difficult to replicate the verdict, as there is no clear standard for true and false. Additionally, some, but not many, fact checks are imprecise with respect to the fact being checked. For example, in the check about the story “Fact Check: Bernie Sanders Video Downplays Health Care Wait Times in Canada,” it is not quite clear what is being checked or how the verdict was awarded. Sanders’ claim was that wait times are not a “major problem.” Then, the story quotes a patient saying that the wait times are not a problem.

It is not clear how the fact check determined what a wait time was (getting into the doctor, waiting in the emergency room, etc.). The fact check seems to treat these as two different kinds of wait times but they are not directly comparable and we don’t have a good sense for how the emergency time compares to other countries. The “major problem” was difficult to disentangle as well – was it related to adverse health outcomes due to extended wait times? Or something else?

The fact check reported upon and linked to excellent information about the length of waits and subsequent admission lengths. That is, the story is a well done piece of journalism, but it is not precisely clear what fact is being checked and how what the verdict means.

Additionally, the fact check “Did the Stock Market ‘Nearly Triple’ Under Obama?” is called true despite the fact that the fact check notes that the Dow Jones Industrial Average did not nearly triple under Obama. A variety of very clearly written and transparent facts about other stock market indexes noted that the S&P 500 did triple. The lack of clarity about what makes a claim “true” or “false” would make it difficult to replicate the fact check – again, not because the reporting isn’t there, but because the standard of judgment is unclear.


done_all 3a marked as Fully compliant by Michael Wagner.

Section 4: Transparency of Funding & Organization

Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.

Check Your Fact
02-Feb-2018 (10 months ago)

http://checkyourfact.com/aboutus/

"Check Your Fact is a for-profit subsidiary wholly owned by The Daily Caller, Inc. Majority owners of The Daily Caller, Inc. are co-founder and publisher Neil Patel and co-founder Tucker Carlson.

We generate revenue for our mission through advertising. Decisions about what to investigate, as well as our conclusions, are guided only by our journalists."

Michael Wagner Assessor
02-Feb-2018 (10 months ago)

I am not able to find the information regarding the first box below “compliance” on the CheckYourFact website. I assume that they are funded by THE DC as it is a site “produced by the journalists of The Daily Caller.”


done 4a marked as Partially compliant by Michael Wagner.

Criterion 4b
Staff
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.

Check Your Fact
02-Feb-2018 (10 months ago)

Check Your Fact is managed by The Daily Caller's Editor-in-Chief Geoff Ingersoll.

Ingersoll graduated from Penn State in 2004. After enlisting in the Marine Corps. in 2005, he deployed twice to Iraq, once to the DMZ in South Korea, and won several journalism awards. Geoff attended graduate journalism school at NYU, then embedded with Marines in Afghanistan. Before joining The Daily Caller in 2017, he served as the defense editor for Military, Defense, Foreign Policy at Business Insider, the managing editor of Marine Corps Times, and the managing editor of The Daily Caller News Foundation. You can find his work highlighted in the New York Times, Wall Street Journal, MSNBC, Salon, New York Daily News, CNN and many others. He is hopelessly addicted to the news.

Check Your Fact's two, dedicated fact-checkers are Kush S. Desai and David Sivak.

Desai was born and raised in northern New Jersey and is an alumnus of Dartmouth College.

Sivak grew up in the Washington, D.C. area and graduated summa cum laude from the University of Maryland, Baltimore County with a B.S. in financial economics and minors in writing and political science. Before joining Check Your Fact in 2017, he spent several years working for a Fortune 500 company in corporate finance.

Fact checks are also performed by The Daily Caller staff and other freelance journalists who are thoroughly vetted by The Daily Caller's editorial team.

Michael Wagner Assessor
02-Feb-2018 (10 months ago)

With respect to biographies, CheckYourFact links to all authors and key actors, but they do not have a biography. The ‘bio’ page does include a photo of the journalist and a link to the stories produced by the journalist.


done 4b marked as Partially compliant by Michael Wagner.

Criterion 4c
Contact
Evidence required: Please link to the section where readers can get in touch with the organization.

Check Your Fact
02-Feb-2018 (10 months ago)
Michael Wagner Assessor
02-Feb-2018 (10 months ago)

Each story also includes an invitation (and link(s)) to follow the journalist on social media.


done_all 4c marked as Fully compliant by Michael Wagner.

Section 5: Transparency of Methodology

Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.

Check Your Fact
02-Feb-2018 (10 months ago)

The Fact-Checking team focuses on claims made by politicians, pundits, authors, newspapers, TV hosts, and other 'public figures' that meet the following criteria:

  1. Claims that are contentious/incite wide disagreement 
  2. Claims that are unclear 
  3. Claims that are surprising or not widely known 
  4. Claims that are timely or newsworthy and meet one of the criteria above

Each pitch is subject to the approval of The Daily Caller's Editor-in-Chief Geoff Ingersoll. To find these claims, the Fact-Checking team will review political debates, town halls, speeches, interviews, and so forth. Our approach to fact-checking newspapers and magazines is summed up by the motto, "Read often and with a red pen."

http://checkyourfact.com/aboutus/

Michael Wagner Assessor
02-Feb-2018 (10 months ago)

Regarding the fact-checking methodology, there is not a process that CheckYourFact provides with respect to how they go about conducting the fact check. They note that all fact check possibilities are pitched to a particular editor, but the process the fact check undergoes at that point is not specified. They link (in the body of the fact check itself) to evidence that helps the audience understand how CheckYourFact reached the conclusion they reached. It appears that there are two possible “verdicts’ to a fact check – true and false, but I could not find a description of how these terms are defined. It is not clear how CheckYourFact determines “how much” truth is required for a claim to be true as compared to false. The decisions the site makes appear to be reasonable given their description of their investigations of the claims they check, but I cannot find or articulate a site-wide criteria. CheckYourFact lists clear criteria for what they check.


done_all 5a marked as Fully compliant by Michael Wagner.

Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.

Check Your Fact
02-Feb-2018 (10 months ago)
Michael Wagner Assessor
02-Feb-2018 (10 months ago)

Regarding indicating how readers can send in claims while “making it clear” what readers can expect is fact-checkable, CheckYourFact does indicate where claims can be sent. There is not a description of the kind of claims that are fact checkable (i.e. “The homicide rate in Chicago is 8 times higher than last year”) and those are not (i.e. “Chicago is terrible.”)


done_all 5b marked as Fully compliant by Michael Wagner.

Section 6: Open & Honest Corrections Policy

Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.

Check Your Fact
02-Feb-2018 (10 months ago)

http://checkyourfact.com/aboutus/

While our job is to ascertain the truth, we will occasionally make mistakes. When we do, we commit to hastily correcting our errors so that our readers can get the real story. If you spot an error, please alert us via our tip line. The prominence of our correction will be commensurate to the gravity of the error. Typos will be corrected without fanfare, minor corrections will be noted at the bottom of the article and major factual corrections will be noted at the top.

Michael Wagner Assessor
02-Feb-2018 (10 months ago)

CheckYourFact described to IFCN that they would make corrections in a way that was commensurathe size of the error but do not note that in a way that I was able to locate on the CheckYourFact site.


done_all 6a marked as Fully compliant by Michael Wagner.

Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.

Check Your Fact
02-Feb-2018 (10 months ago)

Check Your Fact has only been operating for a few months. As a result, we have not yet issued a correction, nor have we been notified of a piece that might need to be checked again. All of our fact checks can be viewed in one location here: https://docs.google.com/spreadsheets/d/1RMncnXj7lgjNutRvg5gZmxLW9YvNAIRi2t7obJecuUg/edit#gid=1700729578

Michael Wagner Assessor
02-Feb-2018 (10 months ago)

CheckYourFact described to IFCN that they would make corrections in a way that was commensurathe size of the error but do not note that in a way that I was able to locate on the CheckYourFact site.


done 6b marked as Partially compliant by Michael Wagner.