Science Feedback

Organization: Science Feedback
Applicant: Dr. Emmanuel Vincent
Assessor: Sarphan Uzunoğlu
Edits made by the organization after this assessment

IFCN Staff wrote:

I have added the publication dates, see:

https://healthfeedback.org/claim-reviews/

https://climatefeedback.org/claim-reviews/


That makes it easier to see that we have published the following number of claim reviews:


4 (Climate) + 5 (Health) = 9 in July

2 (Climate) + 4 (Health) = 6 in June

3 (Climate) + 6 (Health) = 9 in May

2 (Climate) + 4 (Health) = 6 in April


This is more than 1 per week on average and that is without counting the article reviews (which we decreased the output recently)


Conclusion and recommendations
on 14-Jun-2019 (9 months ago)

Sarphan Uzunoğlu wrote:

They need to publish more often to meet IFCN's standards. Otherwise, methodologically and in terms of non-partisanship and transparency, they are a reliable organization who is recognized by many international news outlets and scientific actors. I suggest to accept them but give them feedbacks regarding regularity of their content publication regime. 

on 14-Jun-2019 (9 months ago)

Sarphan Uzunoğlu recommended Accept with edits


Section 1: Organization

Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

Science Feedback is a non-profit organization registered in France whose primary activity is the verification of influential science-related claims and articles online, relying on a global network of scientists to produce analyses and fact-checks. Its mission is summarized in its status as (translated from French):

"Article 2: Object

The association’s object is to evaluate the scientific credibility of information published and broadcasted in the media and on social media. Through the development of digital services dedicated to improve the accuracy of scientific information in the media and its accessibility to the public, the association aims to tackle the issue of online misinformation. The association acts notably via the organization of the scientific community to verify influential information (fact-checking) and to write pedagogical content about the science."

The status published in the “Journal Officiel” can be found here:

http://www.journal-officiel.gouv.fr/publications/assoc/pdf/2018/0036/JOAFE_PDF_Unitaire_20180036_01090.pdf

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

They provided legal document that provides information about the relevancy of the organization in terms of fact-checking. 


done_all 1a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 1b
Archive
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.

Science Feedback
09-Jun-2019 (9 months ago)

Science Feedback provides scientific fact-checks both in the form of article reviews and claim reviews.

Article reviews invite scientists to fact-check a number of claims at once and provide in-depth analysis on a whole article. Beyond checking individual facts, it is often necessary in science to analyze a whole article for quality of logic and scientific reasoning, i.e., how evidence is used to support a conclusion.

Claim reviews are focused on checking the veracity of individual claims, which are extracted from prominent articles, statements by politicians, or influencers on social media.

• Climate article reviews archive: https://climatefeedback.org/feedbacks/

• Climate claim reviews archive: https://climatefeedback.org/claim-reviews/

• Health article reviews archive: https://healthfeedback.org/feedbacks/

• Health claim reviews archive: https://healthfeedback.org/claim-reviews/

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

They do not publish fact-checks as often as other fact-checking actors I've assessed so far. Also their decentralized web structure makes it harder to get information about regularity of their analyses.


done 1b marked as Partially compliant by Sarphan Uzunoğlu.

Section 2: Nonpartisanship and Fairness

Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

In order to maintain coherence in the credibility ratings issued by scientists, we provide a clear guide defining the meaning of each “overall credibility rating” level and ask reviewers to evaluate articles based on 6 criteria (Factual Accuracy, Scientific understanding, Logic/Reasoning, Precision/Clarity, Sources Quality, Fairness/Objectivity). Read our guidelines here: https://sciencefeedback.co/process/#tit4 .

Scientists contributing to our analyses are asked to conform to high quality community standards upon sign up, which require them to comment only on claims related to their expertise. (Read our community standards: https://sciencefeedback.co/community-standards/ )

Science Feedback editors select articles and claims for review from a variety of media—across the political spectrum—according to their influence on social media (typically measured by Buzzsumo.com), the quantity or degree of claimed scientific evidence within the reporting, and potential relevance to shaping public debate.

As stated on our Process page, we strive for our reviews to be representative of the spectrum of influential science related discussion in the media. We review articles and claims in a variety of media, without a priori perspective, regardless of whether they insightfully report on or exaggerate scientific findings.

1. Senator Sanders’ claim that climate change is making tornadoes worse isn’t supported by published research

https://climatefeedback.org/claimreview/senator-sanders-claim-that-climate-change-is-making-tornadoes-worse-isnt-supported-by-published-research/

2. President Trump’s claim that water supply policy has worsened California wildfires is baseless

https://climatefeedback.org/claimreview/president-trumps-claim-that-water-supply-policy-has-worsened-california-wildfires-is-baseless/

3. Guardian story on climate impacts of diet gets mixed reviews from scientists

https://climatefeedback.org/evaluation/guardian-story-on-climate-impacts-of-diet-gets-mixed-reviews-from-scientists-damian-carrington/

4. Claim that human civilization could end in 30 years is speculative, not supported with evidence

https://climatefeedback.org/evaluation/iflscience-story-on-speculative-report-provides-little-scientific-context-james-felton/

5. It’s true that the current carbon dioxide level is higher than any time in human existence

https://climatefeedback.org/claimreview/its-true-that-the-current-carbon-dioxide-level-is-higher-than-any-time-in-human-existence/

6. Breitbart article baselessly claims a study of past climate invalidates human-caused climate change

https://climatefeedback.org/evaluation/breitbart-article-baselessly-claims-a-study-of-past-climate-invalidates-human-caused-climate-change-john-nolte/

7. Article wrongly claims that measles vaccine claims more lives than measles infection, misinterprets epidemiological data

https://healthfeedback.org/claimreview/article-wrongly-claims-that-measles-vaccine-claims-more-lives-than-measles-infection-misinterprets-epidemiological-data/

8. Measles outbreak in US due to unvaccinated citizens, not illegal immigrants

https://healthfeedback.org/claimreview/measles-outbreak-due-to-unvaccinated-citizens-not-illegal-immigrants-alex-jones-infowars-natural-news/

9. New Scientist article accurately summarises polycystic ovary syndrome (PCOS) research but overstates significance of animal studies

https://healthfeedback.org/evaluation/new-scientist-article-accurately-summarises-pcos-research-but-overstates-significance-of-animal-studies-alice-klein/

10. Article claiming vaccines cause autoimmunity and autism due to fetal DNA contaminants found unsupported and implausible

https://healthfeedback.org/evaluation/article-claiming-vaccines-cause-autoimmunity-and-autism-due-to-fetal-dna-contaminants-found-unsupported-and-implausible-theresa-deisher/ 

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

Scientists contributing to their analyses comment only on claims related to their expertise. It is seen on the website looking at the profiles and former studies of experts. Their understanding of measuring influence is interesting and focus on particularly popular subjects is understandable. All the cases they provided seem very relevant.


done_all 2a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

We ensure that our staff are not involved in political parties or advocacy organizations, as mentioned on our Community Standards’ page: https://sciencefeedback.co/community-standards/

“Science Feedback is dedicated to science education and does not advocate for any particular policy, nor does it support any political candidate or party. Science Feedback ensures that our staff are not directly involved in political parties or advocacy organizations that could bias their neutrality and undermine their commitment to scientific accuracy.”

Here is the note we include in our job posting for editorial staffs and contributors: “Non-partisanship: Our staff and freelance contributors are required to not be involved in advocacy/political campaigning.” see eg this job post: https://docs.google.com/document/d/1Kaev6Cnjh4mJPUyLyAkHdhnM_0fXS_5uyz9hu-0K8_E/

Our scientific feedbacks do not constitute endorsements of the authors’ political or economic ideology, rather they are assessments of the scientific foundations and reasoning of the argumentation contained within each article.

Similarly we ask scientists reviewing with Science Feedback not to “evaluate the opinion of the author, but instead the scientific accuracy of facts contained within the text, and the scientific quality of reasoning used.” (As noted on our methods page: https://sciencefeedback.co/process/ )

Note that we do not primarily assess statements made by politicians, but mostly by journalists, pundits, and social media influencers.

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago
Especially thanks to the theme they focus on, they have no problems about political partisanship. They are very non-partisan and both their community standards and job post they exemplified are great proofs of how much do they care about their independent and objective status. Their policy is including both freelancers and full timers and it is also a big advantage.

done_all 2b marked as Fully compliant by Sarphan Uzunoğlu.

Section 3: Transparency of Sources

Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

Our fact-checking process and policy on sources is described in our Process page. At Science Feedback, we ask our sources (scientists) to comment on articles directly and to indicate whether the facts underlying the reasoning are consistent with up-to-date scientific knowledge. That way, readers can hear directly from sources who have actual knowledge and expertise on the subject at hand. Each scientist is clearly identifiable with a link to a professional page listing his/her scientific publications.

In our generally proposed format for comments, we ask scientists to cite the most relevant supporting sources—references from the peer-reviewed literature whenever possible. We ask reviewers to represent the state of knowledge in the scientific literature, using strongly supported scientific theories and observations as references, and to refrain from pointing to isolated or weakly supported findings.

Our scientist reviewers are all listed on our Community page, as well as the objective criteria we use to accept contributing reviewers: https://sciencefeedback.co/for-scientists/#ref. To maintain transparency, we require contributors to identify themselves on our website using their real names and photographs. It is easy for anyone to contact the scientists and replicate the fact-check. Our article rating system is based on the average given by scientists, ensuring transparency and objectivity in the way we reach our final conclusion.

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

I can verify that they ask their experts to comment on articles directly and to indicate whether the facts underlying the reasoning are consistent with up-to-date scientific knowledge. In the instances they provided, it is possible to see many related citations. They are open to contributions of new experts but it would be good for the organization to clarify how to they deal with quality of the experts' work and quality of their work.


done 3a marked as Partially compliant by Sarphan Uzunoğlu.

Section 4: Transparency of Funding & Organization

Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago
Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

They are open about their donors and it is a nice characteristic for such an organization.


done_all 4a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 4b
Staff
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago
Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

They are transparent about identities of their experts and they are accessible through their website.


done_all 4b marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 4c
Contact
Evidence required: Please link to the section where readers can get in touch with the organization.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

Readers contact us via this online form: https://sciencefeedback.co/contact-us/

This form is linked from our Methodology Page (in the first section), from any page in the footer, as well as at the bottom of every review.

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago
Their contact page is visible and available.

done_all 4c marked as Fully compliant by Sarphan Uzunoğlu.

Section 5: Transparency of Methodology

Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

Methodology for article reviews: https://sciencefeedback.co/process/ ; Methodology for claim reviews: https://sciencefeedback.co/claim-reviews-framework/ 

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

They provided a compliant, detailed and totally acceptable methodology. It totall fits to the objectives and manifest of the organization and has detailed explanation regarding their practical routine in fact-checking processes. Of course having two different methodology for claims and articles may seem confusing at first hand but it is totally relevant and acceptable.


done_all 5a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

Readers submit suggestions of articles or claims to review via our contact form.

They are invited to do so at the bottom of every review with the following text:

“Please get in touch if you have any comment or think there is an important claim or article that would need to be reviewed.”

They are also invited to do so on the process page with the following text:

“If you wish to submit a suggestion of an article or claim to review, please use this online form. Please note that we focus on reviewing claims and articles that are scientifically verifiable and that reach large audiences.”

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago
Even if it is not interactive and inviting enough for the readers, they provide a way for people to send claims or suggestions.

done 5b marked as Partially compliant by Sarphan Uzunoğlu.

Section 6: Open & Honest Corrections Policy

Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

Our correction policy is described here: https://sciencefeedback.co/process/#correction

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

Their policy regarding collections is very clear and rational. However, there might be an indication for the articles under review after user feedback.


done 6a marked as Partially compliant by Sarphan Uzunoğlu.

Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.

Science Feedback
09-Jun-2019 (9 months ago) Updated: 9 months ago

We have handled no correction requests, and we have not made any major corrections during our tenure as of today.

In the cases where we made minor corrections to the article after its publication, we included an “Update” section below the summary to explain the nature of the update.

For example, we included this update: “The analysis has been updated to include several comments received just after the time of publication. The main conclusion of the analysis is unchanged.” in this analysis: https://climatefeedback.org/evaluation/scientists-explain-what-new-york-magazine-article-on-the-uninhabitable-earth-gets-wrong-david-wallace-wells/ 

Sarphan Uzunoğlu Assessor
13-Jun-2019 (9 months ago) Updated: 9 months ago

As they didn't have correction requests that often, they don't have a list like that. But they should introduce such a system to be more reliable.


done 6b marked as Partially compliant by Sarphan Uzunoğlu.

Section 7: Eligibility to be a signatory

Criterion 1.1
The applicant is a legally registered organization, or a distinct team or unit within a legally registered organization, and details of this are easily found on its website.

Criterion 1.2
The team, unit or organization is set up exclusively for the purpose of fact-checking.

Criterion 1.3
The applicant has published an average of at least one fact check a week over the course of the three months prior to the date of application.

Criterion 1.4
On average, at least 75% of the applicant’s fact checks focus on claims related to issues that, in the view of the IFCN, relate to or could have an impact on the welfare or well-being of individuals, the general public or society.

Criterion 1.5
The applicant’s editorial output is not, in the view of the IFCN, controlled by the state, a political party or politician.

Criterion 1.6
If the organization receives funding from local or foreign state or political sources, it provides a statement on its site setting out to the satisfaction of the IFCN, how it ensures its funders do not influence the findings of its reports.

Section 8: A commitment to Non-partisanship and Fairness

Criterion 2.1
The applicant fact-checks using the same high standards of evidence and judgement for equivalent claims regardless of who made the claim.

Criterion 2.2
The applicant does not unduly concentrate its fact-checking on any one side, considers the reach and importance of claims it selects to check and publishes a short statement on its website to set out how it selects claims to check.

Criterion 2.3
The applicant discloses in its fact checks relevant interests of the sources it quotes where the reader might reasonably conclude those interests could influence the accuracy of the evidence provided. It also discloses in its fact checks any commercial or other such relationships it has that a member of the public might reasonably conclude could influence the findings of the fact check.

Criterion 2.4
The applicant is not as an organization affiliated with nor declares or shows support for any party, any politician or political candidate, nor does it advocate for or against any policy positions on any issues save for transparency and accuracy in public debate.

Criterion 2.5
The applicant sets out its policy on non-partisanship for staff on its site. Save for the issues of accuracy and transparency, the applicant’s staff do not get involved in advocacy or publicise their views on policy issues the organization might fact check in such a way as might lead a reasonable member of the public to see the organization’s work as biased.

Section 9: A commitment to Standards and Transparency of Sources

Criterion 3.1
The applicant identifies the source of all significant evidence used in their fact checks, providing relevant links where the source is available online, in such a way that users can replicate their work if they wish. In cases where identifying the source would compromise the source’s personal security, the applicant provides as much detail as compatible with the source’s safety.

Criterion 3.2
The applicant uses the best available primary, not secondary, sources of evidence wherever suitable primary sources are available. Where suitable primary sources are not available, the applicant explains the use of a secondary source.

Criterion 3.3
The applicant checks all key elements of claims against more than one named source of evidence save where the one source is the only source relevant on the topic.

Criterion 3.4
The applicant identifies in its fact checks the relevant interests of the sources it uses where the reader might reasonably conclude those interests could influence the accuracy of the evidence provided.

Section 10: A commitment to Transparency of Funding & Organization

Criterion 4.1
Applicants that are independent organizations have a page on their website detailing each source of funding accounting for 5% or more of total revenue for its previous financial year. This page also sets out the legal form in which the organization is registered (e.g. as a non-profit, as a company etc).

Criterion 4.2
Applicants that are the fact-checking section or unit of a media house or other parent organization make a statement on ownership.

Criterion 4.3
A statement on the applicant’s website sets out the applicant’s organizational structure and makes clear how and by whom editorial control is exercised.

Criterion 4.4
A page on the applicant’s website details the professional biography of all those who, according to the organizational structure and play a significant part in its editorial output.

Criterion 4.5
The applicant provides easy means on its website and/or via social media for users to communicate with the editorial team.

Section 11: A commitment to Standards and Transparency of Methodology

Criterion 5.1
The applicant publishes on its website a statement about the methodology it uses to select, research, write and publish its fact checks.

Criterion 5.2
The applicant selects claims to check based primarily on the reach and importance of the claims, and where possible explains the reason for choosing the claim to check.

Criterion 5.3
The applicant sets out in its fact checks relevant evidence that appears to support the claim as well as relevant evidence that appears to undermine it.

Criterion 5.4
The applicant in its fact checks assesses the merits of the evidence found using the same high standards applied to evidence on equivalent claims, regardless of who made the claim.

Criterion 5.5
The applicant seeks where possible to contact those who made the claim to seek supporting evidence, noting that (i) this is often not possible with online claims, (ii) if the person who makes the claim fails to reply in a timely way this should not impede the fact check, (iii) if a speaker adds caveats to the claim, the fact-checker should be free to continue with checking the original claim, (iv) fact-checkers may not wish to contact the person who made the claim for safety or other legitimate reasons.

Criterion 5.6
The applicant encourages users to send in claims to check, while making it clear what readers can legitimately expect will be fact-checked and what isn’t fact-checkable.

Section 12: A commitment to an Open & Honest Corrections Policy

Criterion 6.1
The applicant has a corrections or complaints policy that is easily visible and accessible on the organization’s website or frequently referenced in broadcasts.

Criterion 6.2
The policy sets out clear definitions of what it does and does not cover, how major mistakes, especially those requiring revised conclusions of a fact check, are handled, and the fact that some complaints may justify no response. This policy is adhered to scrupulously.

Criterion 6.3
Where credible evidence is provided that the applicant has made a mistake worthy of correction, the applicant makes a correction openly and transparently, seeking as far as possible to ensure that users of the original see the correction and the corrected version.

Criterion 6.4
The applicant, if an existing signatory, should either on its corrections/complaints page or on the page where it declares itself an IFCN signatory inform users that if they believe the signatory is violating the IFCN Code, they may inform the IFCN, with a link to the IFCN site.

Criterion 6.5
If the applicant is the fact-checking unit of a media company, it is a requirement of signatory status that the parent media company has and adheres to an open and honest corrections policy.