Science Feedback

Organization: Science Feedback
Applicant: Dr. Emmanuel Vincent
Assessor: Sarphan Uzunoğlu
Edits made by the organization after this assessment

IFCN Staff wrote:

I have added the publication dates, see:

https://healthfeedback.org/claim-reviews/

https://climatefeedback.org/claim-reviews/


That makes it easier to see that we have published the following number of claim reviews:


4 (Climate) + 5 (Health) = 9 in July

2 (Climate) + 4 (Health) = 6 in June

3 (Climate) + 6 (Health) = 9 in May

2 (Climate) + 4 (Health) = 6 in April


This is more than 1 per week on average and that is without counting the article reviews (which we decreased the output recently)


Conclusion and recommendations
on 14-Jun-2019 (5 months ago)

Sarphan Uzunoğlu wrote:

They need to publish more often to meet IFCN's standards. Otherwise, methodologically and in terms of non-partisanship and transparency, they are a reliable organization who is recognized by many international news outlets and scientific actors. I suggest to accept them but give them feedbacks regarding regularity of their content publication regime. 

on 14-Jun-2019 (5 months ago)

Sarphan Uzunoğlu recommended Accept with edits


Section 1: Organization

Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

Science Feedback is a non-profit organization registered in France whose primary activity is the verification of influential science-related claims and articles online, relying on a global network of scientists to produce analyses and fact-checks. Its mission is summarized in its status as (translated from French):

"Article 2: Object

The association’s object is to evaluate the scientific credibility of information published and broadcasted in the media and on social media. Through the development of digital services dedicated to improve the accuracy of scientific information in the media and its accessibility to the public, the association aims to tackle the issue of online misinformation. The association acts notably via the organization of the scientific community to verify influential information (fact-checking) and to write pedagogical content about the science."

The status published in the “Journal Officiel” can be found here:

http://www.journal-officiel.gouv.fr/publications/assoc/pdf/2018/0036/JOAFE_PDF_Unitaire_20180036_01090.pdf

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

They provided legal document that provides information about the relevancy of the organization in terms of fact-checking. 


done_all 1a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 1b
Archive
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.

Science Feedback
09-Jun-2019 (5 months ago)

Science Feedback provides scientific fact-checks both in the form of article reviews and claim reviews.

Article reviews invite scientists to fact-check a number of claims at once and provide in-depth analysis on a whole article. Beyond checking individual facts, it is often necessary in science to analyze a whole article for quality of logic and scientific reasoning, i.e., how evidence is used to support a conclusion.

Claim reviews are focused on checking the veracity of individual claims, which are extracted from prominent articles, statements by politicians, or influencers on social media.

• Climate article reviews archive: https://climatefeedback.org/feedbacks/

• Climate claim reviews archive: https://climatefeedback.org/claim-reviews/

• Health article reviews archive: https://healthfeedback.org/feedbacks/

• Health claim reviews archive: https://healthfeedback.org/claim-reviews/

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

They do not publish fact-checks as often as other fact-checking actors I've assessed so far. Also their decentralized web structure makes it harder to get information about regularity of their analyses.


done 1b marked as Partially compliant by Sarphan Uzunoğlu.

Section 2: Nonpartisanship and Fairness

Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

In order to maintain coherence in the credibility ratings issued by scientists, we provide a clear guide defining the meaning of each “overall credibility rating” level and ask reviewers to evaluate articles based on 6 criteria (Factual Accuracy, Scientific understanding, Logic/Reasoning, Precision/Clarity, Sources Quality, Fairness/Objectivity). Read our guidelines here: https://sciencefeedback.co/process/#tit4 .

Scientists contributing to our analyses are asked to conform to high quality community standards upon sign up, which require them to comment only on claims related to their expertise. (Read our community standards: https://sciencefeedback.co/community-standards/ )

Science Feedback editors select articles and claims for review from a variety of media—across the political spectrum—according to their influence on social media (typically measured by Buzzsumo.com), the quantity or degree of claimed scientific evidence within the reporting, and potential relevance to shaping public debate.

As stated on our Process page, we strive for our reviews to be representative of the spectrum of influential science related discussion in the media. We review articles and claims in a variety of media, without a priori perspective, regardless of whether they insightfully report on or exaggerate scientific findings.

1. Senator Sanders’ claim that climate change is making tornadoes worse isn’t supported by published research

https://climatefeedback.org/claimreview/senator-sanders-claim-that-climate-change-is-making-tornadoes-worse-isnt-supported-by-published-research/

2. President Trump’s claim that water supply policy has worsened California wildfires is baseless

https://climatefeedback.org/claimreview/president-trumps-claim-that-water-supply-policy-has-worsened-california-wildfires-is-baseless/

3. Guardian story on climate impacts of diet gets mixed reviews from scientists

https://climatefeedback.org/evaluation/guardian-story-on-climate-impacts-of-diet-gets-mixed-reviews-from-scientists-damian-carrington/

4. Claim that human civilization could end in 30 years is speculative, not supported with evidence

https://climatefeedback.org/evaluation/iflscience-story-on-speculative-report-provides-little-scientific-context-james-felton/

5. It’s true that the current carbon dioxide level is higher than any time in human existence

https://climatefeedback.org/claimreview/its-true-that-the-current-carbon-dioxide-level-is-higher-than-any-time-in-human-existence/

6. Breitbart article baselessly claims a study of past climate invalidates human-caused climate change

https://climatefeedback.org/evaluation/breitbart-article-baselessly-claims-a-study-of-past-climate-invalidates-human-caused-climate-change-john-nolte/

7. Article wrongly claims that measles vaccine claims more lives than measles infection, misinterprets epidemiological data

https://healthfeedback.org/claimreview/article-wrongly-claims-that-measles-vaccine-claims-more-lives-than-measles-infection-misinterprets-epidemiological-data/

8. Measles outbreak in US due to unvaccinated citizens, not illegal immigrants

https://healthfeedback.org/claimreview/measles-outbreak-due-to-unvaccinated-citizens-not-illegal-immigrants-alex-jones-infowars-natural-news/

9. New Scientist article accurately summarises polycystic ovary syndrome (PCOS) research but overstates significance of animal studies

https://healthfeedback.org/evaluation/new-scientist-article-accurately-summarises-pcos-research-but-overstates-significance-of-animal-studies-alice-klein/

10. Article claiming vaccines cause autoimmunity and autism due to fetal DNA contaminants found unsupported and implausible

https://healthfeedback.org/evaluation/article-claiming-vaccines-cause-autoimmunity-and-autism-due-to-fetal-dna-contaminants-found-unsupported-and-implausible-theresa-deisher/ 

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

Scientists contributing to their analyses comment only on claims related to their expertise. It is seen on the website looking at the profiles and former studies of experts. Their understanding of measuring influence is interesting and focus on particularly popular subjects is understandable. All the cases they provided seem very relevant.


done_all 2a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

We ensure that our staff are not involved in political parties or advocacy organizations, as mentioned on our Community Standards’ page: https://sciencefeedback.co/community-standards/

“Science Feedback is dedicated to science education and does not advocate for any particular policy, nor does it support any political candidate or party. Science Feedback ensures that our staff are not directly involved in political parties or advocacy organizations that could bias their neutrality and undermine their commitment to scientific accuracy.”

Here is the note we include in our job posting for editorial staffs and contributors: “Non-partisanship: Our staff and freelance contributors are required to not be involved in advocacy/political campaigning.” see eg this job post: https://docs.google.com/document/d/1Kaev6Cnjh4mJPUyLyAkHdhnM_0fXS_5uyz9hu-0K8_E/

Our scientific feedbacks do not constitute endorsements of the authors’ political or economic ideology, rather they are assessments of the scientific foundations and reasoning of the argumentation contained within each article.

Similarly we ask scientists reviewing with Science Feedback not to “evaluate the opinion of the author, but instead the scientific accuracy of facts contained within the text, and the scientific quality of reasoning used.” (As noted on our methods page: https://sciencefeedback.co/process/ )

Note that we do not primarily assess statements made by politicians, but mostly by journalists, pundits, and social media influencers.

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago
Especially thanks to the theme they focus on, they have no problems about political partisanship. They are very non-partisan and both their community standards and job post they exemplified are great proofs of how much do they care about their independent and objective status. Their policy is including both freelancers and full timers and it is also a big advantage.

done_all 2b marked as Fully compliant by Sarphan Uzunoğlu.

Section 3: Transparency of Sources

Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

Our fact-checking process and policy on sources is described in our Process page. At Science Feedback, we ask our sources (scientists) to comment on articles directly and to indicate whether the facts underlying the reasoning are consistent with up-to-date scientific knowledge. That way, readers can hear directly from sources who have actual knowledge and expertise on the subject at hand. Each scientist is clearly identifiable with a link to a professional page listing his/her scientific publications.

In our generally proposed format for comments, we ask scientists to cite the most relevant supporting sources—references from the peer-reviewed literature whenever possible. We ask reviewers to represent the state of knowledge in the scientific literature, using strongly supported scientific theories and observations as references, and to refrain from pointing to isolated or weakly supported findings.

Our scientist reviewers are all listed on our Community page, as well as the objective criteria we use to accept contributing reviewers: https://sciencefeedback.co/for-scientists/#ref. To maintain transparency, we require contributors to identify themselves on our website using their real names and photographs. It is easy for anyone to contact the scientists and replicate the fact-check. Our article rating system is based on the average given by scientists, ensuring transparency and objectivity in the way we reach our final conclusion.

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

I can verify that they ask their experts to comment on articles directly and to indicate whether the facts underlying the reasoning are consistent with up-to-date scientific knowledge. In the instances they provided, it is possible to see many related citations. They are open to contributions of new experts but it would be good for the organization to clarify how to they deal with quality of the experts' work and quality of their work.


done 3a marked as Partially compliant by Sarphan Uzunoğlu.

Section 4: Transparency of Funding & Organization

Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago
Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

They are open about their donors and it is a nice characteristic for such an organization.


done_all 4a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 4b
Staff
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago
Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

They are transparent about identities of their experts and they are accessible through their website.


done_all 4b marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 4c
Contact
Evidence required: Please link to the section where readers can get in touch with the organization.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

Readers contact us via this online form: https://sciencefeedback.co/contact-us/

This form is linked from our Methodology Page (in the first section), from any page in the footer, as well as at the bottom of every review.

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago
Their contact page is visible and available.

done_all 4c marked as Fully compliant by Sarphan Uzunoğlu.

Section 5: Transparency of Methodology

Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

Methodology for article reviews: https://sciencefeedback.co/process/ ; Methodology for claim reviews: https://sciencefeedback.co/claim-reviews-framework/ 

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

They provided a compliant, detailed and totally acceptable methodology. It totall fits to the objectives and manifest of the organization and has detailed explanation regarding their practical routine in fact-checking processes. Of course having two different methodology for claims and articles may seem confusing at first hand but it is totally relevant and acceptable.


done_all 5a marked as Fully compliant by Sarphan Uzunoğlu.

Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

Readers submit suggestions of articles or claims to review via our contact form.

They are invited to do so at the bottom of every review with the following text:

“Please get in touch if you have any comment or think there is an important claim or article that would need to be reviewed.”

They are also invited to do so on the process page with the following text:

“If you wish to submit a suggestion of an article or claim to review, please use this online form. Please note that we focus on reviewing claims and articles that are scientifically verifiable and that reach large audiences.”

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago
Even if it is not interactive and inviting enough for the readers, they provide a way for people to send claims or suggestions.

done 5b marked as Partially compliant by Sarphan Uzunoğlu.

Section 6: Open & Honest Corrections Policy

Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

Our correction policy is described here: https://sciencefeedback.co/process/#correction

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

Their policy regarding collections is very clear and rational. However, there might be an indication for the articles under review after user feedback.


done 6a marked as Partially compliant by Sarphan Uzunoğlu.

Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.

Science Feedback
09-Jun-2019 (5 months ago) Updated: 5 months ago

We have handled no correction requests, and we have not made any major corrections during our tenure as of today.

In the cases where we made minor corrections to the article after its publication, we included an “Update” section below the summary to explain the nature of the update.

For example, we included this update: “The analysis has been updated to include several comments received just after the time of publication. The main conclusion of the analysis is unchanged.” in this analysis: https://climatefeedback.org/evaluation/scientists-explain-what-new-york-magazine-article-on-the-uninhabitable-earth-gets-wrong-david-wallace-wells/ 

Sarphan Uzunoğlu Assessor
13-Jun-2019 (5 months ago) Updated: 5 months ago

As they didn't have correction requests that often, they don't have a list like that. But they should introduce such a system to be more reliable.


done 6b marked as Partially compliant by Sarphan Uzunoğlu.