Science Feedback

Organization: Science Feedback
Applicant: Dr. Emmanuel Vincent
Assessor: Michael Wagner
Conclusion and recommendations
on 07-Jun-2018 (1 year ago)

Michael Wagner wrote:

Climate Feedback is an excellent fact-checking site that uses scientific experts to check media reporting and newsworthy individual claims about climate change. The site is fair and rigorous, communicates clearly and is transparent about their funding. The annotations provided in the fact-checks give clear explanations to readers about the veracity of the stories and claims they check. It is my view that they respect and adhere to the code of principles.

The site should consider providing readers with greater clarity about the extent to which political involvement is prohibited (and the lack of involvement is verified). In order for readers to better be able to piece together how the ratings were derived, Climate Feedback should consider more robust linking to evidence used in the annotations that the scholars used to make their judgments. Finally, Climate Feedback should consider a Corrections tag so that it is easier for readers to find fact-checks that were published but then corrected in some way.

on 07-Jun-2018 (1 year ago)

Michael Wagner recommended Accept


Section 1: Organization

Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback is a project hosted by the Center for Climate Communication at the University of California, Merced. Please see these articles for reference (here: http://www.ucmerced.edu/news/2015/climate-scientist-aims-change-national-dialogue-one-story-time and here: https://www.universityofcalifornia.edu/news/can-you-trust-what-you-read-about-climate-change). As an official program, Climate Feedback benefits from the tax-exempt status of the University of California, Merced.


Climate Feedback is dedicated to provide insight on the credibility of information that shapes public opinion on climate change. We organize a worldwide network of scientists to produce analyses and fact-checks to help Internet users—from the general public to influential decision-makers— know which climate news to trust. Climate Feedback’s mission is pedagogical: we strive to explain whether and why information is or is not consistent with the science, helping to increase civic understanding of climate issues and to improve readers’ critical thinking skills.

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback is a fairly unique operation in the world of fact-checking. The site, affiliated with the Center for Climate Communication at the University of California-Merced, is not powered by fact-checking journalists. Rather, the fact-checks are written be scientists, Ph.D.-holding scholars who seek to rate the veracity of news (broadly defined) articles containing information about climate change and individual claims those in the public eye make about climate change. The site is an official program at the university, allowing Climate Feedback to enjoy tax exempt status.


done_all 1a marked as Fully compliant by Michael Wagner.

Criterion 1b
Archive
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback provides scientific fact-checks both in the form of article reviews and claim reviews.

Article reviews invite scientists to fact-check a number of claims at once and provide in-depth analysis on a whole article. Beyond checking individual facts, it is often necessary in science to analyze a whole article to check the quality of its logic and scientific reasoning, i.e. how evidence are used to support a conclusion.

Claim reviews are focused on checking the veracity of individual claims, some of which are extracted from articles we provide a full review of, while some are extracted from prominent articles that contain only a few verifiable statements or from statements by influential figures that are being discussed widely in the media.

Claim reviews are typically further commented on in “Insights” post providing explanation and context of the topic at hand.

Article reviews archive: http://climatefeedback.org/feedbacks/

Claim reviews archive: http://climatefeedback.org/claim-reviews/

Insights articles archive: https://climatefeedback.org/insights/

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

I rated the publication criteria of an average of one report a week over the previous three months as complete compliance even though there are weeks were there are no reports published. Publication seems to slow in the middle of the traditional semester season - understandable given that their reviewers are largely academics. Overall, the average since January (aggregating article reviews, claim reviews and insights is slightly under 1 per week for the last five months). Given the scientifically-rooted standards of the site, and comparing the pace of scholarly publishing to journalistic publishing, Climate Feedback’s pace of publication is good. Indeed, the article reviews appear to involve substantially more work to fact-check than the checking of an individual claim made by a newsworthy source (the bread and butter of fact-checking). However, considering the roster of reviewers the site claims, it does appear as though they might be encouraged to pursue a slightly more aggressive publication schedule, but it is my conclusion that the site meets the spirit of the IFCN criteria.


done_all 1b marked as Fully compliant by Michael Wagner.

Section 2: Nonpartisanship and Fairness

Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

In order to maintain coherence in the credibility ratings issued by scientists, we are providing a clear guide defining the meaning of each “overall credibility rating” level and ask reviewers to evaluate articles based on 6 criteria (Factual Accuracy, Scientific understanding, Logic/Reasoning, Precision/Clarity, Sources Quality, Fairness/Objectivity. Read our guidelines here: http://climatefeedback.org/process/#tit4).

Scientists contributing to our analyses are required to conform to high quality community standards upon sign up, which require them to comment only on claims related to their expertise. (read our community standards: http://climatefeedback.org/community-standards/ )

Climate Feedback selects articles and claims for review from a variety of media - across the political spectrum- according to their influence on social media (typically measured by Buzzsumo.com), the quantity or degree of claimed scientific evidence within the reporting and potential relevance to shaping public debate.

As stated on our Process page, we strive for our reviews to be representative of the spectrum of influential climate discussion in the media. We review articles and claims in a variety of media, without a priori perspective, regardless of whether they insightfully report on, exaggerate or downplay the consequences of climate change.

Article reviews:

1. The Australian’s coverage of Great Barrier Reef study creates perception that scientists are divided: https://climatefeedback.org/evaluation/the-australian-coverage-great-barrier-reef-perception-scientists-divided-global-warming-graham-lloyd/

2. Investor’s Business Daily editorial misrepresents study to claim plants will prevent dangerous climate change https://climatefeedback.org/evaluation/investors-business-daily-editorial-misrepresents-study-to-claim-plants-will-prevent-dangerous-climate-change/

3. New satellite measurements show sea level rise is accelerating, as CNN accurately reports https://climatefeedback.org/evaluation/satellite-measurements-accelerating-sea-level-rise-cnn-accurately-reports-brandon-miller/

4. Grist article on an “Ice Apocalypse” mostly accurate, but doesn’t make the likelihood of that apocalypse clear enough to readers https://climatefeedback.org/evaluation/antarctica-doomsday-glaciers-could-flood-coastal-cities-grist-eric-holthaus/

5. The Independent makes a giant leap in stating that modern global warming could be “worse than thought” based on a single study https://climatefeedback.org/evaluation/independent-makes-giant-leap-global-warming-worse-than-thought-single-study-andrew-griffin/

6. Scientists explain what New York Magazine article on “The Uninhabitable Earth” gets wrong

https://climatefeedback.org/evaluation/scientists-explain-what-new-york-magazine-article-on-the-uninhabitable-earth-gets-wrong-david-wallace-wells/

Claim reviews:

7. Rush Limbaugh falsely claims there is no evidence of human-caused global warming https://climatefeedback.org/claimreview/rush-limbaugh-falsely-claims-there-is-no-evidence-of-human-caused-global-warming/

8. Breitbart repeats blogger’s unsupported claim that NOAA manipulates data to exaggerate warming https://climatefeedback.org/claimreview/breitbart-repeats-bloggers-unsupported-claim-noaa-manipulates-data-exaggerate-warming/

9. EPA’s Scott Pruitt incorrectly suggests climate change might not be “a bad thing” https://climatefeedback.org/claimreview/epas-scott-pruitt-incorrectly-suggests-climate-change-might-not-bad-thing/

10. President Trump’s claim of growing ice does not reflect reality https://climatefeedback.org/claimreview/president-trumps-claim-growing-ice-not-reflect-reality/ 

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback performs fact-checks about one large topic - climate change. That said, they cover a variety of speakers and professional media outlets that produce comments or stories about climate change and related issues. The site focuses on a “side” with respect to choosing the side of the preponderance of evidence in science, but Climate Feedback does not endorse political parties, an ideological perspective or the like. They select a wide variety of important sources to fact-check. It is the case that some sources and outlets perform better (i.e. New York Times) than others (i.e. Wall Street Journal), but that is a consequence related to the veracity of the things those sources and outlets claim, not a discernible political bias.


done_all 2a marked as Fully compliant by Michael Wagner.

Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

We ensure that our staff are not involved in political parties or advocacy organizations as mentioned on our Community Standards’ page: http://climatefeedback.org/community-standards/

Our scientific feedbacks do not constitute endorsements of the author’s political or economic ideology, rather they are assessments of the scientific foundations and reasoning of the argumentation contained within each article.

Similarly we ask scientists reviewing with Climate Feedback not to “evaluate the opinion of the author, but instead the scientific accuracy of facts contained within the text, and the scientific quality of reasoning used.” (as noted on our methods page: http://climatefeedback.org/process/)

Note that we do not primarily assess statements made by politicians, but mostly by journalists and pundits.

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback has clear language endorsing the prohibition on support for political parties and advocacy organizations, but the site does not reveal how they verify this requirement with their writers. Most fact-checking sites have journalists who are only working for the fact-checking site. Climate Feedback uses scholars who are generally eminent professors who study climate change. As such, it may be more difficult for Climate Feedback to ensure adherence to their politics-related policies.


done_all 2b marked as Fully compliant by Michael Wagner.

Section 3: Transparency of Sources

Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Our fact-checking process and policy on sources is described in our Process page. At Climate Feedback, we ask our sources (scientists) to comment on articles directly and to indicate whether the facts underlying the reasoning are consistent with up-to-date scientific knowledge. That way, readers can hear directly from sources who have actual knowledge and expertise on the subject at hand. Each scientist is clearly identifiable with a link to a professional page listing his/her scientific publications.

In our generally proposed format for comments, we ask scientists to cite the most relevant supporting sources --reference(s) from the peer-reviewed literature whenever possible. We ask reviewers to represent the state of knowledge in the scientific literature, using strongly supported scientific theories and observations as references, and to refrain from pointing to partial/isolated/weakly supported findings.

Our scientist reviewers are all listed on our Community page, as well as the objective criteria we use to accept contributing reviewers: http://climatefeedback.org/for-scientists/#ref. To maintain transparency, we require contributors to identify themselves on our website using their real names and photographs. It is easy for anyone to contact the scientists and replicate the fact-check. Our ruling system is based on the average given by scientists, ensuring transparency and objectivity in the way we reach our final conclusion.

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback’s writers are already experts in climate change, so, in a sense, they are the writers and the sources. Indeed, this is probably the most impressive part of what the site is doing - using real experts to weigh in on mass mediated claims about climate change. Sometimes, the fact-checkers will also link to articles or share evidence (like a figure, for example, that shows the average temperature of a region over time) that help the audience understand how the rating was arrived at by the writers. However, often, the annotations in the articles and the summaries of the ratings provided do not include the evidence that was likely instrumental in the completing of the fact-check. They describe their process clearly here: https://climatefeedback.org/process/ and provide incredible annotations from the scientists reviewing the claim/story in their reports. The annotations do not always provide evidence beyond the reliance upon the expertise of the annotator.  


done 3a marked as Partially compliant by Michael Wagner.

Section 4: Transparency of Funding & Organization

Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago
Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback details their funding sources in an exceptionally clear and thorough way.


done_all 4a marked as Fully compliant by Michael Wagner.

Criterion 4b
Staff
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Our Team and Advisors page: http://climatefeedback.org/team-advisors-contributors/ ; Our Network of scientist reviewers: http://climatefeedback.org/community/

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

 They also provide clear and easy-to-find biographies of their contributors.


done_all 4b marked as Fully compliant by Michael Wagner.

Criterion 4c
Contact
Evidence required: Please link to the section where readers can get in touch with the organization.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Readers contact us via this online form: http://climatefeedback.org/contact-us/ ; This form is linked from our Methodology Page (in the first section), from any page in the footer, as well as at the bottom of every reviews.

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

 It is easy to find the contact information so that interested readers can contact the site with questions or suggestions.


done_all 4c marked as Fully compliant by Michael Wagner.

Section 5: Transparency of Methodology

Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Methodology for article reviews: http://climatefeedback.org/process/ ; Methodology for claim reviews: http://climatefeedback.org/claim-reviews-framework/

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback uses both a scientific credibility scale that is a Likert-type scale and a list of tags that more comprehensively describe what is laudable or problematic about the story or claim being fact-checked.


done_all 5a marked as Fully compliant by Michael Wagner.

Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Readers submit suggestions of articles or claims to review via our contact form. They are invited to do so at the bottom of every review with the following text: “Please get in touch if you have any comment or think there is an important claim or article that would need to be reviewed.” They are also invited to do so on the process page with the following text: “If you wish to submit a suggestion of an article or claim to review, please use this online form. Please note that we focus on reviewing claims and articles that are scientifically verifiable and that reach large audiences.”

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

 Readers can contact Climate Feedback and ask them to check a claim. The site does a good job of expressing their mission (pedagogical) and explaining to their readers what they do, why they do it and why it is valuable.


done_all 5b marked as Fully compliant by Michael Wagner.

Section 6: Open & Honest Corrections Policy

Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

Our correction policy is described here: https://climatefeedback.org/process/#correction

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

Climate Feedback has a corrections policy (https://climatefeedback.org/process/#correction) that they note is rarely needed. However, the site does not provide readers any way of knowing whether or how often their reporting has been questioned or challenged by readers.


done_all 6a marked as Fully compliant by Michael Wagner.

Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.

Science Feedback
03-May-2018 (1 year ago) Updated: 1 year ago

We have handled no correction requests, and we have not made any major correction during our tenure as of today.

In the cases where we made minor corrections to the article after its publication, we included an “Update” section below the summary to explain the nature of the update.

For example, we included this update: “The analysis has been updated to include several comments received just after the time of publication. The main conclusion of the analysis is unchanged.” in this analysis: https://climatefeedback.org/evaluation/scientists-explain-what-new-york-magazine-article-on-the-uninhabitable-earth-gets-wrong-david-wallace-wells/

Michael Wagner Assessor
03-May-2018 (1 year ago) Updated: 1 year ago

The site makes corrections and notes them, but it would be of use to have a Corrections tag - such as one approximating the one PolitiFact uses - to make it easier to search for corrections. It is also suggested that the Climate Feedback team consider publishing challenges to their work, even if those challenges are ultimately not persuasive with respect to the decision that a correction is necessary.


done_all 6b marked as Fully compliant by Michael Wagner.