Science Feedback

Organization: Science Feedback
Applicant: Dr. Emmanuel Vincent
Assessor: Steve Fox
Conclusion and recommendations
on 25-Mar-2017 (2 years ago)

Steve Fox wrote:

Overall, the methodology and professionalism shown by the editors here is of high quality. The fact that they have multiple scientists fact-checking questions and providing analysis is impressive. The annotation is a strong feature and one that is truly engaging.

However, the editors here fall short when it comes to interactivity. The editors appear to hold the audience at arms-length. That needs to change. To be a true fact-checking site, ClimateFeedback needs to interact with its audience and feature its inner workings more prominently.

The site could benefit from some professional journalists in this area, as well as language that is less vague and navigation that is more specific. The standard journalistic practices of seeking to interact with the audience and being transparent about Correction policies needs to be adopted by this site. It’s a strong academic site but needs some major revisions to be considered a true fact-checking site.

on 25-Mar-2017 (2 years ago)

Steve Fox recommended Accept with edits


Section 1: Organization

Criterion 1a
Proof of registration
Evidence required: Please provide evidence that the signatory is a legally-registered organization set up exclusively for the purpose of fact-checking or the distinct fact-checking project of a recognized media house or research institution.

Science Feedback
25-Mar-2017 (2 years ago)

Climate Feedback is a project hosted by the Center for Climate Communication at the University of California, Merced (http://snri.ucmerced.edu/node/485). Please see these articles for reference (here: http://www.ucmerced.edu/news/2015/climate-scientist-aims-change-national-dialogue-one-story-time and here: https://www.universityofcalifornia.edu/news/can-you-trust-what-you-read-about-climate-change). As an official program, Climate Feedback benefits from the tax-exempt status of the University of California, Merced.

Climate Feedback is dedicated to provide insight on the credibility of information that shapes public opinion on climate change. We organize a worldwide network of scientists to produce analyses and fact-checks to help Internet users—from the general public to influential decision-makers— know which climate news to trust. Climate Feedback’s mission is pedagogical: we strive to explain whether and why information is or is not consistent with the science, helping to increase civic understanding of climate issues and to improve readers’ critical thinking skills.

Steve Fox Assessor
25-Mar-2017 (2 years ago) Updated: 10 months ago

This site is fairly unique and first-time visitors see pretty quickly the reference to this being a fact-checking site by a “network of scientists.” That indicates pretty quickly that this may not be a traditional media organization but may be more of a research organization. The homepage is fairly condensed (which is a good thing) and at the bottom of the page you find language indicating that this site is hosted by an academic institution and appears to be a non-profit group doing the work here. That perception is further reinforced by the group’s “About” page, which links to a “Partners, Funders & Donors” page, giving complete transparency to where its funding is coming from. While the “What We Do” section is strong, it would benefit from a sentence or two explaining where its funding is coming from.


done_all 1a marked as Fully compliant by Steve Fox.

Criterion 1b
Archive
Evidence required: Insert a link to the archive of fact checks published in the previous three months. If you do not collect all fact checks in one place, please explain how the fact-checking is conducted by your organization.

Science Feedback
25-Mar-2017 (2 years ago)

Our article reviews (that we call “Scientific Feedbacks”) are archived here: http://climatefeedback.org/feedbacks/

Our “claim reviews” are archived here: http://climatefeedback.org/claim-reviews/

Our “Scientific Feedbacks” provide in-depth analysis on a whole article, as well as “fact-checking” a number of claims at once. Beyond checking the facts, special attention is given to checking the quality of the logic and reasoning to analyze and expose logical flaws and rhetorical manipulation.

Our “Claim Reviews” are focused on checking the veracity of individual claims, some of which are extracted from articles we provide a full review of, while some are extracted from prominent articles that contain only a few verifiable statements. When the claim reviews are not included in an article review, they are commented on in a post providing further explanation of the topic at hand (these are archived here: http://climatefeedback.org/blog-posts/)

Steve Fox Assessor
25-Mar-2017 (2 years ago)

There are collections of regular fact checks but the phrasing of the navigation isn’t based on traditional phrasings of most news organizations. The vagueness of “Scientific Feedbacks” and “Blog” leaves the first-visitor wondering where to click. The navigation could benefit from clearer language.


done 1b marked as Partially compliant by Steve Fox.

Section 2: Nonpartisanship and Fairness

Criterion 2a
Body of work sample
Evidence required: Please share links to ten fact checks that better represent the scope and consistency of your fact-checking. Provide a short explanation of how your organization strives to maintain coherent standards across fact checks.

Science Feedback
25-Mar-2017 (2 years ago) Updated: 10 months ago

In order to maintain coherence in the credibility ratings issued by scientists, we are providing a clear guide defining the meaning of each rating level and questions guiding reviewers on what to look for in articles (read our suggested guidelines for the overall scientific credibility rating http://climatefeedback.org/process/#tit4).

Scientists contributing to our analyses are also asked to conform to high quality community standards, which require them to comment only on claims related to their expertise. (read more about our community standards: http://climatefeedback.org/community-standards/ )

Climate Feedback selects articles and claims for review from a variety of media - across the political spectrum- according to their influence, the quantity or degree of claimed scientific evidence within the reporting and potential relevance to shaping public debate. We are nonpartisan and apply the same methodology to claims made in a variety of media, as well as exposing both the claims that exaggerate or downplay the consequences of climate change.

List of “Scientific Feedbacks”:

1. Analysis of “Scientists have just detected a major change to the Earth’s oceans linked to a warming climate”

in The Washington Post, by Chris Mooney: http://climatefeedback.org/evaluation/scientists-just-detected-major-change-earths-oceans-linked-warming-climate-chris-mooney-the-washington-post/

2. Analysis of “U.S. scientists officially declare 2016 the hottest year on record. That makes three in a row.”

in The Washington Post, by Chris Mooney: http://climatefeedback.org/evaluation/scientists-officially-declare-2016-hottest-year-record-chris-mooney-the-washington-post/

3. Analysis of “Stunning new data indicates El Nino drove record highs in global temperatures…”

in Daily Mail, by David Rose: http://climatefeedback.org/evaluation/stunning-new-data-indicates-el-nino-drove-record-highs-global-temperatures-david-rose-daily-mail/

4. Analysis of “The Phony War Against CO2”

in The Wall Street Journal, by Rodney Nichols and Harrison Schmitt: http://climatefeedback.org/evaluation/the-phony-war-against-co2-the-wall-street-journal-rodney-nichols-harrison-schmitt/

5. Analysis of “Flooding of Coast, Caused by Global Warming, Has Already Begun”

in The New York Times, by Justin Gillis: http://climatefeedback.org/evaluation/flooding-of-coast-caused-by-global-warming-has-already-begun-justin-gillis-the-new-york-times/

6. Analysis of “Next year or the year after, the Arctic will be free of ice”

in The Guardian, by Robin McKie: http://climatefeedback.org/evaluation/arctic-will-be-ice-free-in-summer-next-year-robin-mckie-peter-wadhams-the-guardian/

7. Analysis of “Scientists warn of ‘global climate emergency’ over shifting jet stream”

in The Independent, by Gabriel Samuels: http://climatefeedback.org/evaluation/climate-change-emergency-jet-stream-shift-warning-global-warming-extreme-weather-gabriel-samuels-the-independent/

8. Analysis of “Thanks to climate change, the Arctic is turning green”

in The Washington Post, by Chris Mooney: http://climatefeedback.org/evaluation/its-official-humans-are-making-the-earth-much-greener-washington-post-chris-mooney/

9. Analysis of “Seas Are Rising at Fastest Rate in Last 28 Centuries”

in The New York Times, by Justin Gillis:

http://climatefeedback.org/evaluation/sea-level-rise-global-warming-climate-change/

10. Analysis of “…in many ways global warming will be a good thing”

Published in The Telegraph, by Bjorn Lomborg

http://climatefeedback.org/evaluation/the-telegraph-bjorn-lomborg-in-many-ways-global-warming-will-be-good-thing/ 

Steve Fox Assessor
25-Mar-2017 (2 years ago)

The standards outlined on the “Community Standards” page are impressive. These standards are linked off the “Community” page and cite the need for reviewers to “practice and promote scientific inquiry.” It’s an impressive, thorough list of standards which includes “limiting authoritative comments to their area of expertise; when commenting outside their direct field of expertise, contributors must back their claims using strongly supported scientific theories and observations and cite reliable sources.” Traditional journalism sites would do well to look at the wording on these standards and adapt them to their own standards.

What’s refreshing to see from this site is that not only they fact-check claims by news organizations at both ends of the political spectrum but they also fact-check columns or claims made under the “opinion” sections of news organizations. In one analysis, one scientist writes: “The article speaks about scientific questions under an “opinion” banner—as if questions about the role of CO2 in the Earth system could be a matter of opinions.” I’m reminded of the old saying “everyone is entitled to their own opinion but not their own set of facts.” It’s refreshing to see such analysis of content in “opinion” sections and to see that analysis to extend to all claims – not only those saying the science is exaggerated but also claims of doom that are overly exaggerated.


done 2a marked as Partially compliant by Steve Fox.

Criterion 2b
Nonpartisanship policy
Evidence required: Please share evidence of your policy preventing staff from direct involvement in political parties and advocacy organizations. Please also indicate the policy your organization has as a whole regarding advocacy and supporting political candidates.

Science Feedback
25-Mar-2017 (2 years ago)

We ensure that our staff are not involved in political parties or advocacy organizations as mentioned on our Community Standards’ page: http://climatefeedback.org/community-standards/

Our “Scientific Feedbacks” do not constitute endorsements of the author’s political or economic ideology, rather they are assessments of the scientific foundations and reasoning of the argumentation contained within each article.

Similarly we ask scientists reviewing with Climate Feedback not to “evaluate the opinion of the author, but instead the scientific accuracy of facts contained within the text, and the scientific quality of reasoning used.” (as noted on our methods page: http://climatefeedback.org/process/)

Note that we do not primarily assess statements made by politicians, but mostly by journalists and pundits.

Steve Fox Assessor
25-Mar-2017 (2 years ago)

The editors make clear on their “Community Standards” page that their work is focused on the science, not the politics of an issue: “Climate Feedback is dedicated to science education and does not advocate for any particular policy, nor does it support any political candidate or party.” Again, this is pretty powerful language and it seems like wording that should be prominently featured on the homepage and/or every analysis that is published. It can be hard for some to distinguish between science and politics when it comes to Climate Change and the dedication to science should be featured prominently.

And, while the “Community Standards” page is impressive, it does not appear to be linked anywhere on the site. Being transparent and prominently linking to this page would only help the site’s credibility.


done_all 2b marked as Fully compliant by Steve Fox.

Section 3: Transparency of Sources

Criterion 3a
Sources Policy
Please share a brief and public explanation (500 words max) of how sources are provided in enough detail that readers could replicate the fact check. If you have a public policy on how you find and use sources for your fact-checking, it should be shared here.

Science Feedback
25-Mar-2017 (2 years ago)

Our fact-checking process and policy on sources is described in our Process page. At Climate Feedback, we ask our sources (scientists) to comment on articles directly and to indicate whether the facts underlying the reasoning are consistent with up-to-date scientific knowledge. That way, readers can hear directly from sources who have actual knowledge and expertise on the subject at hand. Each scientist is clearly identifiable with a link to a professional page listing his/her scientific publications.

In our generally proposed format for comments, we ask scientists to cite the most relevant supporting sources --reference(s) from the peer-reviewed literature whenever possible. We ask reviewers to represent the state of knowledge in the scientific literature, using strongly supported scientific theories and observations as references, and to refrain from pointing to partial/isolated/weakly supported findings.

Our scientist reviewers are all listed on our Community page, as well as the objective criteria we use to accept contributing reviewers: http://climatefeedback.org/for-scientists/#ref. To maintain transparency, we require contributors to identify themselves on our website using their real names and photographs. It is easy for anyone to contact the scientists and replicate the fact-check. Our ruling system is based on the average given by scientists, ensuring transparency and objectivity in the way we reach our final conclusion regarding the “credibility” of an article.

Steve Fox Assessor
25-Mar-2017 (2 years ago)

One of the more exceptions features of this site is the ability for readers to “see the scientists’ annotations in context.” Readers can click on a link that takes them to the article in question, with areas of the article highlighted. When you click on those highlights, notes, questions and analysis from scientists analyzing the article show up in the sidebar. While the “overall rating” given to each article gives the reader an indication of how factual the claims are, this impressive feature allows the reader to go back and replicate the fact-check by going through point-by-point made in the article and seeing whether it is supported or refuted. Scientists provide data, analysis and research in the annotation – providing even more links in support or refutation of different points being made.

The annotation feature is one readers are beginning to see featured on other news sites – but is often restricted to “State of the Union” speeches or other major events. While these efforts can be time-intensive and costly for general news organizations, such approaches should not be restricted to niche organizations and the editors at Climate Feedback should be commended for their dedicated efforts to make such analysis happen on a regular basis.


done_all 3a marked as Fully compliant by Steve Fox.

Section 4: Transparency of Funding & Organization

Criterion 4a
Funding Sources
Evidence required: Please link to the section where you publicly list your sources of funding (including, if they exist, any rules around which types of funding you do or don't accept), or a statement on ownership if you are the branch of an established media organization or research institution.

Science Feedback
25-Mar-2017 (2 years ago) Updated: 10 months ago
Steve Fox Assessor
25-Mar-2017 (2 years ago)

The “Partners, Funders and Donors” page is linked off the “About” page on the site, and is also referenced at the bottom of the site’s homepage. The level of transparency is impressive, with individual donors listed by name – and includes links to the bios of some of the donors as well. The editors list the names of all donors to the site – including those donating as little as $5 – also an indication of how this site appeals to those at the grassroots level.


done_all 4a marked as Fully compliant by Steve Fox.

Criterion 4b
Staff
Evidence required: Please link to the section detailing all authors and key actors behind your fact-checking project with their biographies. You can also list the name and bios of the members of the editorial board, pool of experts, advisory board, etc. if your organization has those.

Science Feedback
25-Mar-2017 (2 years ago)

Our Team and Advisors page: http://climatefeedback.org/team-advisors-contributors/, our Network of scientist reviewers: http://climatefeedback.org/community/

Steve Fox Assessor
25-Mar-2017 (2 years ago)

Each article that is analyzed includes a list of the reviewers on the right sidebar. This is striking because the number of reviewers on many articles is substantial (often more than five and sometimes in double figures.) This list also includes links to the bios of those reviewing the article. A longer list of bios is listed on the site’s community page. It’s an impressive list of scientists from all over the world but it’s hard not to notice that the list is dominated by white males – an issue well-documented within the sciences.


done_all 4b marked as Fully compliant by Steve Fox.

Criterion 4c
Contact
Evidence required: Please link to the section where readers can get in touch with the organization.

Science Feedback
25-Mar-2017 (2 years ago)
Steve Fox Assessor
25-Mar-2017 (2 years ago)

The ability of those in the audience to interact with scientists and reviewers on the site to offer comments, complaints, suggestions for fact-checks is markedly missing from this site. There is one “Contact Us” link at the bottom of the homepage but it gets lost and its lack of a “call to interact” throughout the site almost makes the homepage link non-existent.

This lack of interactivity makes this site more or an academic exercise instead of a traditional fact-checking site by a general news organizations. The site gives off the impression that renowned scientists are providing their analyses and opinions, and while that’s a strong image to create -- that’s the end of it. Such approaches are what create perceptions of arrogance with academics and this omission needs to be remedied. Engaging in a dialogue with the audience is important – especially on the topic of climate change -- and should be part of the mission here.


cancel 4c marked as Non compliant by Steve Fox.

Section 5: Transparency of Methodology

Criterion 5a
Detailed Methodology
Evidence required: Please link to a section or article detailing the steps you follow for your fact-checking work.

Science Feedback
25-Mar-2017 (2 years ago)

Methodology for article reviews: http://climatefeedback.org/process/

Methodology for claim reviews: http://climatefeedback.org/claim-reviews-framework/

Steve Fox Assessor
25-Mar-2017 (2 years ago)

The methodology page is impressive. The page is linked off the “About” page but given the level of analysis the scientists provide, it seems their process should be highlighted off the homepage and off each article. Especially in this political climate, which climate change as a major political football, calling attention to their thorough fact-checking process seems like a smart move. It opens the doors a little while adding credibility and getting past the aloof scientist image. What’s also smart and innovative is that they call out that they provide feedback to journalists, letting them know whether their information is factually correct.


done_all 5a marked as Fully compliant by Steve Fox.

Criterion 5b
Claim submissions
Evidence required: Please link to the page or process through which readers can submit claims to fact-check. If you do not allow this, please briefly explain why.

Science Feedback
25-Mar-2017 (2 years ago)

Readers can submit suggestions of articles or claims to review on our ‘Process’ page: http://climatefeedback.org/process/

Steve Fox Assessor
25-Mar-2017 (2 years ago)

But, the editors run into another issue with interactivity when they send the message to readers that their ideas aren’t really what the editors are considering.

This section perpetuates the image of the expert academic: “If you wish to submit a suggestion of an article or claim to review, please use this online form. Please note that we cannot review every article or claim we receive. We focus on reviewing claims and articles that are scientifically verifiable and that reach large audiences.”

While the editors absolutely send a message of what will and won’t be fact-checked, the tone is not one that is welcoming to readers and I would recommend a revision in how this is phrased.


done 5b marked as Partially compliant by Steve Fox.

Section 6: Open & Honest Corrections Policy

Criterion 6a
Corrections policy
Evidence required: Please link to the page with your policy to address corrections. If it is not public, please share your organization's handbook.

Science Feedback
25-Mar-2017 (2 years ago)

You can find our correction policy here: http://climatefeedback.org/process/

Steve Fox Assessor
25-Mar-2017 (2 years ago)

Missing


done 6a marked as Partially compliant by Steve Fox.

Criterion 6b
Examples of corrections
Evidence required: Please provide two examples of a correction made, or correction requests handled, in the past year.

Science Feedback
25-Mar-2017 (2 years ago)

We have handled no correction requests, nor needed to make a correction during our tenure as of today.

Steve Fox Assessor
25-Mar-2017 (2 years ago)

The editors submitted this when responding to this question: “We have handled no correction requests, nor needed to make a correction during our tenure as of today.”

Perfection when publishing is near impossible. While there may indeed be a lack of corrections on the site, my guess is that it has more to do with the overall lack of interactivity on the site. The methodology and process of the work here is highly scientific but no one is perfect and I’m sure an error was made somewhere, it’s just that readers didn’t know they could point it out.

The Corrections section is located at the bottom of the Process page and reads as follows: “We aim for our “feedbacks” to be as accurate and up-to-date as possible. If we discover a mistake has been made, we will correct it as soon as possible and a note will be added on the original item. If you think we’ve made an error or missed some relevant information, contact us.”

That’s not engaging and gets lost and needs to be a lot more visible throughout the site.


done 6b marked as Partially compliant by Steve Fox.