Applicant: VoxCheck PM
Assessor: Oleg Khomenok
Edits made by the organization after this assessment
IFCN Staff wrote:
Remarks and explanations to the External Assessment Sheet
VoxCheck did not publish articles over the period between 01/02 - 19/02 due to illness of its leader and project manager Olena Shkarpova, as well as the other VoxCheck employees. It was a harsh flu that did not allow us to work on a full potential. However, we did make daily monitorings for the Rating of Liars, released videos and fulfilled all our obligations within the projects that VoxCheck makes together with other media. For example, during this period we made a radio program on Hromadske Radio (see here, 14/02) and released fact-checks for the TV-show “Hra sliv” (see programs on 01/02, 07/02 and 15/02). During the next week after the mentioned period we published three articles (please, check here: one, two, three).
The Annual Report with the financial statement of VoxUkraine for the year 2016 can be found here. The report for the year 2017 should be prepared within the next few months.
The fact-checking methodology, correction policy and claim submission can be found here. We are ready to publish it on our web-site as soon as the IFCN and external assessor will approve it.
Regarding the Liars rating, in our opinion the methodology is accurate from the scientific and statistical point of view. Although the number of the statements for some politicians is relatively smaller than for others, in this case it is important that we take not some sample, but the "general population" of all statements of the specific politicians. So we believe that if we take all public fact-checkable statements, we do not have biases, which could occur if we only rated sample of the statements. The collection method for statements is consistent across time and across politicians, which also eliminates risks for other biases. So the number of statement, although varies a lot from politician to politician does not lead to biases in the evaluation. Of course some politicians may speak less overall, but we do not think it should impact their "lie index".
There can be a risk that some statements can be missed due to the fact that they are collected manually by people - even though we monitor all media appearances of the politicians. In any case, even if we randomly do not take some statements (which is doubtful, but we admit this can be the case), the randomness across population, applied to all politicians, ensures objectivity of the rating. This methodology was developed and approved by the Editorial Board of VoxUkraine as statistically meaningful and sensible.
Comments from the assessor
Oleg Khomenok: VoxCheck did great job updating the site in accordance to the comments in assessment sheet. I checked the methodology description on the Google Drive and to my mind it is well written and ready for publication on the VoxCheck web-site.
Now with all updates VoxCheck is fully compliant and can join IFCN
Conclusion and recommendations
Oleg Khomenok wrote:
I think Voxcheck is compliant with most principles but should make some updates on their web-site to be accepted.
There are several issues that should be addressed by Voxcheck to be compliant and accepted. They are listed in the comments below each section of evaluation together with recommendations. In short there are:
In Criterion 1 there is a need to provide the explanation of the absense of the stories on the site during Jan 30-Feb 20, 2018.
In Criterion 4a there is a need to publish publish financial statements for 2016, 2017 and translate them into to Ukrainian and Russian
In Criterion 5a there is a need to publish the fact-checking methodology on the web-site.
In Criterion 5b there is a need to create direct link that will enable reader in 2 clicks to submit a statement or link to be fact-checked with explanation of what is fact-checkable and what is not.
In Criterion 6 there is a need to develop and update correction policy in the rules®s manual as well publish it on the web-site.
The Lie index https://voxukraine.org/longreads/lie-theory/index-en.html https://voxukraine.org/longreads/lie-theory/index-ua.html in general raises a question of comparative index methodology since comparing 56 statements of Poroshenko with 32 of Vilkul for example and creating a rating chart in percentage looks irrelevant. Some of the politicians were rated on 10 statements, at the same time President of Ukraine was rated on 56 statements.
The Lie index is published under Voxcheck rubric of the site but to my mind should be separate product not directly associated with the Voxcheck since it has a lot of distortions in the comparison based on unequal initial numbers of statements. Notably that this story has switched off commenting option, I believe it would get a lot of criticizing comments.
For further updates of the site here is a good example of methodology description and correction policy placed on the page https://www.factcheck.org/our-process/
Finally I would recommend acceptance of organisation after updates on the web-site and clear explanation of the pause in publishing in February of 2018 will be provided and separating Lie index from the Voxcheck page.