Fleiss' Kappa - Inter-annotator reliability

hasloe
Wed, 07 Jun 2023 07:35:12 +0000

Dear Ulya,

The Next and Finish buttons are disabled/enabled based on choices in the current and previous steps. There are many combinations possible of where tiers are and how they should be combined and when I just tried again it seems that for the Fleiss’ Kappa calculation not all possible combinations are correctly implemented. Resulting in either disabled buttons or errors in the final calculation step. This is something we have to fix.
You may have run into this problem but maybe there is another explanation. If you want you may send an example (set of) file(s), with a description of how you wish to compare, to me (han.sloetjes AT mpi.nl) so that I can confirm if the problem is due to a bug or to inconsistent settings.

-Han