Cohen's Kappa separated annotations

valerie
Sun, 16 Feb 2025 05:29:42 +0000

Hi Divya,

Thank you for the clarification.
Since the above scenario is inapplicable, I would like to suggest another solution.
Referring to the attached image, Scenario A is the inapplicable solution you described. As for the other solution (scenario B), I was thinking of splitting “annot_A_and_B” into 2 separate tiers as indicated by the image so 1 tier contains “annot_A”, 1 tier contains “annot_B”. Then, running kappa analysis by manually selecting the 2 tiers into 1 so there is 1 kappa value computed per 2 tiers.
May I ask if this is possible?
If this message is unclear, please let me know if you need more clarification.

Thank you,
Valerie Fan