Calculate Cohen's kappa κ = (Pₒₙˢ – Pᵉ) / (1 – Pᵉ) for inter-rater reliability between two categorical raters. κ corrects for chance agreement: κ < 0.20 poor, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, >0.80 almost perfect.
Enter the values for the scenario you are assessing. Calculate Cohen's kappa κ = (Pₒₙˢ – Pᵉ) / (1 – Pᵉ) for inter-rater reliability between two categorical raters. κ corrects for chance agreement: κ < 0.20 poor, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, >0.80 almost perfect. Use the cohen's kappa κ result to inform your calculation.