What is weighted kappa used for?

Weighted kappa (Cohen, 1968) accounts for the severity of disagreements in ordinal ratings. A difference of 1 point on a 5-point scale is penalised less than a difference of 4 points. Linear and quadratic weighting schemes are most common; quadratic weights align with Pearson r for continuous data.

What is Gwet's AC1 and when should it replace kappa?

Gwet's AC1 is robust to prevalence and marginal distribution effects that cause the kappa paradox (high observed agreement with low kappa when one category is very rare or common). AC1 is preferred when category prevalence is extreme (>90% or <10% in any category) or when raters have markedly asymmetric marginals.

Cohen's Kappa (κ) Inter-Rater Reliability

How It Works

Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects

Note: Interpret the cohen's kappa result against the clinical thresholds and context described above.

How to Use

Enter the values for the patient or scenario you are assessing. Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects. Use the cohen's kappa result to inform your clinical assessment.

Glossary and Definitions

Cohen's kappa: A key concept in Cohen's Kappa: Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects.
Inter-rater agreement: A key concept in Cohen's Kappa: Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects.
Kappa statistic: A key concept in Cohen's Kappa: Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects.
Rater reliability: A key concept in Cohen's Kappa: Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects.
Categorical agreement: A key concept in Cohen's Kappa: Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects.
IRR: A key concept in Cohen's Kappa: Calculate Cohen's kappa κ = (P_obs – P_exp) / (1 – P_exp) for inter-rater agreement on categorical data. Corrects for chance agreement. Includes weighted kappa for ordinal ratings and Gwet's AC1 as an alternative robust to prevalence effects.

Cohen's Kappa (κ) Inter-Rater Reliability Calculator

How It Works

How to Use

Glossary and Definitions

Frequently Asked Questions