Why does your data keep surprising you?
You run the Kruskal-Wallis test. It says something's happening. But your software just spits out a p-value and walks away. No names. No specific group comparisons. Just "there's a difference somewhere out there.
This isn't just annoying. It's what happens when you actually need to make decisions based on your data Most people skip this — try not to..
The short version is: Dunn's test is your go-to post-hoc test for Kruskal-Wallis. But here's what most guides miss — it's not just about running the test. It's about doing it right so you don't accidentally convince yourself of something that isn't true Simple, but easy to overlook. Took long enough..
What is Kruskal-Wallis and why do you need a post-hoc test?
Let's get real for a second. Here's the thing — kruskal-Wallis is the non-parametric cousin of ANOVA. Because of that, you reach for it when your data doesn't want to behave like normal distributions. Small samples. Also, skewed data. Ordinal scales. Whatever the reason, KW tells you that at least one group differs from the others.
But that's it. It's a gatekeeper. It opens the door but doesn't show you what's inside.
Think of it like a smoke detector. It alerts you to fire, but you still need to find the source. That's where post-hoc testing comes in.
The multiple comparisons problem
Here's where people trip up. If you have three groups (A, B, C) and KW says "something's up," you might be tempted to just compare A vs B, A vs C, and B vs C individually. But each comparison carries its own risk of Type I error — falsely finding a difference when there isn't one.
Short version: it depends. Long version — keep reading.
Do three comparisons at α = 0.Still, 05, and your actual family-wise error rate balloons to about 15%. Suddenly, you're claiming differences that don't exist.
That's why you need adjusted post-hoc tests. They keep your overall error rate in check while telling you which specific pairs differ.
Why Dunn's test is your answer (and why Bonferroni isn't)
Dunn's test isn't just commonly recommended — it's specifically designed for post-Kruskal-Wallis analysis. It uses the same rank sums from your original KW test, which makes it coherent with your initial findings.
Bonferroni correction? Too blunt. Worth adding: it adjusts all p-values equally, regardless of the actual test statistics. You lose power unnecessarily.
What Dunn's test actually does
Dunn's test compares all possible pairs of groups using the ranked data from your KW analysis. It calculates z-statistics for each pair and applies a correction factor — typically Bonferroni or Holm — to control for multiple comparisons.
The math isn't pretty, but you don't need to memorize it. What matters is understanding what it's protecting you from.
How to actually run Dunn's test (without your software lying to you)
Let's cut through the noise. Here's what you need to do, regardless of which statistical package you're using Took long enough..
Step 1: Get your ranked data from the KW test
This is crucial. Some software will let you run Dunn's test automatically after KW. Think about it: others require you to extract the ranks manually. Make sure you're using the same ranking system from your original analysis That alone is useful..
Step 2: Calculate pairwise comparisons
For each pair of groups, you're comparing their mean ranks. The test statistic is:
z = (R₁/n₁ - R₂/n₂) / sqrt[(N(N+1)/12)(1/n₁ + 1/n₂)]
Where R₁ and R₂ are the sum of ranks for each group, n₁ and n₂ are sample sizes, and N is total sample size Simple as that..
Don't panic at the formula. Consider this: your software handles this. But knowing what's under the hood helps you spot when something's wrong Small thing, real impact..
Step 3: Apply your correction
Bonferroni divides your alpha by the number of comparisons. So 05/3 ≈ 0. With 3 groups, that's 3 comparisons, so α becomes 0.0167.
Holm's method is less conservative. It ranks the p-values and applies successively smaller corrections. More power, same error control.
Step 4: Interpret with confidence intervals
Modern implementations give you confidence intervals for the differences in mean ranks. These tell you not just whether groups differ, but how much they differ Worth keeping that in mind. Simple as that..
Common mistakes that derail your analysis
I've seen researchers make these errors countless times. Don't be one of them.
Mistake #1: Using Tukey's HSD on ranks
Tukey's test assumes normality and equal variances. Which means throwing ranks at it is like putting diesel in a gasoline engine. It might run, but it's not running right.
Mistake #2: No correction for multiple comparisons
Running all pairwise comparisons without adjustment is statistical malpractice. That's why your Type I error rate explodes. You'll find "significant" differences that are just random noise Simple, but easy to overlook..
Mistake #3: Ignoring the continuity correction
With small samples, you need continuity corrections. Some software applies them automatically. Others don't. Check your output carefully.
Mistake #4: Post-hoc on different data than KW
Never run KW on one dataset and post-hoc on another. The ranks must come from the same analysis. Otherwise, you're comparing apples to oranges Simple, but easy to overlook..
What actually works in practice
Here's what I've learned from testing this across dozens of real datasets Not complicated — just consistent..
Use exact tests for small samples
When any group has fewer than 5 observations, consider exact versions of Dunn's test. The normal approximation breaks down.
Report effect sizes alongside p-values
Statistical significance isn't the whole story. On top of that, calculate rank-biserial correlations or Cliff's delta for each comparison. These tell you about practical significance.
Visualize your results
Plot the median ranks for each group with confidence intervals. A good visualization often reveals patterns that tables of p-values miss.
Consider step-down procedures
If you have many groups, consider closed testing procedures. They're more powerful than simple Bonferroni corrections while maintaining strong error control.
Handling tied ranks (because real data is messy)
Ties are everywhere in real data. Survey responses, Likert scales, categorical ratings — they all create tied ranks.
Dunn's test handles ties, but you need to account for them properly. The variance calculation includes a tie correction factor. Most modern software does this automatically, but check your documentation Not complicated — just consistent..
If ties are extensive (more than 25% of your data), consider alternative approaches. Sometimes a permutation test works better than asymptotic approximations.
Software-specific gotchas
R
Use the dunnTest() function from the FSA package. So it automatically applies Bonferroni correction. For Holm's method, use p.Now, adjust. method = "holm".
Python
Scipy doesn't have Dunn's test built-in. And posthoc_dunn(). Which means use the scikit-posthocspackage withsp. It defaults to Bonferroni correction Turns out it matters..
SPSS
The nonparametrics menu doesn't include Dunn's test directly. You'll need syntax or a custom calculation.
Excel
Don't even try. The math is too complex and error-prone. Use proper statistical software instead.
FAQ
Q: Can I use Dunn's test after Mood's median test?
A: No. Dunn's test is specifically designed for post-Kruskal-Wallis analysis. Mood's median test is a different procedure altogether Worth keeping that in mind..
Q: How many post-hoc comparisons should I run?
A: Run all pairwise comparisons. That's k(k-1)/2 comparisons for k groups. The correction handles the multiple testing problem.
Q: What if my KW test is significant but post-hoc isn't?
A: This happens when KW detects overall heterogeneity but no specific pairs differ significantly after correction. It suggests subtle patterns rather than clear group differences.
Q: Can I use Dunn's test for ordinal data?
A: Absolutely. In fact, that's exactly what it's designed for. Just make sure your software treats ties appropriately Not complicated — just consistent..
Q: Do I need to adjust for covariates?
A: Standard Dunn's test doesn't handle covariates. For that, you'd need more complex methods like stratified rank tests or permutation approaches Most people skip this — try not to..
The bigger picture
Here's what I want you to remember: post-hoc testing isn't just a formality. It's where your analysis earns its keep.
Dunn's test after Kruskal-Wallis gives you specific, interpretable
results that drive meaningful conclusions. Still, this specificity is crucial: without it, you might miss that Treatment A outperforms Control but not Treatment B, or that Group X differs from both Y and Z while Y and Z remain similar. That's why it transforms the vague signal of a significant Kruskal-Wallis test—indicating merely that differences exist somewhere among your groups—into precise knowledge of which specific groups differ and in what direction. Such nuances inform clinical decisions, policy changes, or further experimental focus in ways an omnibus test alone cannot.
Remember, the goal isn’t merely to clear a statistical hurdle but to uncover actionable patterns. Dunn’s test, when applied thoughtfully—respecting tie corrections, choosing an appropriate step-down method for your comparison volume, and verifying software implementation—delivers that clarity. Treat it not as a rote post-KW ritual, but as the essential step where your nonparametric analysis earns its practical value. Use it wisely, interpret it in context, and let the pairwise contrasts reveal the story your ranked data truly tells Took long enough..
