The Chi-Square Test vs Fisher Exact Test: Which One Should You Actually Use?
Here's the thing — if you've ever run a statistical test on categorical data and gotten a warning about "expected cell counts," you've probably wondered whether you should switch to Fisher's exact test. Or maybe you're just starting out in stats and someone told you there are two ways to test the same thing, and you're not sure which one won't make your advisor side-eye you.
The short version is this: chi-square and Fisher exact tests both test whether two categorical variables are associated, but they work differently, have different assumptions, and one is usually better than the other depending on your data. Let's break down what actually matters Simple, but easy to overlook..
What These Tests Actually Do
Both the chi-square test and Fisher's exact test are used to determine if there's a statistically significant association between two categorical variables. You're working with contingency tables — think rows and columns of counts, like "smokers vs non-smokers" crossed with "lung cancer vs no lung cancer."
The chi-square test compares your observed counts to the counts you'd expect if there were no association between the variables. It asks: "How different is my actual data from what I'd expect under the null hypothesis?" Then it uses a chi-square distribution to calculate a p-value That's the part that actually makes a difference..
Fisher's exact test takes a different approach entirely. Instead of approximating, it calculates the exact probability of observing your data — or something more extreme — given the marginal totals (the row and column sums) are fixed. It's literally computing combinations and permutations to give you an exact answer.
Why the Choice Actually Matters
I know it sounds like splitting hairs, but here's what most people miss: using the wrong test can give you misleading results, especially with small sample sizes.
The chi-square test relies on a large-sample approximation. It assumes your sample size is big enough that the chi-square distribution accurately represents your data. When that assumption breaks down — typically with small samples or sparse tables — your p-values become unreliable. You might think you found something real when it's just noise, or vice versa.
Fisher's exact test doesn't have this problem because it doesn't rely on any approximation. It gives you the exact probability, no matter how small your sample is. But it comes with its own trade-offs, which we'll get to.
How Each Test Works Under the Hood
Chi-Square Test: The Approximation Approach
The chi-square test statistic is calculated as:
χ² = Σ [(Observed - Expected)² / Expected]
For each cell in your contingency table, you calculate how far your observed count is from what you'd expect under independence, square that difference, divide by the expected count, and sum across all cells.
The key assumption here is that this test statistic follows a chi-square distribution. This approximation works well when your expected cell counts are reasonably large — traditionally, we say at least 5 observations per cell. Modern simulation studies suggest it's actually fine with expected counts as low as 1, as long as no more than 20% of cells have expected counts below 5.
But here's the catch: if your expected counts are too small, the chi-square approximation breaks down, and your p-values become inaccurate.
Fisher's Exact Test: The Exact Approach
Fisher's exact test conditions on the marginal totals. It asks: "Given these fixed row and column totals, what's the probability of observing this particular arrangement of counts — or something more extreme?"
It uses the hypergeometric distribution to calculate this probability. For a 2x2 table, it's relatively straightforward. For larger tables, it gets computationally intensive because you have to enumerate all possible tables with the same marginals.
The advantage? On the flip side, no approximation needed. On top of that, the disadvantage? It can be overly conservative, meaning it might miss real effects because it's being extra cautious about false positives That's the whole idea..
Common Mistakes People Make
Using chi-square with small samples. This is the big one. I've seen researchers run a chi-square test on a 2x2 table with 15 total observations and report the results without hesitation. The expected cell counts were probably around 1-2, making the chi-square approximation questionable at best.
Ignoring the conservativeness of Fisher's exact test. Some people think Fisher's exact test is always "better" because it's "exact," but it can be overly conservative, especially with larger samples. This means you might fail to detect a real association because the test is being too cautious Took long enough..
Applying Fisher's exact test to large datasets unnecessarily. If you have 500 observations and all expected cell counts are above 5, Fisher's exact test will give you nearly identical results to chi-square but take much longer to compute. There's no benefit, and you're just wasting processing time Not complicated — just consistent. That alone is useful..
Misunderstanding what "more extreme" means. In Fisher's exact test, "more extreme" refers to tables that are equally or more inconsistent with the null hypothesis, given the fixed marginals. It's not always intuitive, especially with one-sided tests Simple as that..
Practical Guidelines: When to Use Which
Use Chi-Square When:
- Your total sample size is reasonably large (say, 40+ observations)
- All expected cell counts are at least 5 (or at most 20% are below 5, with none below 1)
- You're working with tables larger than 2x2 and computational speed matters
- You're comfortable with the approximation approach
Use Fisher's Exact Test When:
- You have a small sample size (under 40 observations)
- Any expected cell count is below 5
- You have a 2x2 table and want the most accurate p-value possible
- You're dealing with sparse data where many cells might have zero counts
- You need to be absolutely certain about avoiding false positives (like in clinical trials)
A Note on Large Tables
For tables larger than 2x2, Fisher's exact test becomes computationally demanding. Also, many statistical software packages will warn you or refuse to run it. In these cases, chi-square is usually your go-to, even with moderate sample sizes, as long as expected cell counts are adequate Surprisingly effective..
Some people use a hybrid approach: run chi-square first, check the expected cell counts, and only switch to Fisher's if the assumptions are violated. This is perfectly reasonable.
Real Talk About Software Implementation
Here's what's annoying: different software packages handle these tests differently. Also, r's fisher. test() function can handle tables larger than 2x2, but it might take forever with big datasets. Day to day, sPSS will automatically warn you when expected cell counts are too low. Python's scipy has both tests readily available Surprisingly effective..
The bottom line is that you need to know what your software is doing and check the assumptions yourself. Don't just trust the default output.
Frequently Asked Questions
Can I use both tests and report whichever gives me a better p-value? No. That's data dredging. Pick your test based on your sample size and expected cell counts before you look at the results, and stick with it But it adds up..
What if my expected cell counts are between 1 and 5? This is the gray area. Chi-square is probably fine, but Fisher's exact test won't hurt. If you're writing a paper, mention that you checked expected cell counts and chose accordingly.
Does Fisher's exact test work for tables larger than 2x2? Yes, but it can be very slow with large tables. Some software packages have limits on table size for computational reasons.
Should I use a one-sided or two-sided test? For both tests, the default is two-sided. Only use one-sided if you had a strong a priori reason to expect the association in a specific direction It's one of those things that adds up..
What about Yates' correction for continuity? Yates' correction was designed to make chi-square more conservative for 2x2 tables, but it's generally considered overly conservative now. Most statisticians recommend against it.
Making the Decision in Practice
Look, here's what actually works: before you run any test, make a simple table of your expected cell counts. If they're all above 5, chi-square is fine. If any are below 5, especially with small total samples, go with Fisher's exact test And it works..
For 2x2 tables with small samples, Fisher's exact test is almost always the safer choice. For larger tables or larger samples, chi-square is usually more practical and just as accurate Easy to understand, harder to ignore..
And honestly, if you're still unsure, run both. They'll usually
And honestly, if you're still unsure, run both. They'll usually agree when the assumptions are met, and if they disagree, that's a clear signal that something's off with your data or with your chosen significance level. In that case, dig deeper—look at cell frequencies, consider collapsing categories, or even collect more data if possible And it works..
Quick Reference Cheat Sheet
| Situation | Recommended Test | Why |
|---|---|---|
| 2×2 table, n < 30 or any expected cell < 5 | Fisher’s exact | Exact distribution, no asymptotic assumptions |
| Any table, all expected cells ≥ 5 | Chi‑square | Fast, asymptotically accurate |
| 2×2 table, n ≥ 30, all expected cells ≥ 5 | Chi‑square (with continuity correction optional) | Same as above; Yates is usually overkill |
| Very large table (e.g., 10×10) | Chi‑square | Fisher becomes computationally prohibitive |
| You want a p-value for a specific direction | One‑sided test (if justified) | Only if you have a strong a priori hypothesis |
Common Pitfalls to Avoid
- Assumption‑free “just run it” – Even the best software will give you a number, but if the assumptions are violated, that number is meaningless.
- Skipping the expected‑count check – A quick glance at the contingency table can save you a lot of headaches later.
- Reporting only the smallest p-value – That’s data‑dredging. Choose the test before you look at the outcome.
- Over‑reliance on Yates’ correction – It was designed for a different era of data and is rarely needed today.
When to Think Beyond Chi‑Square and Fisher
If your data violate the independence assumption (e.In real terms, , repeated measures or matched pairs), you might need a McNemar test or a conditional logistic regression. Consider this: if you have ordinal categories, consider the Mantel–Haenszel test or a Cochran–Armitage trend test. g.And if you’re dealing with sparse data across many categories, Bayesian approaches or exact multinomial tests can be worthwhile.
Not the most exciting part, but easily the most useful.
Bottom Line
- Check the counts first. If any expected cell is under 5, lean toward Fisher’s exact.
- Use chi‑square when counts are comfortably large. It’s efficient and reliable.
- Stay honest about your choices. Document the criteria you used to pick a test in your methods section.
- If in doubt, run both. If they concur, great. If not, investigate further.
In the end, the goal isn’t to win a p-value battle but to make a valid inference about the relationship between your categorical variables. Because of that, choose the test that aligns with your data’s structure, respect the assumptions, and report your decision transparently. That’s the recipe for dependable, reproducible analyses.
