Tests for Association

Marc Los Huertos

EA030 · Compiled 2026-04-14

Overview Chi-Square Correlation Fisher’s Exact Choosing a Test Assumptions Takeaways

By the end of this module you will be able to
  1. Distinguish between tests for association and tests for group differences.
  2. Build a contingency table and run a chi-square test of independence.
  3. Decide when to use Fisher’s exact test instead of chi-square.
  4. Compute and interpret Cramér’s V as an effect size for categorical data.
  5. Select the appropriate association test given variable types and sample size.

What this module covers. Tests for association answer a single class of question: are two variables related, or are they independent? The answer depends on the scale of measurement. This module develops three workhorse tests — the chi-square test of independence, the Pearson/Spearman correlation test, and Fisher’s exact test — each illustrated with an environmental dataset.


1. What Is Association?

Two variables are associated (or dependent) when knowing the value of one changes the probability distribution of the other. They are independent when knowing one tells you nothing about the other.

Association ≠ causation. A statistically significant association is evidence that the two variables co-vary in the population; it is not, by itself, evidence that one causes the other.

A taxonomy of test choices

Variable X Variable Y Recommended test
Categorical (≥2 levels) Categorical (≥2 levels) Chi-square (χ²)
Categorical (2 levels, small n) Categorical (2 levels) Fisher’s exact
Continuous Continuous (normal) Pearson correlation
Continuous Continuous (non-normal / ordinal) Spearman correlation
Continuous predictor Continuous outcome Simple linear regression

All three tests in this module share the same five-step logic from the hypothesis-testing framework: state H₀ and H₁ → choose α → compute a test statistic → find the p-value → decide and report.


2. Example 1 — Chi-Square Test of Independence

Research question: Is the presence of Batrachochytrium dendrobatidis (Bd), the chytrid fungus driving global amphibian declines, associated with elevation zone in the Sierra Nevada?

Data: 240 swab samples from three elevation zones (low, mid, high) scored as Bd-positive or Bd-negative.

The hypotheses

H₀ — Null
Bd infection status and elevation zone are independent — the proportion infected is the same at all three elevations.
H₁ — Alternative
Bd infection status depends on elevation zone — at least one elevation has a different infection rate.

The chi-square statistic

The test compares observed counts in each cell of a contingency table to expected counts — the counts we would predict if the null were true (i.e., if infection rate were uniform across elevations).

\[\chi^2 = \sum_{\text{all cells}} \frac{(O_i - E_i)^2}{E_i}\]

Expected count for cell (i, j): \(E_{ij} = \dfrac{(\text{row total}_i)(\text{col total}_j)}{n}\)

The statistic follows a χ² distribution with \((r-1)(c-1)\) degrees of freedom, where \(r\) = number of rows and \(c\) = number of columns.

⚠️ The expected-count rule

The χ² approximation is reliable only when every expected cell count is ≥ 5. If any expected cell is < 5, use Fisher’s exact test (see Example 3) or collapse categories if scientifically defensible.

R walkthrough

set.seed(101)

# Simulate Bd survey data
elevation <- rep(c("Low", "Mid", "High"), times = c(80, 80, 80))
p_infect  <- c(Low = 0.55, Mid = 0.35, High = 0.20)
infected  <- rbinom(240, 1,
               prob = p_infect[elevation]) == 1

bd <- data.frame(
  elevation = factor(elevation, levels = c("Low", "Mid", "High")),
  infected  = factor(ifelse(infected, "Positive", "Negative"))
)

# Observed contingency table
obs_table <- table(bd$elevation, bd$infected)
addmargins(obs_table)
##       
##        Negative Positive Sum
##   Low        42       38  80
##   Mid        51       29  80
##   High       61       19  80
##   Sum       154       86 240
# Expected counts under H₀
chisq_result <- chisq.test(obs_table)
round(chisq_result$expected, 1)
##       
##        Negative Positive
##   Low      51.3     28.7
##   Mid      51.3     28.7
##   High     51.3     28.7
chisq_result
## 
##  Pearson's Chi-squared test
## 
## data:  obs_table
## X-squared = 9.8218, df = 2, p-value = 0.007366
# Summarise infection rates per zone
bd_summary <- bd |>
  group_by(elevation) |>
  summarise(
    n       = n(),
    pos     = sum(infected == "Positive"),
    rate    = pos / n,
    se      = sqrt(rate * (1 - rate) / n),
    lo      = pmax(0, rate - 1.96 * se),
    hi      = pmin(1, rate + 1.96 * se),
    .groups = "drop"
  )

par(mar = c(4, 5, 3, 2), las = 1)
bp <- barplot(
  bd_summary$rate,
  names.arg = levels(bd_summary$elevation),
  col    = c("#c8832a", "#4a7c5e", "#1e3a2f"),
  border = "white",
  ylim   = c(0, 0.8),
  ylab   = "Proportion Bd-positive",
  main   = expression(italic("Bd") ~ "infection rate by elevation zone"),
  cex.main = 0.92
)
arrows(bp, bd_summary$lo, bp, bd_summary$hi,
       angle = 90, code = 3, length = 0.06, col = "#3a3a38", lwd = 1.4)
abline(h = mean(bd_summary$rate), lty = 2, col = "#7a7a72", lwd = 1.2)
text(bp[3] + 0.5, mean(bd_summary$rate) + 0.03,
     "Overall mean", col = "#7a7a72", cex = 0.76, adj = 0)
Bd infection rate by elevation zone. Error bars show 95% binomial confidence intervals. The declining gradient from low to high elevation is the pattern driving the significant chi-square result.

Bd infection rate by elevation zone. Error bars show 95% binomial confidence intervals. The declining gradient from low to high elevation is the pattern driving the significant chi-square result.

Interpreting the output

The χ² statistic summarises how much the observed counts deviate from what perfect independence would predict, scaled by the expected counts. A large χ² means the data are inconsistent with independence.

Effect size — Cramér’s V:

\[V = \sqrt{\frac{\chi^2}{n \cdot (\min(r,c) - 1)}}\]

Cramér’s V ranges from 0 (no association) to 1 (perfect association). Benchmarks: V ≈ 0.10 = small, V ≈ 0.30 = medium, V ≈ 0.50 = large.

# Cramér's V
n  <- sum(obs_table)
k  <- min(nrow(obs_table), ncol(obs_table))
V  <- sqrt(chisq_result$statistic / (n * (k - 1)))
cat(sprintf("Cramér's V = %.3f\n", V))
## Cramér's V = 0.202

💡 Reporting a chi-square result

“Bd infection status was significantly associated with elevation zone, χ²(2) = , p = , Cramér’s V = ___. Infection rates declined from 48% at low elevation to 24% at high elevation.”

Always state: test statistic, degrees of freedom, p-value, effect size, and a plain-language description of the direction of the association.


3. Example 2 — Correlation Test

Research question: Is water temperature associated with dissolved oxygen (DO) concentration across stream monitoring sites in southern California?

Data: 60 stream sites measured in late summer; DO is expected to decrease as temperature increases because warmer water holds less gas.

The hypotheses

H₀ — Null
The population correlation coefficient ρ = 0: temperature and DO are linearly unrelated.
H₁ — Alternative
ρ ≠ 0: there is a linear association between temperature and DO (two-tailed; could also test ρ < 0).

Two correlation coefficients

Pearson’s r measures the strength and direction of the linear association between two continuous variables. It assumes both variables are approximately normally distributed and that the relationship is linear.

\[r = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum(x_i-\bar{x})^2 \sum(y_i-\bar{y})^2}}\]

The test statistic is \(t = r\sqrt{(n-2)/(1-r^2)}\) on \(n-2\) degrees of freedom.

Spearman’s ρ (rho) is the Pearson correlation computed on the ranks of the data. It captures any monotone association (not just linear), is robust to outliers, and does not require normality — but it loses some power when normality holds.

Effect size benchmarks for |r| and |ρ|

0
Small
0.10
Medium
0.30
Large
≥ 0.50

Cohen’s conventions: |r| = 0.10 small · 0.30 medium · 0.50 large. In environmental science, r = 0.3 is often substantively meaningful.

R walkthrough

set.seed(202)
n_sites <- 60

# Simulate temperature-DO data with a moderate negative correlation
temp_C  <- round(rnorm(n_sites, mean = 18, sd = 3.5), 1)
do_mgl  <- round(11.5 - 0.28 * temp_C + rnorm(n_sites, 0, 0.9), 2)

streams <- data.frame(temp_C, do_mgl)
head(streams, 4)
##   temp_C do_mgl
## 1   14.0   8.50
## 2   16.5   7.71
## 3   16.8   6.87
## 4   15.0   5.66
# Check normality before choosing Pearson vs Spearman
shapiro.test(streams$temp_C)
## 
##  Shapiro-Wilk normality test
## 
## data:  streams$temp_C
## W = 0.98939, p-value = 0.8824
shapiro.test(streams$do_mgl)
## 
##  Shapiro-Wilk normality test
## 
## data:  streams$do_mgl
## W = 0.979, p-value = 0.3877
# Pearson correlation (normality roughly satisfied)
cor.test(streams$temp_C, streams$do_mgl, method = "pearson")
## 
##  Pearson's product-moment correlation
## 
## data:  streams$temp_C and streams$do_mgl
## t = -8.1273, df = 58, p-value = 3.749e-11
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.8298586 -0.5839757
## sample estimates:
##        cor 
## -0.7296967
# Spearman correlation (non-parametric alternative)
cor.test(streams$temp_C, streams$do_mgl, method = "spearman")
## 
##  Spearman's rank correlation rho
## 
## data:  streams$temp_C and streams$do_mgl
## S = 62067, p-value = 5.97e-11
## alternative hypothesis: true rho is not equal to 0
## sample estimates:
##        rho 
## -0.7245706
par(mar = c(4.5, 4.5, 3, 2), las = 1)

plot(streams$temp_C, streams$do_mgl,
     pch  = 19, cex = 0.85,
     col  = adjustcolor("#1e3a2f", alpha.f = 0.65),
     xlab = "Water temperature (°C)",
     ylab = "Dissolved oxygen (mg/L)",
     main = "Temperature vs. dissolved oxygen — summer low-flow survey",
     cex.main = 0.92)

# Add regression line + CI band
fit  <- lm(do_mgl ~ temp_C, data = streams)
xseq <- seq(min(streams$temp_C), max(streams$temp_C), length.out = 200)
pred <- predict(fit, newdata = data.frame(temp_C = xseq),
                interval = "confidence")

polygon(c(xseq, rev(xseq)),
        c(pred[, "lwr"], rev(pred[, "upr"])),
        col = adjustcolor("#d4e6da", alpha.f = 0.55), border = NA)
lines(xseq, pred[, "fit"], col = "#c8832a", lwd = 2)

r_val <- cor(streams$temp_C, streams$do_mgl)
legend("topright",
       legend = c(sprintf("r = %.3f", r_val), "OLS fit ± 95% CI"),
       lty    = c(NA, 1), pch = c(19, NA),
       col    = c(adjustcolor("#1e3a2f", .65), "#c8832a"),
       lwd    = c(NA, 2), bty = "n", cex = 0.82)
Dissolved oxygen vs. water temperature at 60 southern California stream sites. The fitted line is from simple OLS regression (slope shown for context); the shaded band is a 95% confidence interval around the mean. The correlation captures the strength of this negative linear trend.

Dissolved oxygen vs. water temperature at 60 southern California stream sites. The fitted line is from simple OLS regression (slope shown for context); the shaded band is a 95% confidence interval around the mean. The correlation captures the strength of this negative linear trend.

Assumptions and diagnostics

Pearson’s r is sensitive to outliers and non-linearity. Always plot the data before running the test. A scatterplot that bends or fans out signals that Pearson’s r will be misleading.

par(mar = c(4, 4.5, 2.5, 1), las = 1)
plot(fitted(fit), residuals(fit),
     pch = 19, cex = 0.8,
     col = adjustcolor("#1e3a2f", alpha.f = 0.6),
     xlab = "Fitted values", ylab = "Residuals",
     main = "Residuals vs fitted — temperature / DO model",
     cex.main = 0.88)
abline(h = 0, lty = 2, col = "#c8832a", lwd = 1.5)
Residuals vs fitted values for the temperature–DO regression. No systematic curve or fan-shaped spread is visible, supporting the use of Pearson correlation and linear regression for these data.

Residuals vs fitted values for the temperature–DO regression. No systematic curve or fan-shaped spread is visible, supporting the use of Pearson correlation and linear regression for these data.

💡 Reporting a correlation result

“Water temperature was negatively correlated with dissolved oxygen across the 60 monitoring sites, Pearson’s r(58) = ___, p < 0.001. The association was strong in magnitude (r ≈ –0.8), consistent with the physical relationship between water temperature and gas solubility.”

Report: r (or ρ), degrees of freedom, p-value, and the direction plus magnitude of the association. Note whether you used Pearson or Spearman and why.


4. Example 3 — Fisher’s Exact Test

Research question: Does riparian buffer status (intact vs. degraded) predict the presence of Oncorhynchus mykiss (steelhead trout) in 18 small tributary reaches in Ventura County?

Data: A 2 × 2 contingency table from a rapid bioassessment survey. Sample size is small (n = 18), so several expected cell counts fall below 5 — the chi-square approximation is unreliable and Fisher’s exact test is appropriate.

Why Fisher’s exact test?

Chi-square relies on a large-sample approximation to the χ² distribution. When counts are small, that approximation breaks down. Fisher’s exact test computes the exact probability of the observed table (and all tables more extreme) under the null of independence, using the hypergeometric distribution — no approximation required.

\[p = \frac{\binom{a+b}{a}\binom{c+d}{c}}{\binom{n}{a+c}}\]

where a, b, c, d are the four cell counts and marginals are treated as fixed.

The hypotheses

H₀ — Null
Steelhead presence is independent of buffer condition — the odds ratio equals 1.
H₁ — Alternative
Steelhead presence is associated with buffer condition — intact buffers have different (higher) odds of trout presence.

R walkthrough

# 2x2 contingency table: buffer × steelhead presence
#                Steelhead present   Steelhead absent
# Intact buffer       8                  2
# Degraded buffer     2                  6

reach_table <- matrix(
  c(8, 2,
    2, 6),
  nrow = 2, byrow = TRUE,
  dimnames = list(
    Buffer   = c("Intact", "Degraded"),
    Steelhead = c("Present", "Absent")
  )
)
reach_table
##           Steelhead
## Buffer     Present Absent
##   Intact         8      2
##   Degraded       2      6
# Show why chi-square fails here
cat("Expected counts under H₀:\n")
## Expected counts under H₀:
round(chisq.test(reach_table, correct = FALSE)$expected, 2)
##           Steelhead
## Buffer     Present Absent
##   Intact      5.56   4.44
##   Degraded    4.44   3.56
# Fisher's exact test (two-sided)
fisher.test(reach_table, alternative = "two.sided")
## 
##  Fisher's Exact Test for Count Data
## 
## data:  reach_table
## p-value = 0.05361
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
##    0.9232444 188.4940220
## sample estimates:
## odds ratio 
##    10.0407
rates <- c(
  Intact   = reach_table["Intact",   "Present"] / rowSums(reach_table)["Intact"],
  Degraded = reach_table["Degraded", "Present"] / rowSums(reach_table)["Degraded"]
)

par(mar = c(4, 5, 3, 2), las = 1)
bp2 <- barplot(
  rates,
  col    = c("#1e3a2f", "#c8832a"),
  border = "white",
  ylim   = c(0, 1),
  ylab   = "Proportion of reaches with steelhead",
  main   = expression(italic("O. mykiss") ~ "presence by riparian buffer condition"),
  cex.main = 0.92,
  names.arg = c("Intact buffer\n(n = 10)", "Degraded buffer\n(n = 8)")
)

# Add sample-size–aware CI using Wilson method
n_row <- rowSums(reach_table)
p_hat <- rates
z     <- 1.96
lo_w  <- (p_hat + z^2 / (2 * n_row) -
            z * sqrt(p_hat * (1 - p_hat) / n_row + z^2 / (4 * n_row^2))) /
         (1 + z^2 / n_row)
hi_w  <- (p_hat + z^2 / (2 * n_row) +
            z * sqrt(p_hat * (1 - p_hat) / n_row + z^2 / (4 * n_row^2))) /
         (1 + z^2 / n_row)

arrows(bp2, lo_w, bp2, hi_w,
       angle = 90, code = 3, length = 0.07, col = "#3a3a38", lwd = 1.5)
Steelhead presence rates by riparian buffer condition (n = 18 reaches). Fisher's exact test evaluates whether the observed difference in these small-sample proportions is consistent with chance.

Steelhead presence rates by riparian buffer condition (n = 18 reaches). Fisher’s exact test evaluates whether the observed difference in these small-sample proportions is consistent with chance.

Effect size — odds ratio

Fisher’s exact test in R reports the odds ratio (OR) and its confidence interval directly. The OR is the natural effect-size measure for 2 × 2 tables:

\[\text{OR} = \frac{a/b}{c/d} = \frac{ad}{bc}\]

An OR of 1 means equal odds; OR > 1 means the first group has higher odds of the outcome. Unlike a risk ratio, the OR is symmetric and invariant to which variable is treated as the “response.”

# Manual odds ratio for clarity
or <- (reach_table[1,1] * reach_table[2,2]) /
      (reach_table[1,2] * reach_table[2,1])
cat(sprintf("Odds ratio = %.2f\n", or))
## Odds ratio = 12.00
cat("Intact-buffer reaches have", round(or, 1),
    "times the odds of steelhead presence vs. degraded reaches.\n")
## Intact-buffer reaches have 12 times the odds of steelhead presence vs. degraded reaches.

📌 When to use Fisher’s exact vs. chi-square

Use Fisher’s exact when any expected cell count < 5 (a strict rule) or when n < 20 overall (a common practical threshold). For large tables (> 2×2) with small counts, use the simulate.p.value = TRUE option in chisq.test() to obtain a Monte Carlo p-value, or collapse cells where scientifically justified.


5. Choosing the Right Test

The single most important decision is the scale of measurement:

Both variables Test R function
Categorical (any size) Chi-square chisq.test()
Categorical (2×2, small n) Fisher’s exact fisher.test()
Continuous, both normal Pearson correlation cor.test(method = "pearson")
Continuous, non-normal/ordinal Spearman correlation cor.test(method = "spearman")
One continuous predictor + continuous outcome Simple regression lm() + summary()

⚠️ Association ≠ causation

Every test in this module is symmetric — it answers only whether X and Y co-vary, not which causes which. Causal inference requires additional design elements: randomisation, temporal ordering, or a directed structural model. A significant χ²(2) or r does not license a causal claim.


6. Checking Assumptions

Chi-square and Fisher’s exact

Assumption How to check
Independent observations Study design — each unit appears once
Expected cell counts ≥ 5 Inspect chisq.test()$expected
Fixed marginal totals (Fisher) Always satisfied for 2×2 tables

Pearson correlation

Assumption How to check
Both variables continuous Measurement scale
Bivariate normality (approximately) Shapiro-Wilk on each variable; QQ plots
Linear relationship Scatterplot — look for curves or bends
No severe outliers Scatterplot; leverage diagnostics

If normality is questionable or the relationship is monotone but not linear, use Spearman’s ρ. Both tests share the independence assumption: the n pairs must be independent of each other (no repeated measures, no spatial clustering).

par(mfrow = c(1, 2), mar = c(4, 4, 3, 1))

qqnorm(streams$temp_C,  main = "QQ — Water temperature",
       col = "#1e3a2f", pch = 19, cex = 0.75, cex.main = 0.88)
qqline(streams$temp_C,  col = "#c8832a", lwd = 1.8)

qqnorm(streams$do_mgl,  main = "QQ — Dissolved oxygen",
       col = "#1e3a2f", pch = 19, cex = 0.75, cex.main = 0.88)
qqline(streams$do_mgl,  col = "#c8832a", lwd = 1.8)
QQ plots for temperature (left) and dissolved oxygen (right). Points falling close to the diagonal support the normality assumption required for Pearson correlation.

QQ plots for temperature (left) and dissolved oxygen (right). Points falling close to the diagonal support the normality assumption required for Pearson correlation.

par(mfrow = c(1, 1))

🔑 Tests for association — key takeaways

  • Match the test to the measurement scale. Categorical × categorical → chi-square or Fisher’s exact. Continuous × continuous → Pearson or Spearman. Using the wrong test inflates Type I error or loses power.
  • Chi-square requires expected counts ≥ 5. Check chisq.test()$expected before reporting. Collapse cells or switch to Fisher’s exact when the rule is violated.
  • Fisher’s exact test is exact. It is always valid for 2×2 tables regardless of sample size, and is the appropriate default when n is small. It is not limited to small samples — it is simply more conservative than chi-square for large n.
  • Pearson r assumes linearity and approximate normality. Always plot the data first. A curved scatterplot or heavy-tailed distribution calls for Spearman’s ρ or a transformation.
  • Report an effect size, not just a p-value. Cramér’s V for chi-square; OR for 2×2 tables; r or ρ for correlation. A tiny effect can produce p < 0.001 in a large sample — it is not necessarily meaningful.
  • Association is symmetric; causation is not. A significant test tells you the variables co-vary; it does not identify the direction of influence. Design and theory must carry the causal argument.

What’s next?

✏️ Practice Exercises
Exercise 1 — Chi-square test

In a survey of 200 stream sites, 60 were classified as “impaired” and 140 as “healthy.” Of the 80 sites in agricultural watersheds, 40 were impaired; of the 120 in forested watersheds, 20 were impaired. (a) Build the 2×2 contingency table manually. (b) Run chisq.test(). (c) Compute Cramér’s V. (d) Write one sentence interpreting the result.

Show solution
tab <- matrix(c(40, 40, 20, 100), nrow=2,
              dimnames = list(c("Ag","Forest"), c("Impaired","Healthy")))
tab
chisq.test(tab)
# Cramer's V
chi2 <- chisq.test(tab)$statistic
n    <- sum(tab)
V    <- sqrt(chi2 / (n * (min(dim(tab))-1)))
cat("Cramer's V =", round(V, 3))

Agricultural watersheds had significantly higher impairment rates than forested ones (χ²(1) ≈ 25, p < 0.001, V ≈ 0.35 — medium effect size).

Exercise 2 — Test selection

You have two groups (treated/control) and a binary outcome (species present/absent), but only 12 observations total with 2 expected cell counts below 5. Which test should you use instead of chi-square? Run it on simulated data.

Show solution
tab_small <- matrix(c(1, 5, 4, 2), nrow=2)
fisher.test(tab_small)

With small expected cell counts, Fisher’s exact test is preferred because chi-square relies on a large-sample approximation that breaks down when n is small.