Prof. Marc Los Huertos • Pomona College
What this module covers. Tests for association answer a single class of question: are two variables related, or are they independent? The answer depends on the scale of measurement. This module develops three workhorse tests — the chi-square test of independence, the Pearson/Spearman correlation test, and Fisher’s exact test — each illustrated with an environmental dataset.
Two variables are associated (or dependent) when knowing the value of one changes the probability distribution of the other. They are independent when knowing one tells you nothing about the other.
Association ≠ causation. A statistically significant association is evidence that the two variables co-vary in the population; it is not, by itself, evidence that one causes the other.
| Variable X | Variable Y | Recommended test |
|---|---|---|
| Categorical (≥2 levels) | Categorical (≥2 levels) | Chi-square (χ²) |
| Categorical (2 levels, small n) | Categorical (2 levels) | Fisher’s exact |
| Continuous | Continuous (normal) | Pearson correlation |
| Continuous | Continuous (non-normal / ordinal) | Spearman correlation |
| Continuous predictor | Continuous outcome | Simple linear regression |
All three tests in this module share the same five-step logic from the hypothesis-testing framework: state H₀ and H₁ → choose α → compute a test statistic → find the p-value → decide and report.
Research question: Is the presence of Batrachochytrium dendrobatidis (Bd), the chytrid fungus driving global amphibian declines, associated with elevation zone in the Sierra Nevada?
Data: 240 swab samples from three elevation zones (low, mid, high) scored as Bd-positive or Bd-negative.
The test compares observed counts in each cell of a contingency table to expected counts — the counts we would predict if the null were true (i.e., if infection rate were uniform across elevations).
\[\chi^2 = \sum_{\text{all cells}} \frac{(O_i - E_i)^2}{E_i}\]
Expected count for cell (i, j): \(E_{ij} = \dfrac{(\text{row total}_i)(\text{col total}_j)}{n}\)
The statistic follows a χ² distribution with \((r-1)(c-1)\) degrees of freedom, where \(r\) = number of rows and \(c\) = number of columns.
⚠️ The expected-count rule
The χ² approximation is reliable only when every expected cell count is ≥ 5. If any expected cell is < 5, use Fisher’s exact test (see Example 3) or collapse categories if scientifically defensible.
set.seed(101)
# Simulate Bd survey data
elevation <- rep(c("Low", "Mid", "High"), times = c(80, 80, 80))
p_infect <- c(Low = 0.55, Mid = 0.35, High = 0.20)
infected <- rbinom(240, 1,
prob = p_infect[elevation]) == 1
bd <- data.frame(
elevation = factor(elevation, levels = c("Low", "Mid", "High")),
infected = factor(ifelse(infected, "Positive", "Negative"))
)
# Observed contingency table
obs_table <- table(bd$elevation, bd$infected)
addmargins(obs_table)
##
## Negative Positive Sum
## Low 42 38 80
## Mid 51 29 80
## High 61 19 80
## Sum 154 86 240
# Expected counts under H₀
chisq_result <- chisq.test(obs_table)
round(chisq_result$expected, 1)
##
## Negative Positive
## Low 51.3 28.7
## Mid 51.3 28.7
## High 51.3 28.7
chisq_result
##
## Pearson's Chi-squared test
##
## data: obs_table
## X-squared = 9.8218, df = 2, p-value = 0.007366
# Summarise infection rates per zone
bd_summary <- bd |>
group_by(elevation) |>
summarise(
n = n(),
pos = sum(infected == "Positive"),
rate = pos / n,
se = sqrt(rate * (1 - rate) / n),
lo = pmax(0, rate - 1.96 * se),
hi = pmin(1, rate + 1.96 * se),
.groups = "drop"
)
par(mar = c(4, 5, 3, 2), las = 1)
bp <- barplot(
bd_summary$rate,
names.arg = levels(bd_summary$elevation),
col = c("#c8832a", "#4a7c5e", "#1e3a2f"),
border = "white",
ylim = c(0, 0.8),
ylab = "Proportion Bd-positive",
main = expression(italic("Bd") ~ "infection rate by elevation zone"),
cex.main = 0.92
)
arrows(bp, bd_summary$lo, bp, bd_summary$hi,
angle = 90, code = 3, length = 0.06, col = "#3a3a38", lwd = 1.4)
abline(h = mean(bd_summary$rate), lty = 2, col = "#7a7a72", lwd = 1.2)
text(bp[3] + 0.5, mean(bd_summary$rate) + 0.03,
"Overall mean", col = "#7a7a72", cex = 0.76, adj = 0)
Bd infection rate by elevation zone. Error bars show 95% binomial confidence intervals. The declining gradient from low to high elevation is the pattern driving the significant chi-square result.
The χ² statistic summarises how much the observed counts deviate from what perfect independence would predict, scaled by the expected counts. A large χ² means the data are inconsistent with independence.
Effect size — Cramér’s V:
\[V = \sqrt{\frac{\chi^2}{n \cdot (\min(r,c) - 1)}}\]
Cramér’s V ranges from 0 (no association) to 1 (perfect association). Benchmarks: V ≈ 0.10 = small, V ≈ 0.30 = medium, V ≈ 0.50 = large.
# Cramér's V
n <- sum(obs_table)
k <- min(nrow(obs_table), ncol(obs_table))
V <- sqrt(chisq_result$statistic / (n * (k - 1)))
cat(sprintf("Cramér's V = %.3f\n", V))
## Cramér's V = 0.202
💡 Reporting a chi-square result
“Bd infection status was significantly associated with elevation zone, χ²(2) = , p = , Cramér’s V = ___. Infection rates declined from 48% at low elevation to 24% at high elevation.”
Always state: test statistic, degrees of freedom, p-value, effect size, and a plain-language description of the direction of the association.
Research question: Is water temperature associated with dissolved oxygen (DO) concentration across stream monitoring sites in southern California?
Data: 60 stream sites measured in late summer; DO is expected to decrease as temperature increases because warmer water holds less gas.
Pearson’s r measures the strength and direction of the linear association between two continuous variables. It assumes both variables are approximately normally distributed and that the relationship is linear.
\[r = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum(x_i-\bar{x})^2 \sum(y_i-\bar{y})^2}}\]
The test statistic is \(t = r\sqrt{(n-2)/(1-r^2)}\) on \(n-2\) degrees of freedom.
Spearman’s ρ (rho) is the Pearson correlation computed on the ranks of the data. It captures any monotone association (not just linear), is robust to outliers, and does not require normality — but it loses some power when normality holds.
Effect size benchmarks for |r| and |ρ|
Cohen’s conventions: |r| = 0.10 small · 0.30 medium · 0.50 large. In environmental science, r = 0.3 is often substantively meaningful.
set.seed(202)
n_sites <- 60
# Simulate temperature-DO data with a moderate negative correlation
temp_C <- round(rnorm(n_sites, mean = 18, sd = 3.5), 1)
do_mgl <- round(11.5 - 0.28 * temp_C + rnorm(n_sites, 0, 0.9), 2)
streams <- data.frame(temp_C, do_mgl)
head(streams, 4)
## temp_C do_mgl
## 1 14.0 8.50
## 2 16.5 7.71
## 3 16.8 6.87
## 4 15.0 5.66
# Check normality before choosing Pearson vs Spearman
shapiro.test(streams$temp_C)
##
## Shapiro-Wilk normality test
##
## data: streams$temp_C
## W = 0.98939, p-value = 0.8824
shapiro.test(streams$do_mgl)
##
## Shapiro-Wilk normality test
##
## data: streams$do_mgl
## W = 0.979, p-value = 0.3877
# Pearson correlation (normality roughly satisfied)
cor.test(streams$temp_C, streams$do_mgl, method = "pearson")
##
## Pearson's product-moment correlation
##
## data: streams$temp_C and streams$do_mgl
## t = -8.1273, df = 58, p-value = 3.749e-11
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## -0.8298586 -0.5839757
## sample estimates:
## cor
## -0.7296967
# Spearman correlation (non-parametric alternative)
cor.test(streams$temp_C, streams$do_mgl, method = "spearman")
##
## Spearman's rank correlation rho
##
## data: streams$temp_C and streams$do_mgl
## S = 62067, p-value = 5.97e-11
## alternative hypothesis: true rho is not equal to 0
## sample estimates:
## rho
## -0.7245706
par(mar = c(4.5, 4.5, 3, 2), las = 1)
plot(streams$temp_C, streams$do_mgl,
pch = 19, cex = 0.85,
col = adjustcolor("#1e3a2f", alpha.f = 0.65),
xlab = "Water temperature (°C)",
ylab = "Dissolved oxygen (mg/L)",
main = "Temperature vs. dissolved oxygen — summer low-flow survey",
cex.main = 0.92)
# Add regression line + CI band
fit <- lm(do_mgl ~ temp_C, data = streams)
xseq <- seq(min(streams$temp_C), max(streams$temp_C), length.out = 200)
pred <- predict(fit, newdata = data.frame(temp_C = xseq),
interval = "confidence")
polygon(c(xseq, rev(xseq)),
c(pred[, "lwr"], rev(pred[, "upr"])),
col = adjustcolor("#d4e6da", alpha.f = 0.55), border = NA)
lines(xseq, pred[, "fit"], col = "#c8832a", lwd = 2)
r_val <- cor(streams$temp_C, streams$do_mgl)
legend("topright",
legend = c(sprintf("r = %.3f", r_val), "OLS fit ± 95% CI"),
lty = c(NA, 1), pch = c(19, NA),
col = c(adjustcolor("#1e3a2f", .65), "#c8832a"),
lwd = c(NA, 2), bty = "n", cex = 0.82)
Dissolved oxygen vs. water temperature at 60 southern California stream sites. The fitted line is from simple OLS regression (slope shown for context); the shaded band is a 95% confidence interval around the mean. The correlation captures the strength of this negative linear trend.
Pearson’s r is sensitive to outliers and non-linearity. Always plot the data before running the test. A scatterplot that bends or fans out signals that Pearson’s r will be misleading.
par(mar = c(4, 4.5, 2.5, 1), las = 1)
plot(fitted(fit), residuals(fit),
pch = 19, cex = 0.8,
col = adjustcolor("#1e3a2f", alpha.f = 0.6),
xlab = "Fitted values", ylab = "Residuals",
main = "Residuals vs fitted — temperature / DO model",
cex.main = 0.88)
abline(h = 0, lty = 2, col = "#c8832a", lwd = 1.5)
Residuals vs fitted values for the temperature–DO regression. No systematic curve or fan-shaped spread is visible, supporting the use of Pearson correlation and linear regression for these data.
💡 Reporting a correlation result
“Water temperature was negatively correlated with dissolved oxygen across the 60 monitoring sites, Pearson’s r(58) = ___, p < 0.001. The association was strong in magnitude (r ≈ –0.8), consistent with the physical relationship between water temperature and gas solubility.”
Report: r (or ρ), degrees of freedom, p-value, and the direction plus magnitude of the association. Note whether you used Pearson or Spearman and why.
Research question: Does riparian buffer status (intact vs. degraded) predict the presence of Oncorhynchus mykiss (steelhead trout) in 18 small tributary reaches in Ventura County?
Data: A 2 × 2 contingency table from a rapid bioassessment survey. Sample size is small (n = 18), so several expected cell counts fall below 5 — the chi-square approximation is unreliable and Fisher’s exact test is appropriate.
Chi-square relies on a large-sample approximation to the χ² distribution. When counts are small, that approximation breaks down. Fisher’s exact test computes the exact probability of the observed table (and all tables more extreme) under the null of independence, using the hypergeometric distribution — no approximation required.
\[p = \frac{\binom{a+b}{a}\binom{c+d}{c}}{\binom{n}{a+c}}\]
where a, b, c, d are the four cell counts and marginals are treated as fixed.
# 2x2 contingency table: buffer × steelhead presence
# Steelhead present Steelhead absent
# Intact buffer 8 2
# Degraded buffer 2 6
reach_table <- matrix(
c(8, 2,
2, 6),
nrow = 2, byrow = TRUE,
dimnames = list(
Buffer = c("Intact", "Degraded"),
Steelhead = c("Present", "Absent")
)
)
reach_table
## Steelhead
## Buffer Present Absent
## Intact 8 2
## Degraded 2 6
# Show why chi-square fails here
cat("Expected counts under H₀:\n")
## Expected counts under H₀:
round(chisq.test(reach_table, correct = FALSE)$expected, 2)
## Steelhead
## Buffer Present Absent
## Intact 5.56 4.44
## Degraded 4.44 3.56
# Fisher's exact test (two-sided)
fisher.test(reach_table, alternative = "two.sided")
##
## Fisher's Exact Test for Count Data
##
## data: reach_table
## p-value = 0.05361
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
## 0.9232444 188.4940220
## sample estimates:
## odds ratio
## 10.0407
rates <- c(
Intact = reach_table["Intact", "Present"] / rowSums(reach_table)["Intact"],
Degraded = reach_table["Degraded", "Present"] / rowSums(reach_table)["Degraded"]
)
par(mar = c(4, 5, 3, 2), las = 1)
bp2 <- barplot(
rates,
col = c("#1e3a2f", "#c8832a"),
border = "white",
ylim = c(0, 1),
ylab = "Proportion of reaches with steelhead",
main = expression(italic("O. mykiss") ~ "presence by riparian buffer condition"),
cex.main = 0.92,
names.arg = c("Intact buffer\n(n = 10)", "Degraded buffer\n(n = 8)")
)
# Add sample-size–aware CI using Wilson method
n_row <- rowSums(reach_table)
p_hat <- rates
z <- 1.96
lo_w <- (p_hat + z^2 / (2 * n_row) -
z * sqrt(p_hat * (1 - p_hat) / n_row + z^2 / (4 * n_row^2))) /
(1 + z^2 / n_row)
hi_w <- (p_hat + z^2 / (2 * n_row) +
z * sqrt(p_hat * (1 - p_hat) / n_row + z^2 / (4 * n_row^2))) /
(1 + z^2 / n_row)
arrows(bp2, lo_w, bp2, hi_w,
angle = 90, code = 3, length = 0.07, col = "#3a3a38", lwd = 1.5)
Steelhead presence rates by riparian buffer condition (n = 18 reaches). Fisher’s exact test evaluates whether the observed difference in these small-sample proportions is consistent with chance.
Fisher’s exact test in R reports the odds ratio (OR) and its confidence interval directly. The OR is the natural effect-size measure for 2 × 2 tables:
\[\text{OR} = \frac{a/b}{c/d} = \frac{ad}{bc}\]
An OR of 1 means equal odds; OR > 1 means the first group has higher odds of the outcome. Unlike a risk ratio, the OR is symmetric and invariant to which variable is treated as the “response.”
# Manual odds ratio for clarity
or <- (reach_table[1,1] * reach_table[2,2]) /
(reach_table[1,2] * reach_table[2,1])
cat(sprintf("Odds ratio = %.2f\n", or))
## Odds ratio = 12.00
cat("Intact-buffer reaches have", round(or, 1),
"times the odds of steelhead presence vs. degraded reaches.\n")
## Intact-buffer reaches have 12 times the odds of steelhead presence vs. degraded reaches.
📌 When to use Fisher’s exact vs. chi-square
Use Fisher’s exact when any expected cell count <
5 (a strict rule) or when n < 20 overall (a common practical
threshold). For large tables (> 2×2) with small counts, use the
simulate.p.value = TRUE option in chisq.test()
to obtain a Monte Carlo p-value, or collapse cells where scientifically
justified.
The single most important decision is the scale of measurement:
| Both variables | Test | R function |
|---|---|---|
| Categorical (any size) | Chi-square | chisq.test() |
| Categorical (2×2, small n) | Fisher’s exact | fisher.test() |
| Continuous, both normal | Pearson correlation | cor.test(method = "pearson") |
| Continuous, non-normal/ordinal | Spearman correlation | cor.test(method = "spearman") |
| One continuous predictor + continuous outcome | Simple regression | lm() + summary() |
⚠️ Association ≠ causation
Every test in this module is symmetric — it answers only whether X and Y co-vary, not which causes which. Causal inference requires additional design elements: randomisation, temporal ordering, or a directed structural model. A significant χ²(2) or r does not license a causal claim.
| Assumption | How to check |
|---|---|
| Independent observations | Study design — each unit appears once |
| Expected cell counts ≥ 5 | Inspect chisq.test()$expected |
| Fixed marginal totals (Fisher) | Always satisfied for 2×2 tables |
| Assumption | How to check |
|---|---|
| Both variables continuous | Measurement scale |
| Bivariate normality (approximately) | Shapiro-Wilk on each variable; QQ plots |
| Linear relationship | Scatterplot — look for curves or bends |
| No severe outliers | Scatterplot; leverage diagnostics |
If normality is questionable or the relationship is monotone but not linear, use Spearman’s ρ. Both tests share the independence assumption: the n pairs must be independent of each other (no repeated measures, no spatial clustering).
par(mfrow = c(1, 2), mar = c(4, 4, 3, 1))
qqnorm(streams$temp_C, main = "QQ — Water temperature",
col = "#1e3a2f", pch = 19, cex = 0.75, cex.main = 0.88)
qqline(streams$temp_C, col = "#c8832a", lwd = 1.8)
qqnorm(streams$do_mgl, main = "QQ — Dissolved oxygen",
col = "#1e3a2f", pch = 19, cex = 0.75, cex.main = 0.88)
qqline(streams$do_mgl, col = "#c8832a", lwd = 1.8)
QQ plots for temperature (left) and dissolved oxygen (right). Points falling close to the diagonal support the normality assumption required for Pearson correlation.
par(mfrow = c(1, 1))
chisq.test()$expected before reporting. Collapse cells or
switch to Fisher’s exact when the rule is violated.
In a survey of 200 stream sites, 60 were classified as “impaired” and
140 as “healthy.” Of the 80 sites in agricultural watersheds, 40 were
impaired; of the 120 in forested watersheds, 20 were impaired. (a) Build
the 2×2 contingency table manually. (b) Run chisq.test().
(c) Compute Cramér’s V. (d) Write one sentence interpreting the result.
tab <- matrix(c(40, 40, 20, 100), nrow=2,
dimnames = list(c("Ag","Forest"), c("Impaired","Healthy")))
tab
chisq.test(tab)
# Cramer's V
chi2 <- chisq.test(tab)$statistic
n <- sum(tab)
V <- sqrt(chi2 / (n * (min(dim(tab))-1)))
cat("Cramer's V =", round(V, 3))
Agricultural watersheds had significantly higher impairment rates than forested ones (χ²(1) ≈ 25, p < 0.001, V ≈ 0.35 — medium effect size).
You have two groups (treated/control) and a binary outcome (species present/absent), but only 12 observations total with 2 expected cell counts below 5. Which test should you use instead of chi-square? Run it on simulated data.
tab_small <- matrix(c(1, 5, 4, 2), nrow=2)
fisher.test(tab_small)
With small expected cell counts, Fisher’s exact test is preferred because chi-square relies on a large-sample approximation that breaks down when n is small.