Prof.Β Marc Los Huertos Β β’Β Pomona College Β β’Β Environmental Analysis
dataRetrieval::readNWISdv() β no manual download required.
NA values and deriving year, month, and day-of-year
columns.
What this activity covers. Stream discharge integrates everything happening in a watershed β precipitation, snowmelt, groundwater, evapotranspiration, and land use β into a single measurable quantity at the gauge. The USGS has operated thousands of stream gauges across the United States continuously for decades, and all of that data is freely accessible through the
dataRetrievalpackage. This activity walks you through the complete pipeline: API call β cleaning β summary statistics β four standard visualisations. The only line you change to analyse a different river is the eight-digit site number.
# Run once to install (remove the # to uncomment, run, then re-comment):
# install.packages(c("dataRetrieval", "tidyverse", "lubridate",
# "scales", "knitr", "kableExtra"))
library(dataRetrieval) # official USGS R package β fetches NWIS data
library(tidyverse) # ggplot2, dplyr, tidyr
library(lubridate) # date arithmetic
library(scales) # comma-formatted and log axis labels
library(knitr) # kable() tables
library(kableExtra) # styled HTML tables
π What is dataRetrieval?
dataRetrieval is maintained by the USGS itself and provides
direct R access to the National Water Information System (NWIS) β the
same database behind
waterdata.usgs.gov.
Instead of navigating a web interface and downloading CSVs, you call one
function and the data lands directly in a tidy R data frame. The package
also has functions for water-quality data, rating curves, and site
metadata.
All user-defined parameters live in this one chunk. To analyse a different river, change only these values and re-knit. Everything else β titles, axis labels, table captions β updates automatically.
# ββ What each parameter means
βββββββββββββββββββββββββββββββββ
site_no Β Β # 8-digit USGS
site identifier
param_cd Β # 00060 = discharge (cfs) | 00065 = gage height
(ft)
stat_cd Β Β # 00003 = daily mean (the standard)
start_dt Β # YYYY-MM-DD β
at least 2 full years recommended
end_dt Β Β #
YYYY-MM-DD
# ββ Part 1: run as-is first βββββββββββββββββββββββββββββββββββββββββββββββββββ
# The San Gabriel River at Azusa is a local reference site with a long,
# continuous record. Confirm you get a complete output before changing anything.
site_no <- "11119750" # Ventura River near Ventura, CA
param_cd <- "00060" # discharge (cfs)
stat_cd <- "00003" # daily mean
start_dt <- "2020-01-01"
end_dt <- "2023-12-31"
# ββ Part 2 (assignment): replace with your chosen station βββββββββββββββββββββ
# See Section 7 for instructions on finding a station in any state.
π‘ Southern California default. The default site is the Ventura River near Ventura, CA β a flashy, Mediterranean-climate stream whose discharge is dominated by winter storm events and drops to near zero in summer. This makes it a useful contrast to the groundwater-sustained rivers of the eastern US. You should see a strongly seasonal signal in the monthly boxplot.
readNWISdv() sends an HTTP request to the USGS NWIS
daily-values API and returns a data frame.
renameNWISColumns() replaces the auto-generated column name
(e.g.Β X_00060_00003) with the readable name
Flow. readNWISsite() retrieves station
metadata β official name, coordinates, drainage area β so plot titles
are generated automatically.
raw <- readNWISdv(
siteNumbers = site_no,
parameterCd = param_cd,
startDate = start_dt,
endDate = end_dt
)
raw <- renameNWISColumns(raw)
meta <- readNWISsite(site_no)
cat("Rows downloaded:", nrow(raw), "\n")
## Rows downloaded: 1461
cat("Columns :", paste(names(raw), collapse = ", "), "\n")
## Columns : agency_cd, site_no, Date, Flow, Flow_cd
cat("Site name :", meta$station_nm, "\n")
## Site name : MISSION C NR MISSION ST NR SANTA BARBARA CA
cat("Drainage area :", meta$drain_area_va, "sq mi\n")
## Drainage area : 8.6 sq mi
β οΈ Network errors. readNWISdv()
requires an internet connection. On a campus network with a VPN or
firewall, the call occasionally times out even when the USGS server is
healthy. If you get a connection error, try again in a moment. If the
problem persists, use the CSV fallback in section 3.2 below.
If the API call fails, download the data manually from waterdata.usgs.gov and load it
here. Change eval=FALSE to eval=TRUE on this
chunk and eval=FALSE on the download chunk
above.
# raw_csv <- read_csv("data/usgs_discharge.csv") %>%
# mutate(time = ymd_hms(time)) %>%
# filter(parameter_code == "00060") %>%
# rename(Date = time, Flow = value) %>%
# filter(approval_status == "A")
#
# raw <- raw_csv %>% select(Date, Flow)
This chunk does four things:
Date and
Flow β all other NWIS metadata columns are dropped.Flow is
NA (equipment malfunctions and provisional data gaps appear
as NA).year,
month (JanβDec ordered factor), and doy (day
of year, 1β366).df <- raw %>%
select(Date, Flow) %>%
filter(!is.na(Flow)) %>%
mutate(
year = year(Date),
month = month(Date, label = TRUE, abbr = TRUE),
doy = yday(Date)
)
site_name <- meta$station_nm
units_lab <- "Discharge (cfs)"
cat("Records after cleaning:", nrow(df), "\n")
## Records after cleaning: 1461
cat("Date range :", as.character(min(df$Date)),
"to", as.character(max(df$Date)), "\n")
## Date range : 2020-01-01 to 2023-12-31
cat("Years covered :", paste(sort(unique(df$year)), collapse = ", "), "\n")
## Years covered : 2020, 2021, 2022, 2023
cat("Missing days removed :", nrow(raw) - nrow(df), "\n")
## Missing days removed : 0
π Why are there NA values in USGS data? USGS gauges
measure water-surface elevation (stage) with a pressure transducer or
float, then convert to discharge via a rating curve. NAs appear when the
sensor malfunctions, when the gauge is underwater during an extreme
flood (outside the rating curve range), or when the record is
provisional and awaiting quality review. NWIS marks approved data with
code A and provisional data with P β
renameNWISColumns() retains this as a cd
column you can filter on if needed.
stats <- df %>%
group_by(year) %>%
summarise(
N = n(),
Mean = round(mean(Flow), 1),
Median = round(median(Flow), 1),
SD = round(sd(Flow), 1),
Min = round(min(Flow), 1),
Max = round(max(Flow), 1),
.groups = "drop"
)
stats %>%
kbl(
caption = paste("Annual discharge summary \u2014", site_name),
col.names = c("Year", "N days", "Mean", "Median",
"SD", "Min", "Max"),
format.args = list(big.mark = ",")
) %>%
kable_styling(bootstrap_options = c("striped","hover"), full_width = FALSE) %>%
add_header_above(c(" " = 2, "Discharge (cfs)" = 5))
| Year | N days | Mean | Median | SD | Min | Max |
|---|---|---|---|---|---|---|
| 2,020 | 366 | 1.0 | 0 | 7.3 | 0 | 97.0 |
| 2,021 | 365 | 0.7 | 0 | 5.7 | 0 | 88.2 |
| 2,022 | 365 | 0.4 | 0 | 5.0 | 0 | 75.9 |
| 2,023 | 365 | 11.0 | 0 | 62.9 | 0 | 1,050.0 |
A linear y-axis gives the most intuitive first view of the record. Large flood peaks stand out clearly. For a Mediterranean-climate stream like the Ventura River, you will see near-zero discharge through summer and sharp, narrow peaks in winter.
ggplot(df, aes(x = Date, y = Flow)) +
geom_line(color = "#2e7d94", linewidth = 0.5) +
scale_x_date(date_labels = "%b %Y", date_breaks = "6 months") +
scale_y_continuous(labels = comma) +
labs(
title = paste("Daily Mean Discharge \u2014", site_name),
subtitle = paste("USGS site", site_no, "\u00b7 linear y-axis"),
x = NULL,
y = units_lab
) +
theme_minimal(base_size = 12) +
theme(
plot.title = element_text(family = "serif", face = "bold",
color = "#1e3a2f", size = 13),
plot.subtitle = element_text(color = "#7a7a72", size = 10),
axis.text.x = element_text(angle = 35, hjust = 1),
panel.grid.minor = element_blank()
)
Stream discharge is highly skewed: flood peaks can be 100β1,000 times
larger than base flows. A logββ y-axis compresses the peaks and
stretches the low-flow period so both extremes are readable
simultaneously. annotation_logticks() adds the
characteristic minor tick marks seen on logarithmic axes.
ggplot(df, aes(x = Date, y = Flow)) +
geom_line(color = "#2e7d94", linewidth = 0.5) +
scale_x_date(date_labels = "%b %Y", date_breaks = "6 months") +
scale_y_log10(labels = comma) +
annotation_logticks(sides = "l") +
labs(
title = paste("Daily Mean Discharge (log scale) \u2014", site_name),
subtitle = paste("USGS site", site_no, "\u00b7 log\u2081\u2080 y-axis"),
x = NULL,
y = "log\u2081\u2080 Discharge (cfs)"
) +
theme_minimal(base_size = 12) +
theme(
plot.title = element_text(family = "serif", face = "bold",
color = "#1e3a2f", size = 13),
plot.subtitle = element_text(color = "#7a7a72", size = 10),
axis.text.x = element_text(angle = 35, hjust = 1),
panel.grid.minor = element_blank()
)
π When to use log scale. Use linear scale when you want to show the magnitude of peaks (flood hazard, infrastructure design). Use log scale when you want to see low-flow variability (drought, ecological base flow, groundwater connectivity). For most stream analysis, showing both is standard.
Grouping by calendar month and pooling all years reveals the seasonal pattern of discharge, independent of year-to-year variability. Each box shows the interquartile range, the line is the median, and points beyond the whiskers are outliers.
ggplot(df, aes(x = month, y = Flow, fill = month)) +
geom_boxplot(
outlier.size = 0.9,
outlier.alpha = 0.4,
show.legend = FALSE
) +
scale_y_log10(labels = comma) +
annotation_logticks(sides = "l") +
scale_fill_manual(
values = colorRampPalette(c("#d4e6da", "#1e3a2f"))(12)
) +
labs(
title = paste("Monthly Discharge Distribution \u2014", site_name),
subtitle = "All years pooled \u00b7 log\u2081\u2080 y-axis",
x = NULL,
y = "log\u2081\u2080 Discharge (cfs)"
) +
theme_minimal(base_size = 12) +
theme(
plot.title = element_text(family = "serif", face = "bold",
color = "#1e3a2f", size = 13),
plot.subtitle = element_text(color = "#7a7a72", size = 10),
panel.grid.major.x = element_blank(),
panel.grid.minor = element_blank()
)
The flow duration curve (FDC) is one of the most widely used tools in hydrology. It answers: βWhat discharge is equalled or exceeded X% of the time?β The curve is constructed by ranking all daily values from highest to lowest and computing each valueβs exceedance probability.
fdc <- df %>%
arrange(desc(Flow)) %>%
mutate(exceed_pct = 100 * row_number() / n())
ggplot(fdc, aes(x = exceed_pct, y = Flow)) +
geom_line(color = "#2e7d94", linewidth = 0.9) +
scale_y_log10(labels = comma) +
annotation_logticks(sides = "l") +
scale_x_continuous(
breaks = seq(0, 100, 10),
labels = paste0(seq(0, 100, 10), "%")
) +
labs(
title = paste("Flow Duration Curve \u2014", site_name),
subtitle = paste("USGS site", site_no),
x = "Percent of time flow equalled or exceeded",
y = "log\u2081\u2080 Discharge (cfs)"
) +
theme_minimal(base_size = 12) +
theme(
plot.title = element_text(family = "serif", face = "bold",
color = "#1e3a2f", size = 13),
plot.subtitle = element_text(color = "#7a7a72", size = 10),
panel.grid.minor = element_blank()
)
# Extract key exceedance quantiles
q_vals <- quantile(df$Flow, probs = c(0.90, 0.50, 0.10),
na.rm = TRUE)
cat(sprintf("Q90 (exceeded 90%% of days): %7.1f cfs\n", q_vals["10%"]))
## Q90 (exceeded 90% of days): 0.0 cfs
cat(sprintf("Q50 / median discharge: %7.1f cfs\n", q_vals["50%"]))
## Q50 / median discharge: 0.0 cfs
cat(sprintf("Q10 (exceeded 10%% of days): %7.1f cfs\n", q_vals["90%"]))
## Q10 (exceeded 10% of days): 1.9 cfs
cat(sprintf("Flashiness ratio (Q10/Q90): %7.1f\n",
q_vals["90%"] / q_vals["10%"]))
## Flashiness ratio (Q10/Q90): Inf
βοΈ The flashiness ratio. Q10 / Q90 is a simple index of stream flashiness β the ratio of a high-flow reference to a low-flow reference. A ratio of 2 indicates modest variability (stable, groundwater-fed stream). A ratio of 100 or more indicates extreme flashiness (ephemeral or intermittent stream). Mediterranean-climate streams in Southern California commonly exceed 1,000.
Knit this document as provided and confirm you get a complete HTML output with all four plots and the summary table. The Ventura River example has a continuous record and is a reliable test case. Fix any errors before proceeding.
Option A β USGS WaterWatch map: Go to waterwatch.usgs.gov, zoom to your state, click a gauge dot, and note the 8-digit site number from the popup.
Option B β browse stations in R:
# Run this in the Console to browse sites in any state.
# Replace "CA" with your two-letter state abbreviation.
my_sites <- whatNWISsites(
stateCd = "CA",
parameterCd = "00060",
service = "dv"
)
View(my_sites) # opens a spreadsheet-style viewer in RStudio
Choose a site with:
A) daily recordsReturn to Section 2 and replace the three values in the settings chunk:
site_no <- "YOUR_8_DIGIT_SITE_NUMBER"
start_dt <- "YYYY-MM-DD"
end_dt <- "YYYY-MM-DD"
Re-knit. All titles, labels, and the metadata line update automatically.
Add a new section below this one in your .Rmd and
address these questions:
NA values did you remove during cleaning? What
might cause data gaps at a USGS gauge?whatNWISdata() to find
whether your site also has water-quality records (e.g., parameter 00010
= water temperature). If so, download a year of temperature data and add
a fifth plot showing temperature alongside discharge.readNWISdv() gives you programmatic access
to the entire USGS daily discharge archive. The eight-digit site number
is all you need to pull any stationβs record.
Hirsch, R. M., and De Cicco, L. A. 2015. User Guide to Exploration and Graphics for RivEr Trends (EGRET) and dataRetrieval. USGS Techniques and Methods book 4, chapter A10.
USGS. 2019. National Water Information System: Web Interface. U.S. Geological Survey. https://waterdata.usgs.gov/nwis