30 Case Study: Palmer Penguins
Penguins is clean, and clean data tempt you to skip the steps the earlier chapters argue for. Fit the obvious model and it reports a respectable \(R^2\), four diagnostic plots with nothing alarming in them, and coefficients that read sensibly. Nevertheless, its residuals separate by sex, a variable the model never saw, and none of the four standard plots shows it. This chapter runs the whole sequence of an analysis on that easy dataset, from scaffold to deposited DOI. It labels every step with the chapter that taught it. It performs each step deliberately, including the ones that look unnecessary. The sequence is what you are practicing, so that it is automatic when the data are difficult and the stakes are real. It is also the worked model for the capstone project in Chapter 35.
Everything should be made as simple as possible, but not simpler.
Albert Einstein, attributed; a later paraphrase of On the Method of Theoretical Physics, Herbert Spencer Lecture (1933)
Reading time. About 35 minutes.
You’ll need. The chapters named in Table 30.1, above all Chapter 11 for the compendium, Chapter 21 for complete-case reasoning, Chapter 23 for computed prose, and Chapter 27 for tinytest. The code uses:
You’ll be able to.
- Map each step of a complete analysis to the chapter that taught it, and use the map as the checklist for your own capstone.
- Choose which rows a model needs, drop only those, and report the denominator as a computed number.
- Read a fitted model’s coefficients and diagnostics, and find structure that the four standard diagnostic plots do not show.
- Write executed tests on the data and the fitted model, and wire them into continuous integration and a deposit.
Outline.
If these answers come quickly, skip to the next chapter. The answers are in Section 30.11.
- What data integrity issues does the
palmerpenguins::penguinsdataset contain, and how should they be handled? - What is the appropriate directory structure for a Palmer Penguins analysis compendium, and how do you produce it?
- In a simple regression of
body_mass_gonflipper_length_mm, what is the first diagnostic plot you should produce before interpreting any coefficient?
30.1 The arc, step by step
The Palmer Penguins dataset (Horst et al., 2022) is the successor to iris. It has enough structure to exercise the full arc of an analysis, from wrangling through description and inference to reporting. It does so without the eugenics association that accompanies Fisher’s irises, which he analyzed in the Annals of Eugenics. The dataset is small enough that the analysis runs in seconds.
Table 30.1 is the arc. Each row is a step of this chapter, the chapter that taught it, and the check that tells you the step is done. The capstone project in Chapter 35 asks you to fill in the same table for the HF-HOME study.
| Step | What you do here | Taught in | Done when |
|---|---|---|---|
| Scaffold | Create the compendium: layout, lockfile, container | Section 11.1, Chapter 9, Chapter 10, Chapter 11 | The pillar audit of Section 11.7 passes |
| Clean | Map the missingness; drop only rows the model needs | Chapter 15, Chapter 21 | The analytic n is computed and explained |
| Describe | Table 1 by the design variable | Chapter 15, Chapter 21 | Missing values appear as a row, not silently |
| Visualize | Scatter plot by species, faceted by island | Chapter 20 | The plot motivates the model |
| Fit | Linear model of mass on flipper length and species | Chapter 26 | The model was chosen before the coefficients were seen |
| Diagnose | Standard plots, then residuals against omitted variables | Chapter 20 | Every pattern is explained or reported |
| Report | Methods and results with computed numbers | Chapter 23 | No number in the prose is typed |
| Test | Assertions on the data and the model’s statistics | Chapter 27 | The suite fails when a planted bug is introduced |
| CI (continuous integration) | Tests and render on every push | Section 27.9, Section 27.8 | A failing test turns the job red |
| Deposit | Tag, archive, DOI (digital object identifier), README | Section 7.5, Chapter 34 | The DOI resolves and the README rebuilds the paper |
palmerpenguins::penguins holds 344 observations of three penguin species collected on three islands of the Palmer Archipelago, Antarctica, from 2007 to 2009.
palmerpenguins::penguins.
| Variable | Type | Notes |
|---|---|---|
species |
factor, 3 levels | Adelie, Chinstrap, Gentoo; the design variable |
island |
factor, 3 levels | Biscoe, Dream, Torgersen; largely aliased with species |
bill_length_mm, bill_depth_mm
|
numeric | Bill morphology |
flipper_length_mm |
numeric | The predictor |
body_mass_g |
numeric | The outcome |
sex |
factor, 2 levels | Female, male; the variable with most missing values |
year |
integer | 2007, 2008, or 2009 |
Question. Penguins needs no imputation, has no join, and fits in memory a thousand times over. Why does the chapter scaffold a container, write tests, and set up CI for it?
Answer. The arc is the thing being practiced. A step skipped on easy data is a step not yet automatic on hard data. The steps also catch real problems here. The cleaning step decides the denominator, and the diagnostic step finds structure the model omits. Each step costs minutes. The cost of learning to skip them is paid on the first project where they matter.
30.2 Scaffold and clean
30.2.1 Scaffolding the compendium
Assemble the compendium by hand as Section 11.4 does, or automate it. One way to automate it is the in-house zzcollab framework (Section 11.9), shown here (display-only, because it creates a project and builds an image):
mkdir penguins-analysis
cd penguins-analysis
zzc tidyverse # scaffold the Five Pillars
make docker-build # build the image
make r # enter R inside the containerzzc tidyverse writes the Five Pillars (Chapter 11): Dockerfile, renv.lock, .Rprofile, source-code directories, and a README pointing at the data.1 make docker-build builds the image (slow on the first run; cached afterward), and make r starts an R session inside it. Whichever route you took, run the tool-neutral audit of Section 11.7 before writing analysis code. In a zzcollab project, make check-renv-no-fix adds a report on whether the code, DESCRIPTION, and renv.lock agree.
1 The tidyverse profile was formerly called analysis, and zzc analysis still works as a deprecated alias (as of the repository in October 2026).
30.2.2 Mapping the missingness
library(palmerpenguins)
library(naniar)
vis_miss(penguins)
sex, with two rows missing all four morphology measurements.
miss_var_summary(penguins)
#> # A tibble: 8 × 3
#> variable n_miss pct_miss
#> <chr> <int> <num>
#> 1 sex 11 3.20
#> 2 bill_length_mm 2 0.581
#> 3 bill_depth_mm 2 0.581
#> 4 flipper_length_mm 2 0.581
#> 5 body_mass_g 2 0.581
#> 6 species 0 0
#> 7 island 0 0
#> 8 year 0 0Missingness is mostly in sex (11 rows, 3.2%). Two rows are missing all four morphology measurements, plausibly because of a lost data sheet for each bird.
Ask a model to lay out the options for the missing sex values, not to choose one.
Context. I am analyzing
palmerpenguins::penguins(344 rows; columns and types fromstr(penguins)pasted below).sexis missing in 11 rows; two rows are missing every morphology measurement. My planned model islm(body_mass_g ~ flipper_length_mm + species).Constraints. Do not recommend imputing
sexfrom sex-specific distributions of other variables. Do not use the fraction missing as the criterion.Criteria. First say whether the planned model needs the rows with missing
sexhandled at all. Then, for a second model that addssex, compare complete-case analysis with multiple imputation, stating for each the condition under which the coefficient estimates are unbiased.
Failure mode. Models often answer a different question. They recommend handling sex although the planned model never uses it. Or they propose imputing sex ‘from sex-specific distributions’, which presupposes the sex it is trying to fill in. A second common error states that complete case is biased whenever data are missing at random. Check. The answer should drop no row for the planned model, and its condition for complete case should match Section 21.3. Fit the model with sex both ways (mice::mice() with the outcome in the imputation model) and compare the coefficients. A data dictionary is structure, not data, so it may go to any approved tool (Section 12.3).
30.2.3 Dropping only what the model needs
Complete-case analysis is defensible here. The reason is not that the missing fraction is small. Complete-case estimates of a regression coefficient are unbiased when missingness does not depend on the outcome once the model’s covariates are held fixed (Section 21.3). Here the missingness plausibly reflects field data collection rather than body mass. Moreover, the model below does not use sex at all. Drop only the rows the analysis needs, not every row with any gap, and record the choice:
The two rows missing every morphology measurement are dropped, and analyses use \(n = 342\). The 9 further rows missing only sex stay in, and Table 1 reports their sex as missing.
na.omit() drops rows the model does not need, and the model still fits. na.omit(penguins) and tidyr::drop_na(penguins) with no columns named remove every row with any missing value. 333 rows remain instead of 342. Nothing errors, and the coefficients shift only slightly on this dataset. However, the methods section’s denominator is now wrong. On a dataset where the incidental column is missing for a reason, the estimate moves too. Name the modeled columns in drop_na().
Question. A colleague argues that because only 3.2% of rows have a missing value, complete-case analysis is safe. What is wrong with the argument, and what would make complete case unsafe here?
Answer. The fraction missing governs precision, not bias. Complete-case coefficient estimates are unbiased when missingness does not depend on the outcome given the covariates, whatever the fraction. Suppose, however, that the heaviest birds were the ones whose mass went unrecorded because they could not be weighed. The estimates would then be biased, since missingness would depend on body mass itself.
30.3 Describe and visualize
30.3.1 Table 1
gtsummary (Sjoberg et al., 2021) produces a Table 1 from one call. Stratify by the design variable, and keep missing values as a visible row:
library(gtsummary)
penguins_clean |>
tbl_summary(
by = species,
include = c(bill_length_mm, flipper_length_mm, body_mass_g,
sex, island),
missing_text = "Missing"
) |>
add_overall() |>
as_kable()| Characteristic | Overall N = 342 | Adelie N = 151 | Chinstrap N = 68 | Gentoo N = 123 |
|---|---|---|---|---|
| bill_length_mm | 44.5 (39.2, 48.5) | 38.8 (36.7, 40.8) | 49.6 (46.3, 51.2) | 47.3 (45.3, 49.6) |
| flipper_length_mm | 197 (190, 213) | 190 (186, 195) | 196 (191, 201) | 216 (212, 221) |
| body_mass_g | 4,050 (3,550, 4,750) | 3,700 (3,350, 4,000) | 3,700 (3,475, 3,950) | 5,000 (4,700, 5,500) |
| sex | ||||
| female | 165 (50%) | 73 (50%) | 34 (50%) | 58 (49%) |
| male | 168 (50%) | 73 (50%) | 34 (50%) | 61 (51%) |
| Missing | 9 | 5 | 0 | 4 |
| island | ||||
| Biscoe | 167 (49%) | 44 (29%) | 0 (0%) | 123 (100%) |
| Dream | 124 (36%) | 56 (37%) | 68 (100%) | 0 (0%) |
| Torgersen | 51 (15%) | 51 (34%) | 0 (0%) | 0 (0%) |
The table carries no row p-values. Table 1 describes the sample, and inference belongs to the model fitted below. The island rows show the design. Adelies appear on all three islands, Chinstraps only on Dream, and Gentoos only on Biscoe. Species and island are therefore largely aliased. An island difference can be estimated only among Adelies, and a model with both terms cannot separate them for the other two species. Leaving island out of the model is a modeling choice. It is not a finding that island does not matter.
The in-house zztable1 package (Appendix C) offers a similar formula interface for a journal-styled table. However, as of this writing it leaves the missing-value row blank instead of reporting the count, which breaks the rule just stated. Use gtsummary until that is fixed.
30.3.2 Exploratory scatter plot
library(ggplot2)
theme_set(theme_minimal())
ggplot(penguins_clean,
aes(flipper_length_mm, body_mass_g, color = species)) +
geom_point(alpha = 0.7) +
scale_color_brewer(palette = "Dark2") +
labs(x = "Flipper length (mm)",
y = "Body mass (g)",
color = "Species") +
facet_wrap(~ island)
The plot shows three clusters by species, with flipper length and body mass strongly associated within each. Gentoos (Biscoe only) are the heaviest and longest-flippered. Adelies (all three islands) and Chinstraps (Dream only) overlap on mass and are distinguished by bill length instead. The structure suggests the model: species as a categorical predictor and flipper length as a continuous one.
30.3.3 Exercises
- Write a second Table 1 stratifying by year (2007, 2008, 2009) rather than species. What does the difference between the two tables tell you about the data collection design?
penguins_clean |>
dplyr::group_by(year) |>
dplyr::summarize(
n = dplyr::n(),
across(c(bill_length_mm, flipper_length_mm, body_mass_g),
\(x) round(mean(x), 1)),
.groups = "drop"
)
#> # A tibble: 3 × 5
#> year n bill_length_mm flipper_length_mm body_mass_g
#> <int> <int> <dbl> <dbl> <dbl>
#> 1 2007 109 43.7 197. 4124.
#> 2 2008 114 43.5 203. 4267.
#> 3 2009 119 44.5 203. 4210.
table(penguins_clean$year, penguins_clean$species)
#>
#> Adelie Chinstrap Gentoo
#> 2007 49 26 34
#> 2008 50 18 46
#> 2009 52 24 43The year-stratified table is unremarkable next to the species one. Mean body mass spans about 140 g across years. Across species it spans about 1,380 g. Mean flipper length moves 6 mm by year against 27 mm by species. The between-year differences are real but an order of magnitude smaller. A Table 1 stratified by year would present them to the reader as though they were the structure of the data.
The cross-tabulation says why they are small. Each species was sampled in every year in roughly stable proportions. Year is therefore close to a nuisance variable, and species is the design. A Table 1 that stratifies by the wrong variable looks reassuringly balanced and tells you little.
Question. A reviewer asks you to add island to the model ‘to adjust for location’. What does Table 30.3 tell you about that request?
Answer. That most of the adjustment cannot be estimated. All Chinstraps are on Dream and all Gentoos on Biscoe. For those species an island term and a species term are the same contrast. Only the Adelie data can separate them. Reply with the cross-tabulation, and offer an Adelie-only analysis of island if the question matters.
30.4 Fit and diagnose
30.4.1 The fitted model
fit <- lm(body_mass_g ~ flipper_length_mm + species,
data = penguins_clean)
broom::tidy(fit, conf.int = TRUE)
#> # A tibble: 4 × 7
#> term estimate std.error statistic p.value conf.low conf.high
#> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 (Intercept) -4031. 584. -6.90 2.55e-11 -5181. -2882.
#> 2 flipper_length_mm 40.7 3.07 13.3 1.40e-32 34.7 46.7
#> 3 speciesChinstrap -207. 57.7 -3.58 3.98e- 4 -320. -93.0
#> 4 speciesGentoo 267. 95.3 2.80 5.39e- 3 79.4 454.Every number in the paragraphs that follow is computed by the chunks above rather than typed in. This is the discipline of Chapter 23 applied to the book’s own case study.
The intercept is the predicted body mass of an Adelie penguin at a flipper length of zero. It is an extrapolation far outside the data and has no useful interpretation. It exists to anchor the line. The flipper_length_mm coefficient, 40.7 g per mm, is the within-species slope. That is, it is the expected difference in body mass between two penguins of the same species whose flipper lengths differ by one millimeter.
The species coefficients are where a reader most often misreads the table. They are differences from Adelie at the same flipper length. That is not the difference in average mass between the species. Gentoos average about 1,380 g heavier than Adelies. The speciesGentoo coefficient is 270 g, a small fraction of that. The reason is that Gentoos also have much longer flippers, and the model has already credited the flipper-length term with most of the difference. The adjusted coefficient answers ‘how much heavier is a Gentoo than an Adelie with the same flippers’. That is a narrower question than the one a reader assumes it answers.
The \(R^2\) of 0.78 is respectable. On a clean morphometric dataset, however, a respectable \(R^2\) is not evidence that the model is well specified. It says the predictors and the outcome are related. Whether the functional form is right, or a variable is missing, is a question for the diagnostics.
30.4.2 Diagnostic plots
lm diagnostic plots for the fitted model. Residuals versus fitted and scale-location test the mean and variance structure; the Q-Q plot tests the residual distribution; residuals versus leverage identifies influential observations.
Read the four panels in turn, and say what each shows rather than what it should show.
| Panel | What it shows for this fit | Conclusion |
|---|---|---|
| Residuals vs fitted | Two clusters (Adelie and Chinstrap left, Gentoo right) and a smoother with a shallow dip; a quadratic flipper term improves the fit (p = 0.00071) | Mild curvature; small next to the residual spread, worth a sensitivity analysis |
| Q-Q | Points on the reference line; three birds sit slightly above it in the upper tail; standardized residuals beyond 3: 1 | No meaningful departure from normality |
| Scale-location | Nearly flat; residual standard deviation (SD) is 408 g for Adelies, 298 g for Chinstraps, 370 g for Gentoos | Modest heteroscedasticity by species; robust standard errors would be a reasonable sensitivity check |
| Residuals vs leverage | Largest Cook’s distance 0.023, far below the conventional 0.5 | No single bird drives the fit |
The plots are not featureless, but the features are small. The discipline is the same on clinical data, where the features are rarely small. Check, name each pattern, and decide whether it changes the conclusion.
30.4.3 What the four plots cannot show
The standard plots look only at the variables in the model. A variable left out shows up only when you plot the residuals against it, and sex is the obvious candidate:
penguins_resid <- penguins_clean |>
dplyr::mutate(resid = residuals(fit)) |>
dplyr::filter(!is.na(sex))
ggplot(penguins_resid, aes(sex, resid, color = sex)) +
geom_hline(yintercept = 0, color = "grey60") +
geom_jitter(width = 0.15, alpha = 0.6) +
scale_color_brewer(palette = "Dark2", guide = "none") +
facet_wrap(~ species) +
labs(x = NULL, y = "Residual (g)")
Males sit on average 390 g above females at the same flipper length and species. The gap is large in Adelies and Gentoos and small in Chinstraps, which is itself a finding a report should mention. The four standard plots gave no hint of it. The residuals’ overall distribution is a symmetric mixture of two shifted groups, and it passes for normal. Whether to add sex depends on the question. To describe mass as a function of flipper length within species, the model stands, and the sex structure belongs in the report. To predict mass, add sex and handle its missing values as the LLM prompt above lays out.
Four clean diagnostic plots do not mean a well-specified model. They test the shape of the residuals against the fitted values, not whether a variable is missing. An omitted variable whose effect is roughly symmetric hides inside residuals that look normal and evenly spread. Plot the residuals against every candidate variable you left out.
30.4.4 The interaction model
Resist over-modeling. The main-effects model fits well. An interaction model, body_mass_g ~ flipper_length_mm * species, gives each species its own slope at the cost of two parameters. Its AIC (Akaike information criterion) is 7.1 lower. An AIC difference of \(\Delta\) gives the simpler model a relative likelihood of \(e^{-\Delta/2}\) against the larger one. Here that is about 0.029, which favors the larger model without settling the matter. Whether to report it depends on three things. The first is whether different slopes make biological sense (different body shapes make them plausible). The second is whether the goal is description or prediction. The third is the audience. The defensible choice is to report both, with the AIC comparison, and let the reader see the trade-off. Reporting the larger model because it has more p-values is the wrong reason.
30.4.5 What the model licenses you to say
The coefficient of 40.7 g per mm is an association. It is tempting to call it ‘the effect of flipper length on body mass’. However, the word ‘effect’ claims something the analysis cannot support. Figure 30.6 sketches the plausible causal structure. Species and sex each influence both flipper length and body mass. Both measurements also grow with the bird’s overall size, which is not measured.
flowchart TD SP["Species"] --> FL["Flipper length"] SP --> BM["Body mass"] SX["Sex"] --> FL SX --> BM SZ["Overall size<br/><i>unmeasured</i>"] --> FL SZ --> BM
A causal question needs a hypothetical experiment that would answer it, the target trial (Hernán & Robins, 2016). The target trial specifies who would be enrolled, what would be assigned, and what would be measured. No such trial exists for ‘assign a penguin a longer flipper’. The question is therefore descriptive, and the honest verbs are ‘is associated with’ and ‘predicts’. The opposite error matters as much. Sometimes a study’s goal is causal, as in an observational comparison of two treatments. Calling the estimate an association then does not make the analysis less causal. It only hides the assumptions the reader needs to judge it. Hernán’s argument against that euphemism (Hernán, 2018) is short and worth reading before your first observational analysis. Habit 11 (Section 1.6) states the rule: match the verb to the design.
30.4.6 Exercises
- Extend the analysis: fit the interaction model
body_mass_g ~ flipper_length_mm * species. Test the interaction, and report the AIC against the main-effects model.
main <- lm(body_mass_g ~ flipper_length_mm + species,
data = penguins_clean)
inter <- lm(body_mass_g ~ flipper_length_mm * species,
data = penguins_clean)
anova(main, inter)
#> Analysis of Variance Table
#>
#> Model 1: body_mass_g ~ flipper_length_mm + species
#> Model 2: body_mass_g ~ flipper_length_mm * species
#> Res.Df RSS Df Sum of Sq F Pr(>F)
#> 1 338 47666988
#> 2 336 46147424 2 1519564 5.532 0.004327 **
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
AIC(main, inter)
#> df AIC
#> main 5 5031.523
#> inter 7 5024.443The interaction asks whether the mass-per-millimeter slope differs by species. It does not ask whether the species differ in mass. The anova() F-test and the AIC comparison answer the same question two ways, and they should agree. Where they disagree, the difference is small enough that the choice is not doing any work. Report the comparison you pre-specified, not the one that came out better.
- Fit a multinomial regression of
specieson the four morphometric measurements, and report how often it recovers the observed species.
fit_mn <- nnet::multinom(
species ~ bill_length_mm + bill_depth_mm +
flipper_length_mm + body_mass_g,
data = penguins_clean)Every penguin is classified correctly, which the scatter plot already implied. The three species separate cleanly in morphometric space.
An accuracy of exactly 1 should make you suspicious rather than pleased. The number is resubstitution on the data the model was fitted to, with four continuous predictors and well-separated groups. It therefore measures how flexible the model is at least as much as how learnable the species are. Report it as what the exercise asked for, how often the model recovers the observed species. Never report it as predictive accuracy. That claim needs cross-validation or a held-out set, and would come back lower.
Question. The speciesGentoo coefficient is 270 g. A draft results paragraph says ‘Gentoo penguins were 270 g heavier than Adelies’. What is wrong, and how do you fix the sentence?
Answer. The coefficient is the difference at equal flipper length. The raw difference in mean mass is about 1,380 g. Either report the raw difference as a description, or say ‘at the same flipper length, Gentoos were 270 g heavier on average than Adelies’. The draft’s sentence also says ‘were heavier’, which is a description of these birds. It should not become ‘species increases mass’.
30.5 Test, report, and automate
30.5.1 Tests on the data and the model
The checks so far ran once, in this chapter. A compendium needs them to run every time the analysis does, so they belong in a test file. Following Habit 9, the tests below check statistics, not shapes. They check the denominator, the factor levels the model’s reference category depends on, and the slope itself, re-derived by an independent route. In a model with a common slope, the within-species slope equals the pooled within-species covariance of flipper length and mass divided by the pooled within-species variance of flipper length. It can therefore be computed without lm().
# inst/tinytest/test-penguins-model.R
penguins_clean <- tidyr::drop_na(palmerpenguins::penguins,
body_mass_g, flipper_length_mm)
fit <- lm(body_mass_g ~ flipper_length_mm + species,
data = penguins_clean)
expect_equal(nrow(penguins_clean), 342L,
info = "only the two rows missing a modeled variable dropped")
expect_equal(sum(is.na(penguins_clean$sex)), 9L,
info = "rows missing only sex are kept")
expect_identical(levels(penguins_clean$species),
c("Adelie", "Chinstrap", "Gentoo"),
info = "Adelie is the reference level")
expect_equal(nobs(fit), nrow(penguins_clean),
info = "lm() used every analytic row")
by_species <- split(penguins_clean, penguins_clean$species)
sxy <- sum(vapply(by_species, \(d) {
sum((d$flipper_length_mm - mean(d$flipper_length_mm)) *
(d$body_mass_g - mean(d$body_mass_g)))
}, numeric(1)))
sxx <- sum(vapply(by_species, \(d) {
sum((d$flipper_length_mm - mean(d$flipper_length_mm))^2)
}, numeric(1)))
expect_equal(unname(coef(fit)["flipper_length_mm"]), sxy / sxx,
info = "slope equals the pooled within-species estimate")
ci <- confint(fit)["flipper_length_mm", ]
expect_true(ci[1] > 0,
info = "the within-species association is positive")The expected counts (342 and 9) are written into the test on purpose. They are the denominator the methods section states, and the test exists to fail if a later change to the cleaning moves it. In a project the chunk is the file inst/tinytest/test-penguins-model.R. Here, the chunk below writes it to a temporary file and runs it as tinytest would.
library(tinytest)
test_file <- file.path(tempdir(), "test-penguins-model.R")
writeLines(knitr::knit_code$get("penguins-tests"), test_file)
run_test_file(test_file, verbose = 0)
#> All ok, 6 results (27ms)A test suite that has never failed has not been shown to work. Plant the bug from the cleaning Pitfall, dropping every incomplete row, and confirm that the suite catches it.
mutant_file <- file.path(tempdir(), "test-penguins-mutant.R")
mutant <- sub("tidyr::drop_na(palmerpenguins::penguins,",
"tidyr::drop_na(palmerpenguins::penguins)",
knitr::knit_code$get("penguins-tests"), fixed = TRUE)
mutant <- sub("^ +body_mass_g, flipper_length_mm\\)$", "", mutant)
writeLines(mutant, mutant_file)
mutant_results <- run_test_file(mutant_file, verbose = 0)
stopifnot(any(!as.logical(mutant_results)))
mutant_results
#> ----- FAILED[data]: test-penguins-mutant.R<7--8>
#> call| expect_equal(nrow(penguins_clean), 342, info = "only the two rows missing a modeled variable dropped")
#> diff| Expected '342', got '333'
#> info| only the two rows missing a modeled variable dropped
#> ----- FAILED[data]: test-penguins-mutant.R<9--10>
#> call| expect_equal(sum(is.na(penguins_clean$sex)), 9, info = "rows missing only sex are kept")
#> diff| Expected '9', got '0'
#> info| rows missing only sex are kept
#>
#> Showing 2 out of 6 results: 2 fails, 4 passes (3ms)The denominator and the kept-sex tests fail. The slope test passes, because the slope is recomputed from the same wrong data. That is why the suite needs both kinds of test.
30.5.2 The paper
The compendium’s manuscript, analysis/paper/paper.qmd, is short and contains no new analysis. Its setup chunk loads the data, applies the same drop_na(), and fits the same model. Its Methods section states the denominator with inline code ({r} nrow(penguins_clean), and the number dropped as a difference). Its Results section cross-references the scatter plot and a coefficient table produced by broom::tidy(). The full pattern, a Quarto document whose every number is computed, is the worked example of Section 23.4. The compendium layout that holds it is in Section 11.4. Rendering inside the container (display-only) reproduces the paper from raw data to typeset output:
make docker-render REPORT=analysis/paper/paper.qmdAsk for a draft results paragraph, and treat it as a draft.
Context. Below are the output of
broom::tidy(fit, conf.int = TRUE)andbroom::glance(fit)forlm(body_mass_g ~ flipper_length_mm + species)fitted topalmerpenguins(Adelie is the reference level), and the species means of body mass and flipper length.Constraints. Report estimates with 95% confidence intervals, not p-values alone. Use ‘is associated with’, never ‘effect’ or ‘increases’. Write every number as a Quarto inline expression (
{r} ...) referring tofit, not as a literal.Criteria. State that species coefficients are differences at equal flipper length, and give the raw species difference beside the adjusted one. Name Adelie as the reference.
Failure mode. Models report the adjusted species coefficient as the species difference, call the slope an effect, or type numbers that are rounded differently from the table. Check. Render the paragraph, change the data (drop the Gentoo rows), and render again. Every number must change. Then read the paragraph against the Check at the end of Section 30.4.
30.5.3 Continuous integration
CI (continuous integration) runs the tests and the render on every push, so that a change that moves the denominator turns a check red before anyone reads the paper. The workflow is the one in Section 27.8 with a render step added (display-only; Section 27.9 covers the rest):
# .github/workflows/check.yaml
name: check
on: [push, pull_request]
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: r-lib/actions/setup-r@v2
- uses: r-lib/actions/setup-renv@v2
- uses: quarto-dev/quarto-actions/setup@v2
- name: Run tests
run: |
Rscript -e 'res <- tinytest::run_test_dir("inst/tinytest")' \
-e 'quit(status = as.integer(tinytest::any_fail(res)))'
- name: Render paper
run: quarto render analysis/paper/paper.qmdsetup-renv restores the packages in renv.lock, so CI runs the versions the paper was written with. The quit() line is what turns a failing test into a failing job (Section 27.8).
30.5.4 Exercises
- Add the multinomial model of exercise 3 to the compendium as a second analysis script with its own test file, and confirm that the CI job passes, then fails when you plant a bug.
A correct result has three parts. The script (analysis/scripts/02-multinom.R) fits the model and saves the confusion table. The test file asserts a statistic, not a shape: for example, that the confusion table’s total equals nrow(penguins_clean) and that its diagonal proportion equals the value computed by mean(predict(fit_mn) == penguins_clean$species) in the test itself. The CI log shows the test file’s results line, and a commit that changes the formula (dropping bill_length_mm, say) makes the job red. If the job stays green after the planted change, the test checks something the change does not affect.
Question. In the planted-bug Verification, the slope test still passed. Why is that not a weakness of the slope test?
Answer. The slope test checks that lm() computed what it claims from the data it was given. It re-derives the estimate independently, which catches a wrong formula or a wrong model. It cannot catch wrong data, because both routes see the same rows. The denominator test catches wrong data. Each test covers a failure the other cannot.
30.6 Deposit
A tag marks the commit the paper was rendered from. An archive gives it a DOI (digital object identifier) that outlives the repository (Chapter 34). From the shell (display-only):
git tag -a v1.0 -m "Penguins case study, complete"
git push origin v1.0The Zenodo integration is enabled for the repository once, in the Zenodo account settings. With it enabled, publishing a GitHub release from the tag archives the repository at that commit and mints a DOI. The deposit is the repository at the tagged commit. Its README answers four questions:
| README section | Content |
|---|---|
| What | One paragraph on the question and the dataset |
| How to reproduce |
make docker-build, then make docker-render REPORT=analysis/paper/paper.qmd
|
| Where the data are | In the palmerpenguins package, at the version in renv.lock
|
| How to cite | The DOI and the authors |
A reader given the DOI can rebuild the paper in two commands.
Resolve the DOI in a browser. Confirm that it reaches the archived release, not the live repository, and that the archived files include renv.lock and the Dockerfile. Then clone the tagged commit into a new directory and run the two README commands. The rendered numbers should match the paper’s.
Question. Why archive on Zenodo when the repository is already public on GitHub?
Answer. Hosting is not archiving. A GitHub repository can be renamed, rewritten, or deleted, and its URL then points nowhere. A Zenodo record is preserved under a DOI that cites one fixed version, which is what a paper’s citation needs.
30.7 Rules of thumb
- Habit 1: the compendium is the unit. Scaffold before the first line of analysis code, and deposit the whole compendium, not the paper.
- Habit 9: test the statistics, not the shape. Assert the denominator and re-derive a key estimate; a column-exists test passes on a wrong answer.
- Habit 11: match the verb to the design. A cross-sectional coefficient is an association; say so, and say why.
30.8 Cumulative practice
- Cumulative (uses Chapter 10). Reproduce this chapter’s compendium from scratch: scaffold it, add the cleaning, model, paper, tests, and CI workflow, and render it inside the container. Confirm that every number in the rendered paper matches the numbers in this chapter, and that the test suite catches the planted bug from Section 30.5.
- On your own work (no key). Deposit a compendium of your own to Zenodo, and confirm that the DOI resolves to the archived release and that the README is the first thing a reader sees.
30.9 Cheat sheet
| Step | Command | Check |
|---|---|---|
| Scaffold |
zzc tidyverse, or the layout of Section 11.4
|
Pillar audit passes |
| Map missingness | naniar::vis_miss(df) |
Pattern explained |
| Drop for the model | tidyr::drop_na(df, y, x) |
nrow() matches the methods |
| Table 1 | gtsummary::tbl_summary(df, by = group) |
Missing shown as a row |
| Fit | lm(y ~ x + group, data = df) |
Reference level stated |
| Diagnose |
plot(fit); residuals against omitted variables |
Each pattern named |
| Test | tinytest::run_test_dir("inst/tinytest") |
A planted bug fails |
| Render | make docker-render REPORT=analysis/paper/paper.qmd |
Numbers change when data change |
| Tag | git tag -a v1.0 -m "..." |
git show v1.0 |
| Deposit | GitHub release with Zenodo enabled | DOI resolves |
30.10 Further reading
- (Horst et al., 2022), the R Journal paper introducing the dataset.
- (Hernán, 2018), on saying ‘causal’ when the question is causal.
- (Hernán & Robins, 2016), the target-trial framework for observational questions.
30.11 Prerequisites answers
-
palmerpenguins::penguinscontains 11 rows with at least one missing value (most commonlysex). First document the missing pattern (naniar::vis_miss()), then choose a strategy before analysis. Drop only rows missing a variable the model uses. Complete case is then defensible when missingness plausibly does not depend on the outcome given the covariates, as the chapter argues is the case here. That is the condition. It is not MCAR (missing completely at random), and it is not a percentage threshold. Multiple imputation is preferable when missingness may depend on the outcome or when the dropped rows carry precision worth recovering. - A research compendium (Chapter 11): the Five Pillars, assembled by hand from the
rrtoolslayout,renv, and the canonical Dockerfile, or automated, for example withmkdir penguins && cd penguins && zzc tidyverse. The analysis code goes inanalysis/scripts/, the manuscript inanalysis/paper/, and the tests ininst/tinytest/. - A residuals-vs-fitted plot. It reveals non-linearity, non-constant variance, and systematic lack of fit in a single view. Read it, and the other three diagnostic plots, for what they actually show. Then plot the residuals against variables the model omits. Only then interpret the regression coefficient as the association between flipper length and body mass.