30  Case Study: Palmer Penguins

Penguins is clean, and clean data tempt you to skip the steps the earlier chapters argue for. Fit the obvious model and it reports a respectable \(R^2\), four diagnostic plots with nothing alarming in them, and coefficients that read sensibly. Nevertheless, its residuals separate by sex, a variable the model never saw, and none of the four standard plots shows it. This chapter runs the whole sequence of an analysis on that easy dataset, from scaffold to deposited DOI. It labels every step with the chapter that taught it. It performs each step deliberately, including the ones that look unnecessary. The sequence is what you are practicing, so that it is automatic when the data are difficult and the stakes are real. It is also the worked model for the capstone project in Chapter 35.

Everything should be made as simple as possible, but not simpler.

Albert Einstein, attributed; a later paraphrase of On the Method of Theoretical Physics, Herbert Spencer Lecture (1933)

Diagram of the analysis pipeline: six stages in order (Plan, Acquire, Wrangle, Analyze, Report, Share) above two bands that run under every stage (Version control and Environment). Highlighted for this chapter: Report.
Figure 30.1: Where this chapter sits: the case study runs every stage once. The Report stage is highlighted because the chapter’s product is a document a reader can rebuild from a DOI.
NoteIn this chapter

Reading time. About 35 minutes.

You’ll need. The chapters named in Table 30.1, above all Chapter 11 for the compendium, Chapter 21 for complete-case reasoning, Chapter 23 for computed prose, and Chapter 27 for tinytest. The code uses:

You’ll be able to.

  • Map each step of a complete analysis to the chapter that taught it, and use the map as the checklist for your own capstone.
  • Choose which rows a model needs, drop only those, and report the denominator as a computed number.
  • Read a fitted model’s coefficients and diagnostics, and find structure that the four standard diagnostic plots do not show.
  • Write executed tests on the data and the fitted model, and wire them into continuous integration and a deposit.

Outline.

If these answers come quickly, skip to the next chapter. The answers are in Section 30.11.

  1. What data integrity issues does the palmerpenguins::penguins dataset contain, and how should they be handled?
  2. What is the appropriate directory structure for a Palmer Penguins analysis compendium, and how do you produce it?
  3. In a simple regression of body_mass_g on flipper_length_mm, what is the first diagnostic plot you should produce before interpreting any coefficient?

30.1 The arc, step by step

The Palmer Penguins dataset (Horst et al., 2022) is the successor to iris. It has enough structure to exercise the full arc of an analysis, from wrangling through description and inference to reporting. It does so without the eugenics association that accompanies Fisher’s irises, which he analyzed in the Annals of Eugenics. The dataset is small enough that the analysis runs in seconds.

Table 30.1 is the arc. Each row is a step of this chapter, the chapter that taught it, and the check that tells you the step is done. The capstone project in Chapter 35 asks you to fill in the same table for the HF-HOME study.

Table 30.1: The analysis arc, with the chapter that taught each step. Use it as the checklist for the capstone.
Step What you do here Taught in Done when
Scaffold Create the compendium: layout, lockfile, container Section 11.1, Chapter 9, Chapter 10, Chapter 11 The pillar audit of Section 11.7 passes
Clean Map the missingness; drop only rows the model needs Chapter 15, Chapter 21 The analytic n is computed and explained
Describe Table 1 by the design variable Chapter 15, Chapter 21 Missing values appear as a row, not silently
Visualize Scatter plot by species, faceted by island Chapter 20 The plot motivates the model
Fit Linear model of mass on flipper length and species Chapter 26 The model was chosen before the coefficients were seen
Diagnose Standard plots, then residuals against omitted variables Chapter 20 Every pattern is explained or reported
Report Methods and results with computed numbers Chapter 23 No number in the prose is typed
Test Assertions on the data and the model’s statistics Chapter 27 The suite fails when a planted bug is introduced
CI (continuous integration) Tests and render on every push Section 27.9, Section 27.8 A failing test turns the job red
Deposit Tag, archive, DOI (digital object identifier), README Section 7.5, Chapter 34 The DOI resolves and the README rebuilds the paper

palmerpenguins::penguins holds 344 observations of three penguin species collected on three islands of the Palmer Archipelago, Antarctica, from 2007 to 2009.

Table 30.2: Variables in palmerpenguins::penguins.
Variable Type Notes
species factor, 3 levels Adelie, Chinstrap, Gentoo; the design variable
island factor, 3 levels Biscoe, Dream, Torgersen; largely aliased with species
bill_length_mm, bill_depth_mm numeric Bill morphology
flipper_length_mm numeric The predictor
body_mass_g numeric The outcome
sex factor, 2 levels Female, male; the variable with most missing values
year integer 2007, 2008, or 2009

Question. Penguins needs no imputation, has no join, and fits in memory a thousand times over. Why does the chapter scaffold a container, write tests, and set up CI for it?

Answer. The arc is the thing being practiced. A step skipped on easy data is a step not yet automatic on hard data. The steps also catch real problems here. The cleaning step decides the denominator, and the diagnostic step finds structure the model omits. Each step costs minutes. The cost of learning to skip them is paid on the first project where they matter.

30.2 Scaffold and clean

30.2.1 Scaffolding the compendium

Assemble the compendium by hand as Section 11.4 does, or automate it. One way to automate it is the in-house zzcollab framework (Section 11.9), shown here (display-only, because it creates a project and builds an image):

mkdir penguins-analysis
cd penguins-analysis
zzc tidyverse            # scaffold the Five Pillars
make docker-build        # build the image
make r                   # enter R inside the container

zzc tidyverse writes the Five Pillars (Chapter 11): Dockerfile, renv.lock, .Rprofile, source-code directories, and a README pointing at the data.1 make docker-build builds the image (slow on the first run; cached afterward), and make r starts an R session inside it. Whichever route you took, run the tool-neutral audit of Section 11.7 before writing analysis code. In a zzcollab project, make check-renv-no-fix adds a report on whether the code, DESCRIPTION, and renv.lock agree.

1 The tidyverse profile was formerly called analysis, and zzc analysis still works as a deprecated alias (as of the repository in October 2026).

30.2.2 Mapping the missingness

Missingness map of the raw penguin data, one row per observation and one column per variable, with missing cells marked. The sex column shows scattered missing cells throughout; two rows are missing across all four morphology columns at once, appearing as short horizontal gaps.
Figure 30.2: Missingness map for the raw penguin data. Missingness is concentrated in sex, with two rows missing all four morphology measurements.
miss_var_summary(penguins)
#> # A tibble: 8 × 3
#>   variable          n_miss pct_miss
#>   <chr>              <int>    <num>
#> 1 sex                   11    3.20 
#> 2 bill_length_mm         2    0.581
#> 3 bill_depth_mm          2    0.581
#> 4 flipper_length_mm      2    0.581
#> 5 body_mass_g            2    0.581
#> 6 species                0    0    
#> 7 island                 0    0    
#> 8 year                   0    0

Missingness is mostly in sex (11 rows, 3.2%). Two rows are missing all four morphology measurements, plausibly because of a lost data sheet for each bird.

ImportantLLM prompt

Ask a model to lay out the options for the missing sex values, not to choose one.

Context. I am analyzing palmerpenguins::penguins (344 rows; columns and types from str(penguins) pasted below). sex is missing in 11 rows; two rows are missing every morphology measurement. My planned model is lm(body_mass_g ~ flipper_length_mm + species).

Constraints. Do not recommend imputing sex from sex-specific distributions of other variables. Do not use the fraction missing as the criterion.

Criteria. First say whether the planned model needs the rows with missing sex handled at all. Then, for a second model that adds sex, compare complete-case analysis with multiple imputation, stating for each the condition under which the coefficient estimates are unbiased.

Failure mode. Models often answer a different question. They recommend handling sex although the planned model never uses it. Or they propose imputing sex ‘from sex-specific distributions’, which presupposes the sex it is trying to fill in. A second common error states that complete case is biased whenever data are missing at random. Check. The answer should drop no row for the planned model, and its condition for complete case should match Section 21.3. Fit the model with sex both ways (mice::mice() with the outcome in the imputation model) and compare the coefficients. A data dictionary is structure, not data, so it may go to any approved tool (Section 12.3).

30.2.3 Dropping only what the model needs

Complete-case analysis is defensible here. The reason is not that the missing fraction is small. Complete-case estimates of a regression coefficient are unbiased when missingness does not depend on the outcome once the model’s covariates are held fixed (Section 21.3). Here the missingness plausibly reflects field data collection rather than body mass. Moreover, the model below does not use sex at all. Drop only the rows the analysis needs, not every row with any gap, and record the choice:

penguins_clean <- tidyr::drop_na(penguins, body_mass_g,
                                 flipper_length_mm)
nrow(penguins_clean)
#> [1] 342
TipVerification

Assert the denominator you intend, and that no modeled column carries a missing value.

stopifnot(
  nrow(penguins) - nrow(penguins_clean) == 2,
  !anyNA(penguins_clean[c("body_mass_g", "flipper_length_mm",
                          "species")])
)

The two rows missing every morphology measurement are dropped, and analyses use \(n = 342\). The 9 further rows missing only sex stay in, and Table 1 reports their sex as missing.

WarningPitfall

na.omit() drops rows the model does not need, and the model still fits. na.omit(penguins) and tidyr::drop_na(penguins) with no columns named remove every row with any missing value. 333 rows remain instead of 342. Nothing errors, and the coefficients shift only slightly on this dataset. However, the methods section’s denominator is now wrong. On a dataset where the incidental column is missing for a reason, the estimate moves too. Name the modeled columns in drop_na().

Question. A colleague argues that because only 3.2% of rows have a missing value, complete-case analysis is safe. What is wrong with the argument, and what would make complete case unsafe here?

Answer. The fraction missing governs precision, not bias. Complete-case coefficient estimates are unbiased when missingness does not depend on the outcome given the covariates, whatever the fraction. Suppose, however, that the heaviest birds were the ones whose mass went unrecorded because they could not be weighed. The estimates would then be biased, since missingness would depend on body mass itself.

30.3 Describe and visualize

30.3.1 Table 1

gtsummary (Sjoberg et al., 2021) produces a Table 1 from one call. Stratify by the design variable, and keep missing values as a visible row:

library(gtsummary)

penguins_clean |>
  tbl_summary(
    by = species,
    include = c(bill_length_mm, flipper_length_mm, body_mass_g,
                sex, island),
    missing_text = "Missing"
  ) |>
  add_overall() |>
  as_kable()
Table 30.3: Characteristics of the analytic sample by species, median (first quartile, third quartile) or n (%).
Characteristic Overall N = 342 Adelie N = 151 Chinstrap N = 68 Gentoo N = 123
bill_length_mm 44.5 (39.2, 48.5) 38.8 (36.7, 40.8) 49.6 (46.3, 51.2) 47.3 (45.3, 49.6)
flipper_length_mm 197 (190, 213) 190 (186, 195) 196 (191, 201) 216 (212, 221)
body_mass_g 4,050 (3,550, 4,750) 3,700 (3,350, 4,000) 3,700 (3,475, 3,950) 5,000 (4,700, 5,500)
sex
female 165 (50%) 73 (50%) 34 (50%) 58 (49%)
male 168 (50%) 73 (50%) 34 (50%) 61 (51%)
Missing 9 5 0 4
island
Biscoe 167 (49%) 44 (29%) 0 (0%) 123 (100%)
Dream 124 (36%) 56 (37%) 68 (100%) 0 (0%)
Torgersen 51 (15%) 51 (34%) 0 (0%) 0 (0%)

The table carries no row p-values. Table 1 describes the sample, and inference belongs to the model fitted below. The island rows show the design. Adelies appear on all three islands, Chinstraps only on Dream, and Gentoos only on Biscoe. Species and island are therefore largely aliased. An island difference can be estimated only among Adelies, and a model with both terms cannot separate them for the other two species. Leaving island out of the model is a modeling choice. It is not a finding that island does not matter.

The in-house zztable1 package (Appendix C) offers a similar formula interface for a journal-styled table. However, as of this writing it leaves the missing-value row blank instead of reporting the count, which breaks the rule just stated. Use gtsummary until that is fixed.

30.3.2 Exploratory scatter plot

library(ggplot2)

theme_set(theme_minimal())

ggplot(penguins_clean,
       aes(flipper_length_mm, body_mass_g, color = species)) +
  geom_point(alpha = 0.7) +
  scale_color_brewer(palette = "Dark2") +
  labs(x = "Flipper length (mm)",
       y = "Body mass (g)",
       color = "Species") +
  facet_wrap(~ island)
Scatter plot of body mass against flipper length, points colored by species and split into one panel per island. Three separated clusters are visible, with Gentoo heaviest and longest-flippered; within every cluster body mass rises with flipper length.
Figure 30.3: Body mass against flipper length, colored by species and faceted by island. Three clear clusters emerge, and the within-cluster association motivates the model that follows.

The plot shows three clusters by species, with flipper length and body mass strongly associated within each. Gentoos (Biscoe only) are the heaviest and longest-flippered. Adelies (all three islands) and Chinstraps (Dream only) overlap on mass and are distinguished by bill length instead. The structure suggests the model: species as a categorical predictor and flipper length as a continuous one.

30.3.3 Exercises

  1. Write a second Table 1 stratifying by year (2007, 2008, 2009) rather than species. What does the difference between the two tables tell you about the data collection design?
penguins_clean |>
  dplyr::group_by(year) |>
  dplyr::summarize(
    n            = dplyr::n(),
    across(c(bill_length_mm, flipper_length_mm, body_mass_g),
           \(x) round(mean(x), 1)),
    .groups = "drop"
  )
#> # A tibble: 3 × 5
#>    year     n bill_length_mm flipper_length_mm body_mass_g
#>   <int> <int>          <dbl>             <dbl>       <dbl>
#> 1  2007   109           43.7              197.       4124.
#> 2  2008   114           43.5              203.       4267.
#> 3  2009   119           44.5              203.       4210.

table(penguins_clean$year, penguins_clean$species)
#>       
#>        Adelie Chinstrap Gentoo
#>   2007     49        26     34
#>   2008     50        18     46
#>   2009     52        24     43

The year-stratified table is unremarkable next to the species one. Mean body mass spans about 140 g across years. Across species it spans about 1,380 g. Mean flipper length moves 6 mm by year against 27 mm by species. The between-year differences are real but an order of magnitude smaller. A Table 1 stratified by year would present them to the reader as though they were the structure of the data.

The cross-tabulation says why they are small. Each species was sampled in every year in roughly stable proportions. Year is therefore close to a nuisance variable, and species is the design. A Table 1 that stratifies by the wrong variable looks reassuringly balanced and tells you little.

Question. A reviewer asks you to add island to the model ‘to adjust for location’. What does Table 30.3 tell you about that request?

Answer. That most of the adjustment cannot be estimated. All Chinstraps are on Dream and all Gentoos on Biscoe. For those species an island term and a species term are the same contrast. Only the Adelie data can separate them. Reply with the cross-tabulation, and offer an Adelie-only analysis of island if the question matters.

30.4 Fit and diagnose

30.4.1 The fitted model

fit <- lm(body_mass_g ~ flipper_length_mm + species,
          data = penguins_clean)
broom::tidy(fit, conf.int = TRUE)
#> # A tibble: 4 × 7
#>   term              estimate std.error statistic  p.value conf.low conf.high
#>   <chr>                <dbl>     <dbl>     <dbl>    <dbl>    <dbl>     <dbl>
#> 1 (Intercept)        -4031.     584.       -6.90 2.55e-11  -5181.    -2882. 
#> 2 flipper_length_mm     40.7      3.07     13.3  1.40e-32     34.7      46.7
#> 3 speciesChinstrap    -207.      57.7      -3.58 3.98e- 4   -320.      -93.0
#> 4 speciesGentoo        267.      95.3       2.80 5.39e- 3     79.4     454.
broom::glance(fit) |>
  dplyr::select(r.squared, adj.r.squared, sigma, df, nobs)
#> # A tibble: 1 × 5
#>   r.squared adj.r.squared sigma    df  nobs
#>       <dbl>         <dbl> <dbl> <dbl> <int>
#> 1     0.783         0.781  376.     3   342

Every number in the paragraphs that follow is computed by the chunks above rather than typed in. This is the discipline of Chapter 23 applied to the book’s own case study.

The intercept is the predicted body mass of an Adelie penguin at a flipper length of zero. It is an extrapolation far outside the data and has no useful interpretation. It exists to anchor the line. The flipper_length_mm coefficient, 40.7 g per mm, is the within-species slope. That is, it is the expected difference in body mass between two penguins of the same species whose flipper lengths differ by one millimeter.

The species coefficients are where a reader most often misreads the table. They are differences from Adelie at the same flipper length. That is not the difference in average mass between the species. Gentoos average about 1,380 g heavier than Adelies. The speciesGentoo coefficient is 270 g, a small fraction of that. The reason is that Gentoos also have much longer flippers, and the model has already credited the flipper-length term with most of the difference. The adjusted coefficient answers ‘how much heavier is a Gentoo than an Adelie with the same flippers’. That is a narrower question than the one a reader assumes it answers.

penguins_clean |>
  dplyr::group_by(species) |>
  dplyr::summarize(mean_mass = mean(body_mass_g),
                   mean_flipper = mean(flipper_length_mm),
                   .groups = "drop")
#> # A tibble: 3 × 3
#>   species   mean_mass mean_flipper
#>   <fct>         <dbl>        <dbl>
#> 1 Adelie        3701.         190.
#> 2 Chinstrap     3733.         196.
#> 3 Gentoo        5076.         217.

The \(R^2\) of 0.78 is respectable. On a clean morphometric dataset, however, a respectable \(R^2\) is not evidence that the model is well specified. It says the predictors and the outcome are related. Whether the functional form is right, or a variable is missing, is a question for the diagnostics.

30.4.2 Diagnostic plots

par(mfrow = c(2, 2))
plot(fit)
par(mfrow = c(1, 1))
Four-panel regression diagnostic grid. Residuals against fitted values fall in two clusters, Adelie and Chinstrap on the left and Gentoo on the right, with a smoother that dips slightly in the middle. The normal quantile-quantile plot follows its reference line, with three points slightly above it in the upper tail. The scale-location smoother is nearly flat, a little lower at the largest fitted values. Residuals against leverage show every point far inside the Cook's distance contours.
Figure 30.4: The four standard lm diagnostic plots for the fitted model. Residuals versus fitted and scale-location test the mean and variance structure; the Q-Q plot tests the residual distribution; residuals versus leverage identifies influential observations.

Read the four panels in turn, and say what each shows rather than what it should show.

Table 30.4: What the standard diagnostics show for the penguin model.
Panel What it shows for this fit Conclusion
Residuals vs fitted Two clusters (Adelie and Chinstrap left, Gentoo right) and a smoother with a shallow dip; a quadratic flipper term improves the fit (p = 0.00071) Mild curvature; small next to the residual spread, worth a sensitivity analysis
Q-Q Points on the reference line; three birds sit slightly above it in the upper tail; standardized residuals beyond 3: 1 No meaningful departure from normality
Scale-location Nearly flat; residual standard deviation (SD) is 408 g for Adelies, 298 g for Chinstraps, 370 g for Gentoos Modest heteroscedasticity by species; robust standard errors would be a reasonable sensitivity check
Residuals vs leverage Largest Cook’s distance 0.023, far below the conventional 0.5 No single bird drives the fit

The plots are not featureless, but the features are small. The discipline is the same on clinical data, where the features are rarely small. Check, name each pattern, and decide whether it changes the conclusion.

30.4.3 What the four plots cannot show

The standard plots look only at the variables in the model. A variable left out shows up only when you plot the residuals against it, and sex is the obvious candidate:

penguins_resid <- penguins_clean |>
  dplyr::mutate(resid = residuals(fit)) |>
  dplyr::filter(!is.na(sex))

ggplot(penguins_resid, aes(sex, resid, color = sex)) +
  geom_hline(yintercept = 0, color = "grey60") +
  geom_jitter(width = 0.15, alpha = 0.6) +
  scale_color_brewer(palette = "Dark2", guide = "none") +
  facet_wrap(~ species) +
  labs(x = NULL, y = "Residual (g)")
Strip plot of model residuals, one panel per species, with female and male penguins side by side. In the Adelie and Gentoo panels the female residuals sit mostly below zero and the male residuals mostly above, with little overlap; in the Chinstrap panel the two groups overlap heavily.
Figure 30.5: Residuals of the main-effects model by sex, within species. The model omits sex, and its residuals separate by it, strongly in Adelies and Gentoos and weakly in Chinstraps.

Males sit on average 390 g above females at the same flipper length and species. The gap is large in Adelies and Gentoos and small in Chinstraps, which is itself a finding a report should mention. The four standard plots gave no hint of it. The residuals’ overall distribution is a symmetric mixture of two shifted groups, and it passes for normal. Whether to add sex depends on the question. To describe mass as a function of flipper length within species, the model stands, and the sex structure belongs in the report. To predict mass, add sex and handle its missing values as the LLM prompt above lays out.

WarningPitfall

Four clean diagnostic plots do not mean a well-specified model. They test the shape of the residuals against the fitted values, not whether a variable is missing. An omitted variable whose effect is roughly symmetric hides inside residuals that look normal and evenly spread. Plot the residuals against every candidate variable you left out.

30.4.4 The interaction model

Resist over-modeling. The main-effects model fits well. An interaction model, body_mass_g ~ flipper_length_mm * species, gives each species its own slope at the cost of two parameters. Its AIC (Akaike information criterion) is 7.1 lower. An AIC difference of \(\Delta\) gives the simpler model a relative likelihood of \(e^{-\Delta/2}\) against the larger one. Here that is about 0.029, which favors the larger model without settling the matter. Whether to report it depends on three things. The first is whether different slopes make biological sense (different body shapes make them plausible). The second is whether the goal is description or prediction. The third is the audience. The defensible choice is to report both, with the AIC comparison, and let the reader see the trade-off. Reporting the larger model because it has more p-values is the wrong reason.

30.4.5 What the model licenses you to say

The coefficient of 40.7 g per mm is an association. It is tempting to call it ‘the effect of flipper length on body mass’. However, the word ‘effect’ claims something the analysis cannot support. Figure 30.6 sketches the plausible causal structure. Species and sex each influence both flipper length and body mass. Both measurements also grow with the bird’s overall size, which is not measured.

flowchart TD
  SP["Species"] --> FL["Flipper length"]
  SP --> BM["Body mass"]
  SX["Sex"] --> FL
  SX --> BM
  SZ["Overall size<br/><i>unmeasured</i>"] --> FL
  SZ --> BM
Figure 30.6: A plausible causal graph for the penguin measurements. Flipper length and body mass share causes, one of them unmeasured; no arrow runs from one to the other.

A causal question needs a hypothetical experiment that would answer it, the target trial (Hernán & Robins, 2016). The target trial specifies who would be enrolled, what would be assigned, and what would be measured. No such trial exists for ‘assign a penguin a longer flipper’. The question is therefore descriptive, and the honest verbs are ‘is associated with’ and ‘predicts’. The opposite error matters as much. Sometimes a study’s goal is causal, as in an observational comparison of two treatments. Calling the estimate an association then does not make the analysis less causal. It only hides the assumptions the reader needs to judge it. Hernán’s argument against that euphemism (Hernán, 2018) is short and worth reading before your first observational analysis. Habit 11 (Section 1.6) states the rule: match the verb to the design.

30.4.6 Exercises

  1. Extend the analysis: fit the interaction model body_mass_g ~ flipper_length_mm * species. Test the interaction, and report the AIC against the main-effects model.
main <- lm(body_mass_g ~ flipper_length_mm + species,
           data = penguins_clean)
inter <- lm(body_mass_g ~ flipper_length_mm * species,
            data = penguins_clean)

anova(main, inter)
#> Analysis of Variance Table
#> 
#> Model 1: body_mass_g ~ flipper_length_mm + species
#> Model 2: body_mass_g ~ flipper_length_mm * species
#>   Res.Df      RSS Df Sum of Sq     F   Pr(>F)   
#> 1    338 47666988                               
#> 2    336 46147424  2   1519564 5.532 0.004327 **
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
AIC(main, inter)
#>       df      AIC
#> main   5 5031.523
#> inter  7 5024.443

The interaction asks whether the mass-per-millimeter slope differs by species. It does not ask whether the species differ in mass. The anova() F-test and the AIC comparison answer the same question two ways, and they should agree. Where they disagree, the difference is small enough that the choice is not doing any work. Report the comparison you pre-specified, not the one that came out better.

  1. Fit a multinomial regression of species on the four morphometric measurements, and report how often it recovers the observed species.
fit_mn <- nnet::multinom(
  species ~ bill_length_mm + bill_depth_mm +
            flipper_length_mm + body_mass_g,
  data = penguins_clean)
predicted <- predict(fit_mn)
tab <- table(observed = penguins_clean$species, predicted)
tab
#>            predicted
#> observed    Adelie Chinstrap Gentoo
#>   Adelie       151         0      0
#>   Chinstrap      0        68      0
#>   Gentoo         0         0    123
round(sum(diag(tab)) / sum(tab), 3)
#> [1] 1

Every penguin is classified correctly, which the scatter plot already implied. The three species separate cleanly in morphometric space.

An accuracy of exactly 1 should make you suspicious rather than pleased. The number is resubstitution on the data the model was fitted to, with four continuous predictors and well-separated groups. It therefore measures how flexible the model is at least as much as how learnable the species are. Report it as what the exercise asked for, how often the model recovers the observed species. Never report it as predictive accuracy. That claim needs cross-validation or a held-out set, and would come back lower.

Question. The speciesGentoo coefficient is 270 g. A draft results paragraph says ‘Gentoo penguins were 270 g heavier than Adelies’. What is wrong, and how do you fix the sentence?

Answer. The coefficient is the difference at equal flipper length. The raw difference in mean mass is about 1,380 g. Either report the raw difference as a description, or say ‘at the same flipper length, Gentoos were 270 g heavier on average than Adelies’. The draft’s sentence also says ‘were heavier’, which is a description of these birds. It should not become ‘species increases mass’.

30.5 Test, report, and automate

30.5.1 Tests on the data and the model

The checks so far ran once, in this chapter. A compendium needs them to run every time the analysis does, so they belong in a test file. Following Habit 9, the tests below check statistics, not shapes. They check the denominator, the factor levels the model’s reference category depends on, and the slope itself, re-derived by an independent route. In a model with a common slope, the within-species slope equals the pooled within-species covariance of flipper length and mass divided by the pooled within-species variance of flipper length. It can therefore be computed without lm().

# inst/tinytest/test-penguins-model.R
penguins_clean <- tidyr::drop_na(palmerpenguins::penguins,
                                 body_mass_g, flipper_length_mm)
fit <- lm(body_mass_g ~ flipper_length_mm + species,
          data = penguins_clean)

expect_equal(nrow(penguins_clean), 342L,
             info = "only the two rows missing a modeled variable dropped")
expect_equal(sum(is.na(penguins_clean$sex)), 9L,
             info = "rows missing only sex are kept")
expect_identical(levels(penguins_clean$species),
                 c("Adelie", "Chinstrap", "Gentoo"),
                 info = "Adelie is the reference level")
expect_equal(nobs(fit), nrow(penguins_clean),
             info = "lm() used every analytic row")

by_species <- split(penguins_clean, penguins_clean$species)
sxy <- sum(vapply(by_species, \(d) {
  sum((d$flipper_length_mm - mean(d$flipper_length_mm)) *
        (d$body_mass_g - mean(d$body_mass_g)))
}, numeric(1)))
sxx <- sum(vapply(by_species, \(d) {
  sum((d$flipper_length_mm - mean(d$flipper_length_mm))^2)
}, numeric(1)))
expect_equal(unname(coef(fit)["flipper_length_mm"]), sxy / sxx,
             info = "slope equals the pooled within-species estimate")

ci <- confint(fit)["flipper_length_mm", ]
expect_true(ci[1] > 0,
            info = "the within-species association is positive")

The expected counts (342 and 9) are written into the test on purpose. They are the denominator the methods section states, and the test exists to fail if a later change to the cleaning moves it. In a project the chunk is the file inst/tinytest/test-penguins-model.R. Here, the chunk below writes it to a temporary file and runs it as tinytest would.

library(tinytest)

test_file <- file.path(tempdir(), "test-penguins-model.R")
writeLines(knitr::knit_code$get("penguins-tests"), test_file)
run_test_file(test_file, verbose = 0)
#> All ok, 6 results (27ms)
TipVerification

A test suite that has never failed has not been shown to work. Plant the bug from the cleaning Pitfall, dropping every incomplete row, and confirm that the suite catches it.

mutant_file <- file.path(tempdir(), "test-penguins-mutant.R")
mutant <- sub("tidyr::drop_na(palmerpenguins::penguins,",
              "tidyr::drop_na(palmerpenguins::penguins)",
              knitr::knit_code$get("penguins-tests"), fixed = TRUE)
mutant <- sub("^ +body_mass_g, flipper_length_mm\\)$", "", mutant)
writeLines(mutant, mutant_file)
mutant_results <- run_test_file(mutant_file, verbose = 0)
stopifnot(any(!as.logical(mutant_results)))
mutant_results
#> ----- FAILED[data]: test-penguins-mutant.R<7--8>
#>  call| expect_equal(nrow(penguins_clean), 342, info = "only the two rows missing a modeled variable dropped")
#>  diff| Expected '342', got '333'
#>  info| only the two rows missing a modeled variable dropped
#>  ----- FAILED[data]: test-penguins-mutant.R<9--10>
#>  call| expect_equal(sum(is.na(penguins_clean$sex)), 9, info = "rows missing only sex are kept")
#>  diff| Expected '9', got '0'
#>  info| rows missing only sex are kept
#>  
#> Showing 2 out of 6 results: 2 fails, 4 passes (3ms)

The denominator and the kept-sex tests fail. The slope test passes, because the slope is recomputed from the same wrong data. That is why the suite needs both kinds of test.

30.5.2 The paper

The compendium’s manuscript, analysis/paper/paper.qmd, is short and contains no new analysis. Its setup chunk loads the data, applies the same drop_na(), and fits the same model. Its Methods section states the denominator with inline code ({r} nrow(penguins_clean), and the number dropped as a difference). Its Results section cross-references the scatter plot and a coefficient table produced by broom::tidy(). The full pattern, a Quarto document whose every number is computed, is the worked example of Section 23.4. The compendium layout that holds it is in Section 11.4. Rendering inside the container (display-only) reproduces the paper from raw data to typeset output:

make docker-render REPORT=analysis/paper/paper.qmd
ImportantLLM prompt

Ask for a draft results paragraph, and treat it as a draft.

Context. Below are the output of broom::tidy(fit, conf.int = TRUE) and broom::glance(fit) for lm(body_mass_g ~ flipper_length_mm + species) fitted to palmerpenguins (Adelie is the reference level), and the species means of body mass and flipper length.

Constraints. Report estimates with 95% confidence intervals, not p-values alone. Use ‘is associated with’, never ‘effect’ or ‘increases’. Write every number as a Quarto inline expression ({r} ...) referring to fit, not as a literal.

Criteria. State that species coefficients are differences at equal flipper length, and give the raw species difference beside the adjusted one. Name Adelie as the reference.

Failure mode. Models report the adjusted species coefficient as the species difference, call the slope an effect, or type numbers that are rounded differently from the table. Check. Render the paragraph, change the data (drop the Gentoo rows), and render again. Every number must change. Then read the paragraph against the Check at the end of Section 30.4.

30.5.3 Continuous integration

CI (continuous integration) runs the tests and the render on every push, so that a change that moves the denominator turns a check red before anyone reads the paper. The workflow is the one in Section 27.8 with a render step added (display-only; Section 27.9 covers the rest):

# .github/workflows/check.yaml
name: check
on: [push, pull_request]
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: r-lib/actions/setup-r@v2
      - uses: r-lib/actions/setup-renv@v2
      - uses: quarto-dev/quarto-actions/setup@v2
      - name: Run tests
        run: |
          Rscript -e 'res <- tinytest::run_test_dir("inst/tinytest")' \
            -e 'quit(status = as.integer(tinytest::any_fail(res)))'
      - name: Render paper
        run: quarto render analysis/paper/paper.qmd

setup-renv restores the packages in renv.lock, so CI runs the versions the paper was written with. The quit() line is what turns a failing test into a failing job (Section 27.8).

30.5.4 Exercises

  1. Add the multinomial model of exercise 3 to the compendium as a second analysis script with its own test file, and confirm that the CI job passes, then fails when you plant a bug.

A correct result has three parts. The script (analysis/scripts/02-multinom.R) fits the model and saves the confusion table. The test file asserts a statistic, not a shape: for example, that the confusion table’s total equals nrow(penguins_clean) and that its diagonal proportion equals the value computed by mean(predict(fit_mn) == penguins_clean$species) in the test itself. The CI log shows the test file’s results line, and a commit that changes the formula (dropping bill_length_mm, say) makes the job red. If the job stays green after the planted change, the test checks something the change does not affect.

Question. In the planted-bug Verification, the slope test still passed. Why is that not a weakness of the slope test?

Answer. The slope test checks that lm() computed what it claims from the data it was given. It re-derives the estimate independently, which catches a wrong formula or a wrong model. It cannot catch wrong data, because both routes see the same rows. The denominator test catches wrong data. Each test covers a failure the other cannot.

30.6 Deposit

A tag marks the commit the paper was rendered from. An archive gives it a DOI (digital object identifier) that outlives the repository (Chapter 34). From the shell (display-only):

git tag -a v1.0 -m "Penguins case study, complete"
git push origin v1.0

The Zenodo integration is enabled for the repository once, in the Zenodo account settings. With it enabled, publishing a GitHub release from the tag archives the repository at that commit and mints a DOI. The deposit is the repository at the tagged commit. Its README answers four questions:

Table 30.5: The four questions a deposit README answers.
README section Content
What One paragraph on the question and the dataset
How to reproduce make docker-build, then make docker-render REPORT=analysis/paper/paper.qmd
Where the data are In the palmerpenguins package, at the version in renv.lock
How to cite The DOI and the authors

A reader given the DOI can rebuild the paper in two commands.

TipVerification

Resolve the DOI in a browser. Confirm that it reaches the archived release, not the live repository, and that the archived files include renv.lock and the Dockerfile. Then clone the tagged commit into a new directory and run the two README commands. The rendered numbers should match the paper’s.

Question. Why archive on Zenodo when the repository is already public on GitHub?

Answer. Hosting is not archiving. A GitHub repository can be renamed, rewritten, or deleted, and its URL then points nowhere. A Zenodo record is preserved under a DOI that cites one fixed version, which is what a paper’s citation needs.

30.7 Rules of thumb

  1. Habit 1: the compendium is the unit. Scaffold before the first line of analysis code, and deposit the whole compendium, not the paper.
  2. Habit 9: test the statistics, not the shape. Assert the denominator and re-derive a key estimate; a column-exists test passes on a wrong answer.
  3. Habit 11: match the verb to the design. A cross-sectional coefficient is an association; say so, and say why.

30.8 Cumulative practice

  1. Cumulative (uses Chapter 10). Reproduce this chapter’s compendium from scratch: scaffold it, add the cleaning, model, paper, tests, and CI workflow, and render it inside the container. Confirm that every number in the rendered paper matches the numbers in this chapter, and that the test suite catches the planted bug from Section 30.5.
  2. On your own work (no key). Deposit a compendium of your own to Zenodo, and confirm that the DOI resolves to the archived release and that the README is the first thing a reader sees.

30.9 Cheat sheet

Table 30.6: The analysis arc at a glance.
Step Command Check
Scaffold zzc tidyverse, or the layout of Section 11.4 Pillar audit passes
Map missingness naniar::vis_miss(df) Pattern explained
Drop for the model tidyr::drop_na(df, y, x) nrow() matches the methods
Table 1 gtsummary::tbl_summary(df, by = group) Missing shown as a row
Fit lm(y ~ x + group, data = df) Reference level stated
Diagnose plot(fit); residuals against omitted variables Each pattern named
Test tinytest::run_test_dir("inst/tinytest") A planted bug fails
Render make docker-render REPORT=analysis/paper/paper.qmd Numbers change when data change
Tag git tag -a v1.0 -m "..." git show v1.0
Deposit GitHub release with Zenodo enabled DOI resolves

30.10 Further reading

30.11 Prerequisites answers

  1. palmerpenguins::penguins contains 11 rows with at least one missing value (most commonly sex). First document the missing pattern (naniar::vis_miss()), then choose a strategy before analysis. Drop only rows missing a variable the model uses. Complete case is then defensible when missingness plausibly does not depend on the outcome given the covariates, as the chapter argues is the case here. That is the condition. It is not MCAR (missing completely at random), and it is not a percentage threshold. Multiple imputation is preferable when missingness may depend on the outcome or when the dropped rows carry precision worth recovering.
  2. A research compendium (Chapter 11): the Five Pillars, assembled by hand from the rrtools layout, renv, and the canonical Dockerfile, or automated, for example with mkdir penguins && cd penguins && zzc tidyverse. The analysis code goes in analysis/scripts/, the manuscript in analysis/paper/, and the tests in inst/tinytest/.
  3. A residuals-vs-fitted plot. It reveals non-linearity, non-constant variance, and systematic lack of fit in a single view. Read it, and the other three diagnostic plots, for what they actually show. Then plot the residuals against variables the model omits. Only then interpret the regression coefficient as the association between flipper length and body mass.