10  Docker for Reproducibility

Two analysts ran the same bootstrap script, from the same commit and with the same seed, and reported different confidence intervals for the same trial. Neither had made an error. One machine ran R 3.5 and the other R 3.6. R 3.6.0 changed the algorithm sample() uses to draw integers, so the same seed produced different resamples. The difference sat in the second decimal place. It was small enough to pass for rounding and large enough to change a table a reviewer had already seen. renv could not have caught it, because no package changed. What changed was the R build beneath the packages. A container fixes R, the system libraries, and the operating system along with the packages. The same code then gives the same number on every machine that runs the image.

But it works on my machine.

A folk theorem of software (n.d.)

Diagram of the analysis pipeline: six stages in order (Plan, Acquire, Wrangle, Analyze, Report, Share) above two bands that run under every stage (Version control and Environment). Highlighted for this chapter: Environment.
Figure 10.1: Where this chapter sits: Docker pins the whole Environment band (operating system, system libraries, R, and, through renv, the packages) under every stage of the analysis.
NoteIn this chapter

Reading time. About 25 minutes.

You’ll need. Chapter 9 for the lockfile, and Docker Desktop (or Docker Engine on Linux) installed. The one executed example uses base R and the HF-HOME files; every Docker command is display-only, because the book does not run Docker when it renders.

You’ll be able to.

  • Explain what a container pins that renv does not, and show a number that changes with the R version alone.
  • Write a Dockerfile whose layers cache correctly, with a digest-pinned base image and a .dockerignore, and confirm both from the build output.
  • Run RStudio Server in a container reachable only from your own machine.
  • Archive a built image so that a reader can run it after the registry and the base image have moved on.

Outline.

If these answers come quickly, skip to the next chapter. The answers are in Section 10.13.

  1. What is the difference between a Docker image and a Docker container?
  2. Why might a data analyst prefer Docker over renv for long-term reproducibility, and what does Docker capture that renv alone does not?
  3. What does the base image rocker/tidyverse:4.4.0 give you out of the box?

10.1 Images, containers, and layers

10.1.1 What renv leaves out

renv pins the R packages, and R packages are not the whole environment. Beneath them sit the R build itself, the system libraries packages compile against, and the operating system (OS). The lockfile installs none of these. Most of the time this does not matter, which is exactly why it is easy to overlook. The chunk below reproduces the opening on the HF-HOME trial. It bootstraps the 30-day risk difference, in percentage points, with the same seed under two integer samplers. The first is the one R used before version 3.6.0, and the second is the one it has used since.

hf_dir <- file.path('..', 'data', 'raw_data', 'hfhome')
rand <- read.csv(file.path(hf_dir, 'randomization.csv'),
                 colClasses = c(patient_id = 'character'))
outc <- read.csv(file.path(hf_dir, 'outcomes.csv'),
                 colClasses = c(patient_id = 'character'))
hf <- merge(rand[c('patient_id', 'arm')],
            outc[c('patient_id', 'event_30d')], by = 'patient_id')
stopifnot(nrow(hf) == nrow(rand))
hf <- hf[!is.na(hf$event_30d), ]
usual <- hf$event_30d[hf$arm == 'Usual care']
home <- hf$event_30d[hf$arm == 'Home health']

boot_rd <- function(sample_kind, reps = 2000, seed = 2019) {
  old <- RNGkind()
  on.exit(RNGkind(old[1], old[2], old[3]))
  suppressWarnings(RNGkind(sample.kind = sample_kind))
  set.seed(seed)
  rd <- replicate(reps, {
    mean(sample(usual, replace = TRUE)) -
      mean(sample(home, replace = TRUE))
  })
  100 * quantile(rd, c(0.025, 0.975))
}

intervals <- rbind(r_before_3_6 = boot_rd('Rounding'),
                   r_3_6_and_later = boot_rd('Rejection'))
round(intervals, 2)
#>                  2.5% 97.5%
#> r_before_3_6    -1.00  5.97
#> r_3_6_and_later -0.91  5.88

The data, the seed, and the resampling code are the same. Only the sampler differs, as it does between R 3.5 and any R since 3.6.0. The lower bound moves from -1.00 to -0.91. Both intervals include zero, as the trial’s result is inconclusive. However, a reader comparing your table with a rerun sees two different numbers and no explanation. The sample.kind argument lets one R imitate the other here. A change in a system linear-algebra library offers no such switch. It can move the last digits of a regression coefficient between two machines with identical lockfiles. Docker closes the gap by specifying the whole stack rather than its top layer.

The cost is a second technology to learn and a build step to wait for. The cost is worth paying for work that must reproduce years later, and for work that depends on system libraries at all. It is not worth paying for a self-contained analysis that will be finished this quarter. Boettiger (2015) made the case for containers in research computing, and this chapter assumes his argument.

10.1.2 The mental model

Table 10.1: Docker’s vocabulary.
Term What it is What follows
Image An immutable, read-only template: OS, system libraries, R, packages, code The artifact you build, share, and archive
Container A running instance of an image, with a writable layer on top Changes made inside it vanish when it is removed
Layer The result of one RUN, COPY, or ADD instruction Cached and reused, which is why order matters
Registry A server that stores images (Docker Hub, GitHub Container Registry) Hosting, not archiving (Section 10.5)
Tag A movable name for an image, such as 4.4.0 Can be rebuilt to point at different bytes
Digest The content hash of an image, sha256:... The only identifier that cannot move

On an Apple Silicon Mac, add --platform linux/amd64 to every docker pull, docker run, and docker build. Many R images and binary package repositories target x86_64 only. An image built for arm64 will not run on a typical Linux server (see Table 10.3). Every command in this chapter carries the flag, which costs nothing on an Intel machine.

The everyday commands (display-only):

docker pull --platform linux/amd64 rocker/r-ver:4.4.0
docker run -it --rm --platform linux/amd64 rocker/r-ver:4.4.0 R
docker run -it --rm --platform linux/amd64 -v "$(pwd)":/work \
  rocker/r-ver:4.4.0 R               # with the project mounted
docker build --platform linux/amd64 -t my-analysis:v1.0 .
docker images                        # list local images
docker rmi my-analysis:v1.0          # remove an image

The --rm flag removes the container when it exits. Without it, stopped containers accumulate.

TipVerification

Confirm that the image gives you the R and the OS you expect, whatever your laptop runs:

docker run --rm --platform linux/amd64 rocker/r-ver:4.4.0 R --version
docker run --rm --platform linux/amd64 rocker/r-ver:4.4.0 \
  grep PRETTY /etc/os-release

The first should begin R version 4.4.0, and the second should name Ubuntu 22.04 LTS (long-term support), the base of the rocker/r-ver 4.4 images. If you see your laptop’s R version, you ran R outside the container.

Question. You build an image readmissions:v1.0, run a container from it, install a new R package inside the running container, then exit. Is the package available the next time you run a container from the same image?

Answer. No. The image is immutable. Changes inside a container live only in its writable layer, which is discarded when the container is removed (at exit, with --rm). To add the package permanently, install it in the project, snapshot renv.lock, and rebuild the image. Docker reproduces an analysis only if changes happen in the Dockerfile and the lockfile, in version control. They should never happen in running containers. ‘Works on my machine’ becomes ‘works on this image, which everyone has’.

10.2 Choosing and pinning a base image

10.2.1 The rocker images

The rocker project (Boettiger & Eddelbuettel, 2017) maintains the standard R images at rocker-project.org. Each builds on the one before it and is tagged with its R version:

Table 10.2: Base images for R. Sizes are approximate and uncompressed, as observed for recent tags.
Image Adds Use when
rocker/r-ver R on Ubuntu LTS (about 0.8 GB) renv-based projects that install their own packages; continuous integration
rocker/rstudio RStudio Server Interactive work in a browser
rocker/tidyverse The tidyverse and common dependencies (about 3 GB) Teaching, and quick starts where most projects use the tidyverse
rocker/verse TeX and publishing tools Rendering PDF documents
rocker/geospatial GDAL, GEOS, and PROJ, the spatial system libraries Spatial analysis
bioconductor/bioconductor_docker Bioconductor’s toolchain Genomics

Pick the smallest base that has your prerequisites. A larger base is convenient during development. It costs build time and disk, and it gives more things the chance to change underneath you. For a compendium whose packages are pinned by renv, rocker/r-ver is the right base. The lockfile, not the image, decides the packages.

10.2.2 Tags, digests, and what each pins

FROM rocker/r-ver:4.4.0 pins the R version. FROM rocker/r-ver:latest drifts. Neither pins the bytes. Only a digest does:

FROM rocker/r-ver:4.4.0@sha256:73adabf378930f8744f56bac1fe5e99870532eb4ced1a15075991bac909d21f3

The same reasoning applies to system packages installed with apt-get. Either pin their versions, or accept that a rebuild months later installs newer ones. A digest-pinned base fixes the packages that come with the base. It cannot fix what a later apt-get update fetches.

TipVerification

Find the digest of a tag you have pulled:

docker inspect --format '{{index .RepoDigests 0}}' rocker/r-ver:4.4.0

It prints rocker/r-ver@sha256: followed by 64 hexadecimal characters. Paste the hash after the tag in your FROM line, as above, and rebuild. The build log’s FROM step now names the digest.

10.2.3 Exercises

  1. Choose a base image from Table 10.2 for each project.
    1. A trial analysis whose renv.lock pins every package, rendered to HTML. (b) The same analysis, rendered to PDF.
    2. A course in which forty students need RStudio in a browser on the first day. (d) A spatial analysis of ZIP-code-level readmission rates with sf.
  1. rocker/r-ver at the R version in the lockfile. The lockfile supplies the packages, so a larger base only adds things to drift.
  2. Still rocker/r-ver, plus quarto install tinytex in the Dockerfile (Section 10.3). rocker/verse also works and is larger. (c) rocker/tidyverse, which includes RStudio Server and the packages a course needs. The first day is then not spent installing. (d) rocker/r-ver with the spatial system libraries added to the apt-get line, as in the worked example (Section 10.7), or rocker/geospatial if you prefer to inherit them.

Question. Your Dockerfile begins FROM rocker/verse:4.4.1. That is an exact version, not latest. Does rebuilding it in two years give you the same image?

Answer. Probably not. A tag is a movable label, not a fixed identity. 4.4.1 pins the R version. However, the maintainers can and do rebuild that tag against a newer Ubuntu base, refreshed apt packages, and patched system libraries. Your FROM line resolves to whatever the tag points at today. The digest cannot change. If it resolves at all, it resolves to exactly the bytes you built against.

A pinned digest has its own cost. It never picks up a security patch, so it ages into a liability, and the hash is unreadable in a diff. The usual compromise is a version tag during development and a digest recorded at the moment of submission. The digest is recorded alongside the sap-v1.0 tag of Chapter 26. The archived artifact is then reconstructible even though day-to-day work is not frozen. Archiving the built image (Section 10.5) avoids the question entirely.

10.3 A research-compendium Dockerfile

This is the book’s one canonical Dockerfile, and Chapter 9 and Section 11.1 use it unchanged. It assumes three things. The first is a compendium with the paper at analysis/paper/, the layout that Section 11.1, the next chapter, sets out in full. The second is a renv.lock that includes rmarkdown and knitr (Quarto needs both to run R chunks). The third is the .Rprofile and renv/activate.R that renv::init() writes.

# syntax=docker/dockerfile:1
FROM rocker/r-ver:4.4.0

ARG QUARTO_VERSION=1.6.40

RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    ca-certificates curl \
    libcurl4-openssl-dev libssl-dev libxml2-dev libuv1-dev \
  && rm -rf /var/lib/apt/lists/*

# rocker/r-ver ships R but not the Quarto CLI
RUN arch="$(dpkg --print-architecture)" \
  && curl -fsSL -o /tmp/quarto.deb \
    "https://github.com/quarto-dev/quarto-cli/releases/download/v${QUARTO_VERSION}/quarto-${QUARTO_VERSION}-linux-${arch}.deb" \
  && dpkg -i /tmp/quarto.deb \
  && rm /tmp/quarto.deb

# outside /work, so a bind mount of the project cannot hide it
ENV RENV_PATHS_LIBRARY=/opt/renv/library

WORKDIR /work
COPY renv.lock .Rprofile ./
COPY renv/activate.R renv/activate.R
RUN R -q -e "renv::restore(prompt = FALSE)"

COPY . .

CMD ["quarto", "render", "analysis/paper/paper.qmd"]

10.3.1 Three details that are easy to get wrong

  • Quarto is not in the base image. rocker/r-ver does not include the Quarto command line interface (CLI), so the image installs a pinned release. Without it, quarto render fails.
  • The render command is quarto render. rmarkdown::render() is the R Markdown renderer and does not process a .qmd project.
  • The base tag must match the R version in renv.lock. Otherwise renv restores the right packages onto the wrong R.

For the strongest pin, replace the tag with the digest, as in Section 10.2. The image above renders HTML. For PDF output, add RUN quarto install tinytex --no-prompt after the Quarto layer.

WarningPitfall

A base image that does not match the lockfile builds without error. renv::restore() installs the recorded package versions onto whatever R the image has. It notes the mismatch in one line of a build log nobody reads (Using R 4.4.0 (lockfile was generated with R 4.6.1)), and the image renders. The packages are pinned and R is not, which is the gap the opening described. Read the R version from the lockfile before you write the FROM line: renv::lockfile_read('renv.lock')$R$Version.

10.3.2 .dockerignore

Next to the Dockerfile, commit a .dockerignore. It keeps the local package library, credentials, and other machine-specific files out of the build context. They therefore stay out of an image you may later push to a public registry:

.git/
.Rproj.user/
.Rhistory
.RData
.Renviron
.env
secrets/
renv/library/
renv/staging/
renv/local/
_freeze/

Without it, COPY . . copies your .Renviron, with whatever API tokens it holds, into a layer of the image. Deleting the file in a later layer does not remove it from the earlier one.

10.3.3 Layer order

The order of the Dockerfile is lockfile, then restore, then project source, and Figure 10.2 shows why. Docker caches each layer and reuses it on the next build unless that layer or something above it has changed. We therefore copy the lockfile and restore packages before copying the frequently edited manuscript. An ordinary day of writing then invalidates only the fast final COPY, and the slow package-restore layer is left untouched.

flowchart TD
  A["FROM rocker/r-ver<br/><i>cached base image</i>"] --> B["RUN apt-get system deps<br/><i>rarely changes</i>"]
  B --> C["COPY renv.lock + restore<br/><i>changes when packages change</i>"]
  C --> D["COPY . /work<br/><i>changes every edit</i>"]
  D --> E["CMD quarto render"]
  style C fill:#e8f4f4,stroke:#2c7a7b
  style D fill:#fbe9e7,stroke:#9d2235
Figure 10.2: Docker layer caching. Slow, slow-changing layers go early; the fast, frequently-edited source goes last. Editing the manuscript invalidates only the final COPY, so the expensive package restore is reused from cache.

10.3.4 Building and running

From the compendium root (display-only):

docker build --platform linux/amd64 -t readmissions:v1.0 .
docker run --rm --platform linux/amd64 \
  -v "$(pwd)/analysis/paper:/work/analysis/paper" \
  readmissions:v1.0

quarto render writes its output next to the source, in /work/analysis/paper/, so that is the directory to mount. The rendered file then appears in your host’s analysis/paper/. A mount at any other container path receives nothing.

Data involve a similar choice. Copying data into the image makes it self-contained and larger. Mounting them keeps the image small, but the image alone then no longer reproduces the analysis. Embed small, shareable data, and mount large or sensitive data. Never build an image containing protected health information that might reach a registry.

TipVerification

Check the cache and the environment, not only that the build succeeded:

docker build --progress=plain --platform linux/amd64 \
  -t readmissions:v1.0 . 2>&1 | grep -A1 'renv::restore'
docker run --rm --platform linux/amd64 readmissions:v1.0 R --version
docker run --rm --platform linux/amd64 readmissions:v1.0 \
  Rscript -e 'renv::status()'

After you edit a .qmd file and rebuild, the line after the renv::restore step should read CACHED. If the step runs again, the layer order is wrong. R --version should match the lockfile’s R$Version, and renv::status() should report no issues. A mismatched base tag shows up here as a status message that the lockfile was generated with a different R.

ImportantLLM prompt

Context. My compendium’s renv.lock records R <output of renv::lockfile_read('renv.lock')$R$Version> and the packages listed below. The paper renders with quarto render analysis/paper/paper.qmd. The project tree is: <output of tree -L 2>.

Constraints. Base the image on rocker/r-ver at the tag equal to that R version, with a FROM line pinned by digest (I will fill in the digest from docker inspect; do not invent one). Install a pinned Quarto CLI release. Copy renv.lock, .Rprofile, and renv/activate.R and restore before copying the project. Write a .dockerignore that excludes .git/, .Renviron, .env, and renv/library/. Add only the system libraries my packages need, and say which package needs each.

Criteria. The image builds with --platform linux/amd64; after I edit a .qmd file and rebuild, the restore step is CACHED; inside the container, R --version matches the lockfile and renv::status() reports no issues.

Failure mode. Models write a base tag from their training data (rocker/r-ver:4.4.0, or latest) that does not match your lockfile. They also invent a plausible-looking digest, omit the .dockerignore, or copy the whole project before the restore, so that every edit reinstalls every package. Check. Run the Verification above, and compare the draft line by line with the canonical Dockerfile. The prompt carries file names and package names, not data (Section 12.3).

10.3.5 Exercises

  1. A colleague’s Dockerfile reads, in order: FROM rocker/r-ver:latest; COPY . /work; RUN R -q -e "renv::restore(prompt = FALSE)"; CMD ["Rscript", "-e", "rmarkdown::render('analysis/paper/paper.qmd')"]. There is no .dockerignore. List every problem and the fix for each.
  • latest drifts: use the tag equal to the lockfile’s R version, pinned by digest at submission.
  • COPY . /work before the restore invalidates the restore layer on every edit: copy renv.lock, .Rprofile, and renv/activate.R first, restore, then COPY . ..
  • Without a .dockerignore, the build context includes renv/library/ (slow, and the wrong platform’s binaries) and any .Renviron or .env, which then sit in a layer of an image that may be pushed publicly.
  • rmarkdown::render() does not render a Quarto project, and rocker/r-ver has no Quarto: install a pinned Quarto CLI and use CMD ["quarto", "render", "analysis/paper/paper.qmd"].
  • With the library inside /work, a bind mount of the project would hide it: set RENV_PATHS_LIBRARY outside /work.

The canonical Dockerfile above is the corrected file.

Question. You add one package to renv.lock and rebuild. Which layers rebuild, and why is that correct?

Answer. The two COPY layers that bring in renv.lock and its bootstrap files change. The restore layer and everything after it therefore rebuild. The base image, the system libraries, and the Quarto install are reused from cache. That is the intended behavior. A dependency change should reinstall packages, and an edit to the manuscript should not.

10.4 Interactive and multi-service containers

10.4.1 RStudio Server in a container

For interactive work in a pinned environment, run RStudio Server from a rocker image (display-only):

docker run -d --rm --platform linux/amd64 \
  -p 127.0.0.1:8787:8787 \
  -e PASSWORD="$RSTUDIO_PASSWORD" \
  -v "$(pwd)":/home/rstudio/project \
  --name analysis-rstudio \
  rocker/tidyverse:4.4.0

Set RSTUDIO_PASSWORD in your shell first, rather than typing a password into a command that is saved in your shell history. Binding to 127.0.0.1 keeps the server off your network. A bare -p 8787:8787 publishes it on every interface, so anyone who can reach your machine can try the login. Then open http://localhost:8787 and log in as rstudio. Your project is mounted under /home/rstudio/project. Inside the container, run renv::restore() to install its packages. The pattern suits teaching (every student gets the same environment) and collaborative analysis.

TipVerification

Confirm where the server is listening:

docker port analysis-rstudio

It should print 8787/tcp -> 127.0.0.1:8787. An address of 0.0.0.0 means the server is reachable from your network. In that case, stop the container and restart it with the 127.0.0.1: prefix.

10.4.2 Several services: a pointer to Compose

When an analysis needs a second service, such as a PostgreSQL database, Docker Compose starts several containers from one compose.yaml file. Three rules prevent most of its problems. Read passwords from a .env file beside the Compose file (POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set it in .env}), list .env in both .gitignore and .dockerignore, and never write a password into the committed file. Use the docker compose subcommand. The hyphenated docker-compose binary is Compose version 1, which reached end of life in 2023. The top-level version: key that older examples show is obsolete. Docker’s Compose documentation covers the rest. For most biostatistical analyses a single container is enough.

10.5 Sharing and archiving an image

Push the image to a registry so that collaborators can pull it (display-only):

docker tag readmissions:v1.0 ghcr.io/username/readmissions:v1.0
docker push ghcr.io/username/readmissions:v1.0

# a reader
docker pull --platform linux/amd64 ghcr.io/username/readmissions:v1.0
docker run --rm --platform linux/amd64 ghcr.io/username/readmissions:v1.0

Docker Hub works the same way with username/readmissions:v1.0. A registry provides hosting, not archiving. Accounts lapse, free tiers change, and images are deleted. For the archival copy, save the built image as a file and deposit it with the compendium. One option is Zenodo, which issues a DOI (digital object identifier):

docker save readmissions:v1.0 | gzip > readmissions-v1.0.tar.gz
# upload readmissions-v1.0.tar.gz to Zenodo with the compendium

The saved image is the environment itself, not instructions for rebuilding it. It therefore survives the base image’s tag being rebuilt, the apt mirrors moving on, and the registry disappearing. Record the image ID and the base digest in the compendium’s README at submission.

TipVerification

Confirm that the archive restores the image you built:

docker image inspect --format '{{.Id}}' readmissions:v1.0
docker rmi readmissions:v1.0
docker load -i readmissions-v1.0.tar.gz
docker image inspect --format '{{.Id}}' readmissions:v1.0

docker load reads the gzipped file directly. The two IDs must be identical. Then run the loaded image and compare the rendered paper with yours.

Question. A reviewer asks you to reproduce a table two years after submission. You have the Dockerfile with a version tag, and the image on a registry under an account that has lapsed. What can you reproduce, and what would have made it exact?

Answer. You can rebuild from the Dockerfile. However, the tag may now point at a different base, and apt-get will fetch newer system libraries. You get a close environment, not the same one. The exact environment needed one of two things. The first is the archived image file, which restores the identical bytes. The second, at least, is a digest-pinned FROM line, which fixes the base. With either, the R build and system libraries are those that produced the table.

10.6 Diagnosing build failures

Most build failures are one of five kinds, and each has a recognizable symptom:

Table 10.3: Docker build failures.
Symptom Cause Fix
cannot find -lcurl, fatal error: openssl/ssl.h, or a configure error naming a library A system library is missing for a package that compiles from source (curl, xml2, openssl, httr2, sf are common) Add the matching lib*-dev package to the apt-get install line
Every edit reinstalls every package The project is copied before the restore Copy the lockfile and bootstrap files first (Figure 10.2)
The image is gigabytes larger than expected apt-get without --no-install-recommends, or the apt lists left behind Use the install-and-clean pattern of the canonical Dockerfile
exec format error on a server Built for arm64 on Apple Silicon, run on x86_64 Build with --platform linux/amd64, or build for both
renv::status() in the container reports a different R The base tag does not match the lockfile Set the FROM tag to the lockfile’s R$Version

To build one image for both architectures and push it (display-only):

docker buildx build --platform linux/amd64,linux/arm64 \
  -t ghcr.io/username/readmissions:v1.0 --push .
ImportantLLM prompt

Context. A docker build of the Dockerfile below fails. Here are the Dockerfile and the last 40 lines of the build log, from the first line containing ERROR or error:.

Constraints. Do not change the base image, the layer order, or the R version, and do not suggest installing packages outside renv. Propose the smallest change.

Criteria. Name the R package whose installation failed, the system library it needs, and the Ubuntu package that provides that library, quoting the log line that shows it.

Failure mode. A model that cannot read the cause from a truncated log proposes a different base image or a long list of lib*-dev packages. This makes the build pass and the image larger without saying why. Check. Rebuild with the one added package. The failing step should now complete, and the log line the model quoted should be the one that named the missing library.

10.6.1 Exercises

  1. Map each error to the Ubuntu package that fixes it.
    1. configuration failed for package 'xml2' with libxml-2.0 was not found. (b) gdal-config: not found while installing sf. (c) ERROR: dependency 'openssl' is not available, preceded by ssl.h: No such file or directory.
  1. libxml2-dev. (b) libgdal-dev. sf also needs libproj-dev and libgeos-dev, so add all three, as in the worked example. (c) libssl-dev. The canonical Dockerfile already installs (a) and (c), so a log showing either means the line was edited away. In each case the log names the library or the header file. The fix is the Ubuntu -dev package that provides it.

Question. An image built on your Apple Silicon Mac runs there and fails on the department’s Linux server with exec format error. Your colleague suggests reinstalling Docker on the server. What is the actual cause?

Answer. The image was built for arm64, your laptop’s architecture, and the server is x86_64. Docker is working correctly on both machines. Rebuild with --platform linux/amd64, or build a multi-architecture image with docker buildx.

10.7 Worked example: a dockerized compendium

Start from the canonical Dockerfile. A compendium that also uses sf needs the geospatial system libraries, so extend the apt-get line and change nothing else:

RUN apt-get update \
  && apt-get install -y --no-install-recommends \
    ca-certificates curl \
    libcurl4-openssl-dev libssl-dev libxml2-dev libuv1-dev \
    libgdal-dev libproj-dev libgeos-dev \
  && rm -rf /var/lib/apt/lists/*

Build, run, and share (display-only):

docker build --platform linux/amd64 -t readmissions:v1.0 .
docker run --rm --platform linux/amd64 \
  -v "$(pwd)/analysis/paper:/work/analysis/paper" \
  readmissions:v1.0
docker tag readmissions:v1.0 ghcr.io/username/readmissions:v1.0
docker push ghcr.io/username/readmissions:v1.0

A reader, given the GitHub repository of the compendium, runs:

git clone https://github.com/username/readmissions.git
cd readmissions
docker build --platform linux/amd64 -t readmissions:v1.0 .
docker run --rm --platform linux/amd64 \
  -v "$(pwd)/analysis/paper:/work/analysis/paper" \
  readmissions:v1.0

The paper renders. The reader now has the same R version and the same package versions. The system libraries are those current on the day the reader built the image. This is close to your environment but not identical to it, since the tag may have been rebuilt in the meantime (Section 10.2). For an identical environment, give the reader the digest-pinned Dockerfile or the archived image.

TipVerification

The reader’s check is the one the opening needed, which is the same number. Compare the key estimate in the reader’s rendered paper with the one in yours, and compare the environments behind them:

docker run --rm --platform linux/amd64 readmissions:v1.0 \
  Rscript -e 'sessionInfo()'

The R version, platform, and BLAS (Basic Linear Algebra Subprograms) library lines should match the ones recorded at submission. If the estimates differ and the environments match, the cause is in the code or the data, not the machine.

10.8 An aside: Nix and rix

Everything above pins reproducibility with a lockfile and a container image. This is the mainstream path for an R research compendium and the one this book teaches. A smaller but active community takes a different route to the same goal. That route is Nix, a package manager and build system built around the idea that a build should be a pure function of its declared inputs. Recall that apt-get install or a base image’s own history can change what FROM rocker/r-ver:4.4.0 resolves to over time (Section 10.2). A Nix build, by contrast, specifies every dependency, down to the compiler toolchain, as an explicit input. It stores each built package in a content-addressed store keyed by the hash of its own inputs. Two builds from the same Nix expression on two machines produce bit-identical results. In principle this needs no container at all, because the isolation Nix relies on is the content-addressed store rather than a container runtime.

The R package rix (Rodrigues & Baumann, 2024) is maintained by Bruno Rodrigues. As of this writing it is on CRAN and under active development (incubated through rOpenSci). It exists to make the Nix approach reachable from R without writing Nix’s own expression language by hand. Its central function, rix(), takes an R version, a vector of packages, and optionally system dependencies. It writes a default.nix file describing the environment, and the Nix tool then builds exactly that environment from the file. The pattern is shown below. It is display-only, since it needs Nix installed, and this book’s environment does not have Nix:

library(rix)
rix(
  r_ver = "4.4.0",
  r_pkgs = c("dplyr", "ggplot2"),
  system_pkgs = NULL,
  project_path = ".",
  overwrite = TRUE
)
# writes default.nix; then, in a shell with Nix installed:
# nix-build  # or: nix-shell, for an interactive session

A Nix-built environment can run inside a Docker container as well as outside one. The two technologies answer slightly different questions. Nix answers ‘exactly which inputs built this package’, and Docker answers ‘what filesystem and process isolation does this run under’. Some reproducibility-focused teams layer them. They use rix to generate the environment description and a thin Dockerfile only to package it for a system that expects a container image. That combination is outside the scope of this aside. We present rix and Nix as a documented, maintained alternative to the renv-plus-Docker path this book teaches, without recommending either one on its own.

On the plus side, weighed against renv plus Docker, Nix offers a stronger reproducibility guarantee in principle. The whole dependency closure, not just the OS and R packages, is pinned and content-addressed. Nix also has no dependency on a container daemon. That matters on systems where Docker itself is unavailable or unwanted. The clusters of Section 25.2 are exactly such a system, though they typically lack Nix as well. On the minus side, the learning curve is steep. The Nix expression language, the distinction between channels and flakes, and the unfamiliar error messages when a build fails are a larger investment than learning Docker. The R community’s documentation and troubleshooting resources for Nix are also thinner than for renv and Docker. This book assumes you will use both of those day to day. Some reproducibility-focused groups use rix and Nix instead of renv plus Docker for these reasons, particularly where Nix is already part of the organization’s infrastructure. For most research compendia starting from nothing, the mainstream path of this book remains the one to learn first. A future collaborator may hand you a default.nix instead of a Dockerfile. If so, bring the same diagnostic instinct this chapter teaches for Docker. Ask what each declared input pins, and read the build log for the dependency that failed to resolve. Distrust a cached result that was built from a different declared input than the one in front of you.

Question. A collaborator who already knows Nix suggests replacing this project’s Dockerfile with a default.nix generated by rix. What would the team gain, and what would it need to learn that it does not already know from this chapter?

Answer. It would gain a stronger, content-addressed pin on every dependency, including the compiler toolchain. It would also gain the ability to reproduce the environment without a container runtime at all. It would need to learn the Nix expression language well enough to debug a failed build. The diagnostic habits of Section 10.6 (reading an apt-get error, matching a missing library to a Ubuntu package) do not transfer directly to Nix’s own error messages and build logs.

10.9 Rules of thumb

  1. Habit 4: pin the environment. The lockfile pins the packages. The image pins R, the system libraries, and the OS. Only a digest or an archived image pins the bytes.
  2. Habit 8: a plausible number is the expensive bug. A different R or BLAS produces a different number without an error. Check the R version and renv::status() inside the container, not only that the build succeeded.
  3. Habit 3: tag what you submit. Record the base digest and the image ID with the Git tag, and archive the image beside the compendium. A registry provides hosting, not archiving.

10.10 Cumulative practice

  1. Cumulative (uses Chapter 9). Take a project with a committed renv.lock. Write its Dockerfile from the canonical one, with the FROM tag read from the lockfile and pinned by digest, and a .dockerignore. Build it, edit a .qmd file, rebuild, and record the evidence that the restore layer was cached. Then run renv::status() and R --version inside the container.
  2. On your own work (no key). Dockerize an analysis of yours that you will need to reproduce after submission. Archive the image with docker save, remove it, load it again, and confirm the image ID and a key number in the output match.
  1. The evidence is the build log line after the renv::restore step reading CACHED on the second build, an R --version matching renv::lockfile_read('renv.lock')$R$Version, and renv::status() reporting no issues. If status() reports that the lockfile was generated with a different R, the FROM tag is wrong. If packages are reported as not installed, the restore failed partway, usually for a missing system library that the build log names (Table 10.3).

10.11 Cheat sheet

Table 10.4: Docker at a glance.
Task Command Guards against
Run R in a pinned image docker run --rm --platform linux/amd64 rocker/r-ver:4.4.0 R Your laptop’s R leaking into results
Find a digest docker inspect --format '{{index .RepoDigests 0}}' <image> A tag that moves
Build for the server docker build --platform linux/amd64 -t name:v1 . exec format error
Check the cache docker build --progress=plain ... Reinstalling every package on every edit
Check the environment docker run ... Rscript -e 'renv::status()' Right packages on the wrong R
Keep secrets out .dockerignore with .Renviron, .env, renv/library/ Credentials in a pushed image
RStudio, local only -p 127.0.0.1:8787:8787 A login page open to the network
Share docker push ghcr.io/<user>/<image>:<tag> Collaborators rebuilding by hand
Archive docker save, compressed with gzip A registry or base tag that changes
Restore an archive docker load -i image.tar.gz Unverified archives

10.12 Further reading

  • (Boettiger, 2015), the motivating paper for containers in reproducible research.
  • (Boettiger & Eddelbuettel, 2017), the rocker project and its R images.
  • Docker’s documentation at docs.docker.com, in particular the Dockerfile reference and the build-cache guide.

10.13 Prerequisites answers

  1. A Docker image is an immutable read-only template that includes an OS, system libraries, and application code. A container is a running instance of an image, with its own writable filesystem layer. Many containers can be run from one image; removing a container does not modify the image. The image is the artifact; the container is the process.
  2. renv captures R packages only. Docker captures the operating system, system libraries (BLAS, geospatial libraries, compilers), the R build itself, and R packages. When an analysis depends on a specific OS or system library, Docker comes far closer than renv alone to reproducing it years later. A digest-pinned or archived image comes closer still. For pure-R analyses renv may suffice. For anything with non-R dependencies, Docker is the prudent choice.
  3. rocker/tidyverse:4.4.0 provides Ubuntu LTS (the base of the rocker/r-ver images it builds on; 22.04 for this tag), R 4.4.0, RStudio Server, and the tidyverse packages pre-installed. A container starts immediately without package installation, which suits teaching and quick collaborative work. It is larger than rocker/r-ver. The trade-off is justified when most projects use the tidyverse anyway.