Appendix A — Alternative Wrangling Paradigms

NoteWhy this appendix exists

The practicum teaches the tidyverse as its default wrangling stack, because that is the idiom most biostatistics students and collaborators read and write in 2026. A meaningful fraction of R code in the wild, however, uses different tools: base R (always), data.table (industry, performance-critical code, several CRAN packages internally), arrow (Parquet and Python interop), and, more recently, polars for R.

The goal here is reading-level fluency, not conversion. A finishing biostatistics student should be able to open an R script written by someone else and recognize whatever paradigm the author chose. Writing idiomatic data.table is a separate investment. This appendix does not claim to make you fluent, only literate.

A.1 Four paradigms

Paradigm Strength Typical use
base R no dependencies; works anywhere; ubiquitous small scripts, package code, teaching
tidyverse readable pipelines; coherent ecosystem most applied biostatistics, this book
data.table fastest; memory-efficient; terse large data, production pipelines, pharma
arrow lazy; Parquet; Python interop columnar data, cross-language projects
polars fast, Python-inspired; young experimental as of 2026

A.2 base R wrangling

Before the tidyverse stabilized in 2016, essentially all R data manipulation was base R. It is still the lingua franca of package development, and a student who cannot read it is cut off from most of CRAN.

The four idioms that matter:

df <- data.frame(
  age    = c(17, 42, 64, 71, 55),
  sex    = c('F', 'M', 'F', 'M', 'F'),
  race   = c('A', 'B', 'A', 'C', 'B'),
  weight = c(58, 82, 71, 68, 90),
  height = c(160, 178, 165, 172, 169)
)

# Subset rows: logical, numeric, or negative-numeric indexing
adults <- df[df$age >= 18, ]

# Select columns: character vector or dollar sign
demog  <- df[, c('age', 'sex', 'race')]
ages   <- df$age

# Add or modify: assign into a column
df$bmi <- df$weight / (df$height / 100)^2

# Group and summarize: aggregate() or split()+sapply()
aggregate(bmi ~ sex, data = df, FUN = mean)
#>   sex      bmi
#> 1   F 26.74891
#> 2   M 24.43298

Base R is verbose but predictable: no hidden NSE, no pipe precedence surprises, and you can always fall back to [.data.frame semantics.

A.3 data.table

data.table (Barrett et al., 2026) is the high-performance alternative. Its syntax is a compressed DT[i, j, by] form that takes roughly a week to become fluent in but is dramatically faster than base or dplyr on any dataset above a few million rows. All modern clinical-trials R packages (admiral, pharmaverseadam) use data.table internally; so does tidymodels for its cross-validation machinery.

The three mental pieces:

  • i selects rows (like filter).
  • j computes columns or does assignment (like select plus mutate plus summarize).
  • by groups (like group_by).
library(data.table)
dt <- as.data.table(df)

# Filter
dt[age >= 18]
#>      age    sex   race weight height      bmi
#>    <num> <char> <char>  <num>  <num>    <num>
#> 1:    42      M      B     82    178 25.88057
#> 2:    64      F      A     71    165 26.07897
#> 3:    71      M      C     68    172 22.98540
#> 4:    55      F      B     90    169 31.51150

# Select / compute
dt[, .(age, bmi = weight / (height / 100)^2)]
#>      age      bmi
#>    <num>    <num>
#> 1:    17 22.65625
#> 2:    42 25.88057
#> 3:    64 26.07897
#> 4:    71 22.98540
#> 5:    55 31.51150

# Assign in place (the := operator)
dt[, bmi := weight / (height / 100)^2]

# Group and summarize
dt[, .(mean_bmi = mean(bmi, na.rm = TRUE)), by = sex]
#>       sex mean_bmi
#>    <char>    <num>
#> 1:      F 26.74891
#> 2:      M 24.43298

# Chaining
dt[age >= 18][, .(mean_bmi = mean(bmi)), by = sex]
#>       sex mean_bmi
#>    <char>    <num>
#> 1:      M 24.43298
#> 2:      F 28.79524

Four features of this syntax repay explanation. First, the := operator assigns by reference: it modifies the data.table in place rather than returning a copy, which is what lets data.table avoid the copy-on-modify cost that both dplyr and base R pay on every mutation. On a million-row cohort this is the difference between an instantaneous update and a multi-second one. Second, keyby behaves like by but additionally sorts the result by the grouping variable and sets it as a key, which speeds subsequent joins on that key. Third, the .SD idiom (‘Subset of Data’) is how one applies a function across many columns at once, as in dt[, lapply(.SD, mean), by = sex, .SDcols = is.numeric]; it is the data.table analog of across(). Fourth, a gotcha worth internalizing early: data.table returns a data.table when you subset with [, where a data.frame would sometimes drop to a vector. The drop = TRUE surprise that catches base-R users does not arise, which is a mercy, but code that assumed a vector will need adjusting.

A.4 Side-by-side: eight common operations

Operation tidyverse data.table base R
Filter rows df \|> filter(age >= 18) dt[age >= 18] df[df$age >= 18, ]
Select columns df \|> select(age, sex) dt[, .(age, sex)] df[, c('age', 'sex')]
Add a column df \|> mutate(bmi = wt / ht^2) dt[, bmi := wt / ht^2] df$bmi <- df$wt / df$ht^2
Group and summarize df \|> group_by(sex) \|> summarize(m = mean(x)) dt[, .(m = mean(x)), by = sex] aggregate(x ~ sex, df, mean)
Sort df \|> arrange(age) dt[order(age)] df[order(df$age), ]
Rename df \|> rename(new = old) setnames(dt, 'old', 'new') names(df)[names(df) == 'old'] <- 'new'
Inner join inner_join(x, y, by = 'id') x[y, on = 'id', nomatch = NULL] merge(x, y, by = 'id')
Pivot wide to long pivot_longer(df, cols = v1:v3) melt(dt, measure.vars = v1:v3) reshape(df, direction = 'long', ...)

A.5 arrow

The arrow package (Richardson et al., 2026) provides a lazy, columnar back-end to the tidyverse. You write dplyr verbs against an Arrow Dataset object, which plans the computation as a directed acyclic graph and executes it only when you call collect(). This makes it possible to run dplyr-style pipelines over datasets that do not fit in memory, or to read Parquet files at speeds that exceed native data.frame for large data.

library(arrow)
library(dplyr)

# A stand-in for the Parquet dataset the text describes
trial_dir <- file.path(tempdir(), 'big-trial')
data.frame(
  site  = rep(c('SITE01', 'SITE02'), each = 4),
  visit = rep(c('Baseline', 'Week12'), 4),
  arm   = rep(c('Placebo', 'Active'), 4),
  age   = c(61, 64, 58, 70, 66, 59, 72, 63)
) |>
  write_dataset(trial_dir)

# Open a Parquet dataset lazily (no RAM cost)
ds <- open_dataset(trial_dir)

# dplyr pipeline against the lazy dataset
summary <- ds |>
  filter(site == 'SITE01', visit == 'Baseline') |>
  group_by(arm) |>
  summarize(mean_age = mean(age), n = n()) |>
  collect()

summary
#> # A tibble: 1 × 3
#>   arm     mean_age     n
#>   <chr>      <dbl> <int>
#> 1 Placebo     59.5     2

Use arrow when: (a) data is stored as Parquet or partitioned Parquet, (b) total data exceeds RAM, (c) you work with a Python colleague who sends Arrow-native datasets. Skip arrow for small single-file analyses; the overhead exceeds the benefit.

A.6 polars for R

polars is a Rust-based data-frame library, originally Python-first, with an R binding released in 2023. As of April 2026 the R binding is usable but not mature enough to recommend as a default. It is worth tracking for the combination of data.table-level speed with a syntax that is closer to dplyr.

# Illustrative; API may change
library(polars)
df <- pl$DataFrame(mtcars)
df$filter(pl$col('mpg') > 20)$select('mpg', 'hp')

For now, prefer data.table if you need speed or arrow if you need lazy evaluation and Parquet. Revisit polars in 2027.

A.7 When to reach for each

  • tidyverse, the default for applied biostatistics, this book’s default, and what most collaborators will send you.
  • base R, for package development, for reading older code, and when minimizing dependencies matters (e.g., a Shiny app served from a container you want to keep small).
  • data.table, when a dataset is millions of rows or when pipeline speed is a bottleneck; when reading code from industry pharma or CRO colleagues; when a CRAN package you depend on uses it internally and the underlying layout shows through.
  • arrow, when data is columnar, large, or cross-language.
  • polars, not yet, in R, as of early 2026.

A.8 Exercises

On your own machine, data, or collaborations. These exercises time code on your own hardware or read package source, so no answer key is possible. The comparison, not the number, is the point.

  1. Take a wrangling pipeline from Chapter 18 and rewrite it three ways: in data.table, in base R, and with arrow against a Parquet file. Time all three on a 1-million-row version of the data.
  2. Read the source of one function in admiral (a pharmaverse ADaM package). Identify the data.table idioms used (:=, .SD, keyby). Write a one- paragraph summary of what the function does in plain English.
  3. Install arrow. Write a 10-million-row data.frame as Parquet. Read it back with open_dataset() and run a dplyr-style aggregation. Compare wall-clock time to the equivalent readr::read_csv() + dplyr::group_by() |> summarize() path.