Appendix A — Alternative Wrangling Paradigms

NoteWhy this appendix exists

The practicum teaches the tidyverse as its default wrangling stack, because that is the idiom most biostatistics students and collaborators read and write in 2026. However, a meaningful fraction of R code in the wild uses different tools. These are base R (always), data.table (industry, performance-critical code, several CRAN packages internally), arrow (Parquet and Python interop), and, more recently, polars for R.

The goal here is reading-level fluency. A finishing biostatistics student should be able to open an R script written by someone else and recognize whatever paradigm the author chose. Writing idiomatic data.table is a separate investment that this appendix does not attempt.

A.1 Five paradigms

Paradigm Strength Typical use
base R no dependencies; works anywhere; ubiquitous small scripts, package code, teaching
tidyverse readable pipelines; coherent ecosystem most applied biostatistics, this book
data.table fastest; memory-efficient; terse large data, production pipelines, pharma
arrow lazy; Parquet; Python interop columnar data, cross-language projects
polars fast, Python-inspired; young experimental as of early 2026

A.2 base R wrangling

Before the tidyverse stabilized in 2016, essentially all R data manipulation was base R. It is still the common language of package development. A student who cannot read it is cut off from most of CRAN.

Four idioms cover most of it:

df <- data.frame(
  age    = c(17, 42, 64, 71, 55),
  sex    = c('F', 'M', 'F', 'M', 'F'),
  race   = c('A', 'B', 'A', 'C', 'B'),
  weight = c(58, 82, 71, 68, 90),
  height = c(160, 178, 165, 172, 169)
)

# Subset rows: logical, numeric, or negative-numeric indexing
adults <- df[df$age >= 18, ]

# Select columns: character vector or dollar sign
demog  <- df[, c('age', 'sex', 'race')]
ages   <- df$age

# Add or modify: assign into a column
df$bmi <- df$weight / (df$height / 100)^2

# Group and summarize: aggregate() or split()+sapply()
aggregate(bmi ~ sex, data = df, FUN = mean)
#>   sex      bmi
#> 1   F 26.74891
#> 2   M 24.43298

Base R is verbose but predictable: no hidden NSE, no pipe precedence surprises, and you can always fall back to [.data.frame semantics.

A.3 data.table

data.table (Barrett et al., 2026) is the high-performance alternative. Its syntax is a compressed DT[i, j, by] form that takes roughly a week to become fluent in. In return, it is much faster than base R or dplyr on any dataset above a few million rows. It is common in industry code and inside a number of CRAN packages. The pharmaverse is not among them: admiral and its companions are built on dplyr, tidyr, and purrr, so data.table fluency is not a prerequisite for reading them.

The three mental pieces:

  • i selects rows (like filter).
  • j computes columns or does assignment (like select plus mutate plus summarize).
  • by groups (like group_by).
library(data.table)
dt <- as.data.table(df)

# Filter
dt[age >= 18]
#>      age    sex   race weight height      bmi
#>    <num> <char> <char>  <num>  <num>    <num>
#> 1:    42      M      B     82    178 25.88057
#> 2:    64      F      A     71    165 26.07897
#> 3:    71      M      C     68    172 22.98540
#> 4:    55      F      B     90    169 31.51150

# Select / compute
dt[, .(age, bmi = weight / (height / 100)^2)]
#>      age      bmi
#>    <num>    <num>
#> 1:    17 22.65625
#> 2:    42 25.88057
#> 3:    64 26.07897
#> 4:    71 22.98540
#> 5:    55 31.51150

# Assign in place (the := operator)
dt[, bmi := weight / (height / 100)^2]

# Group and summarize
dt[, .(mean_bmi = mean(bmi, na.rm = TRUE)), by = sex]
#>       sex mean_bmi
#>    <char>    <num>
#> 1:      F 26.74891
#> 2:      M 24.43298

# Chaining
dt[age >= 18][, .(mean_bmi = mean(bmi)), by = sex]
#>       sex mean_bmi
#>    <char>    <num>
#> 1:      M 24.43298
#> 2:      F 28.79524

Four features of this syntax need explanation. First, the := operator assigns by reference. That is, it modifies the data.table in place rather than returning a copy. This is what lets data.table avoid the copy-on-modify cost that both dplyr and base R pay on every mutation. On a million-row cohort this is the difference between an instantaneous update and a multi-second one. Second, keyby behaves like by but also sorts the result by the grouping variable and sets it as a key. This speeds subsequent joins on that key. Third, the .SD idiom (‘Subset of Data’) is how one applies a function across many columns at once, as in dt[, lapply(.SD, mean), by = sex, .SDcols = is.numeric]. It is the data.table analog of across(). Fourth, there is a trap worth learning early: what j returns decides the class of the result. A bare column name, dt[, age], returns a vector, just as df$age does. Wrapping it in .() or passing a character vector, dt[, .(age)] or dt[, 'age'], returns a one-column data.table. Selecting rows alone, dt[age >= 18], always returns a data.table. Code that needs one or the other should ask for it explicitly rather than rely on which form a colleague happened to write.

A.4 Side-by-side: eight common operations

Operation tidyverse data.table base R
Filter rows df \|> filter(age >= 18) dt[age >= 18] df[df$age >= 18, ]
Select columns df \|> select(age, sex) dt[, .(age, sex)] df[, c('age', 'sex')]
Add a column df \|> mutate(bmi = wt / ht^2) dt[, bmi := wt / ht^2] df$bmi <- df$wt / df$ht^2
Group and summarize df \|> group_by(sex) \|> summarize(m = mean(x)) dt[, .(m = mean(x)), by = sex] aggregate(x ~ sex, df, mean)
Sort df \|> arrange(age) dt[order(age)] df[order(df$age), ]
Rename df \|> rename(new = old) setnames(dt, 'old', 'new') names(df)[names(df) == 'old'] <- 'new'
Inner join inner_join(x, y, by = 'id') x[y, on = 'id', nomatch = NULL] merge(x, y, by = 'id')
Pivot wide to long pivot_longer(df, cols = v1:v3) melt(dt, measure.vars = patterns('^v')) reshape(df, direction = 'long', ...)

A.5 arrow

The arrow package (Richardson et al., 2026) provides a lazy, columnar back-end to the tidyverse. You write dplyr verbs against an Arrow Dataset object. The object plans the computation as a directed acyclic graph and executes it only when you call collect(). This makes it possible to run dplyr-style pipelines over datasets that do not fit in memory. It also reads Parquet files, for large data, faster than a native data.frame can.

library(arrow)
library(dplyr)

# A stand-in for the Parquet dataset the text describes
trial_dir <- file.path(tempdir(), 'big-trial')
data.frame(
  site  = rep(c('SITE01', 'SITE02'), each = 4),
  visit = rep(c('Baseline', 'Week12'), 4),
  arm   = rep(c('Placebo', 'Active'), 4),
  age   = c(61, 64, 58, 70, 66, 59, 72, 63)
) |>
  write_dataset(trial_dir)

# Open a Parquet dataset lazily (no RAM cost)
ds <- open_dataset(trial_dir)

# dplyr pipeline against the lazy dataset
summary <- ds |>
  filter(site == 'SITE01', visit == 'Baseline') |>
  group_by(arm) |>
  summarize(mean_age = mean(age), n = n()) |>
  collect()

summary
#> # A tibble: 1 × 3
#>   arm     mean_age     n
#>   <chr>      <dbl> <int>
#> 1 Placebo     59.5     2

Use arrow when: (a) data is stored as Parquet or partitioned Parquet, (b) total data exceeds RAM, (c) you work with a Python colleague who sends Arrow-native datasets. Skip arrow for small single-file analyses, where the overhead exceeds the benefit.

A.6 polars for R

polars is a Rust-based data-frame library, originally Python-first, with a younger R binding. As of early 2026 the R binding is usable but not mature enough to recommend as a default. It is worth tracking, because it combines data.table-level speed with a syntax that is closer to dplyr.

# Illustrative; API may change
library(polars)
df <- pl$DataFrame(mtcars)
df$filter(pl$col('mpg') > 20)$select('mpg', 'hp')

For now, prefer data.table if you need speed or arrow if you need lazy evaluation and Parquet. Revisit polars as its R binding matures.

A.7 When to reach for each

  • tidyverse, the default for applied biostatistics, this book’s default, and what most collaborators will send you.
  • base R, for package development, for reading older code, and when minimizing dependencies matters (e.g., a Shiny app served from a container you want to keep small).
  • data.table, when a dataset is millions of rows or when pipeline speed is a bottleneck; when reading code from industry pharma or CRO colleagues; when a CRAN package you depend on uses it internally and the underlying layout shows through.
  • arrow, when data is columnar, large, or cross-language.
  • polars, not yet, in R, as of early 2026.

A.8 Exercises

On your own machine, data, or collaborations. These exercises time code on your own hardware or read package source, so no answer key is possible. Compare the paradigms with one another; the absolute times depend on the machine.

  1. Take a wrangling pipeline from Chapter 15 and rewrite it three ways: in data.table, in base R, and with arrow against a Parquet file. Time all three on a 1-million-row version of the data.
  2. Find an installed package that imports data.table (tools::dependsOnPkgs('data.table') lists them) and read the source of one of its functions. Identify the data.table idioms used (:=, .SD, keyby). Write a one-paragraph summary of what the function does in plain English. For contrast, read one admiral derivation and note that it is written in dplyr.
  3. Install arrow. Write a 10-million-row data.frame as Parquet. Read it back with open_dataset() and run a dplyr-style aggregation. Compare wall-clock time to the equivalent readr::read_csv() + dplyr::group_by() |> summarize() path.