Behavioural reproducibility auditing in R with reproducr

You finish an analysis. The code runs. The numbers look right. But if you ran the same code again next month — after a routine update.packages() — would you get the same results?

Often, you would not. Package updates change function behaviour silently. A stochastic function called without a fixed seed returns different values every run. A sort() on character data can return different orderings depending on the system locale. None of these produce errors. None produce warnings. They just produce different numbers.

reproducr makes these risks visible before they reach a regulator, a journal, or a collaborator.

What reproducr actually does

The package is organised around three tiers, each building on the previous:

Tier	Functions	What it does
1 — Scan & score	`audit_script()`, `risk_score()`	Parse scripts, extract all `pkg::function` calls, check against the breaking-changes database
2 — Baseline & drift	`certify()`, `check_drift()`, `list_certs()`	Hash and store analytical outputs; detect any numerical change on subsequent runs
3 — Report & export	`repro_report()`, `repro_badge()`	Generate audit reports in three styles; badge your README

Tier 1 alone is useful for a quick scan. Tier 2 adds a persistent audit trail. All three together produce the kind of documentation that satisfies a journal's reproducibility policy or a QC reviewer's sign-off requirements.

Tier 1: scanning scripts for risk

audit_script() parses an R script and extracts every qualified pkg::function call, along with the installed version of that package and the line number of each call.

library(reproducr)

report <- audit_script("analysis.R")
print(report)
#>
#> -- reproducr audit report [2026-07-09 09:14] --
#>
#>   Files scanned:    1
#>   Packages found:   4
#>   Calls detected:   23
#>   R version:        4.4.2
#>   Platform:         aarch64-apple-darwin20
#>   Versions from:    installed library
#>
#>   Next step: risks <- risk_score(report)

risk_score() then checks every detected call against the built-in breaking-changes database. A call is only flagged if the installed version falls within a known risky version window — so if you are already on a version beyond the breaking change, it passes cleanly.

risks <- risk_score(report)
print(risks)
#>
#> -- reproducr risk score --
#>
#>   HIGH:      1
#>   MEDIUM:    2
#>   LOW:       1
#>
#> [HIGH]   dplyr::summarise (line 14 in analysis.R)
#>          Check    : changelog
#>          Details  : In dplyr 1.1.0, summarise() changed its default
#>                     grouping behaviour. Results from grouped pipelines
#>                     may differ silently from pre-1.1.0 outputs.
#>          Reference: https://dplyr.tidyverse.org/news/index.html#dplyr-110

The three risk checks

Breaking-changes database ("changelog") — the core check. A curated database of known cases where a package update changed function output silently, without an error or deprecation warning. Current coverage includes dplyr, tidyr, ggplot2, readr, purrr, stringr, lubridate, broom, data.table, lme4, caret, rstan, and base R itself (including the RNG change in R 3.6.0 and hclust() tie-breaking in R 4.0.0). Each entry specifies the version window, risk level, and a reference URL.

Seed check ("seed_check") — flags any call to a stochastic function (rnorm, sample, rbinom, and others) where no set.seed() appears within 50 lines above. The check is deliberately local — a set.seed() at the top of a 500-line script does not protect a stochastic call 400 lines later from being inadvertently affected by earlier random draws.

# Flagged — no set.seed() within 50 lines:
x <- stats::rnorm(100)

# Not flagged:
set.seed(42)
x <- stats::rnorm(100)

Locale check ("locale_check") — flags functions whose output depends on the system locale: sort(), format(), strftime(), and similar. A sort on character data returns different orderings on a Belgian server and a US server if the locale differs. Relevant any time code runs on remote infrastructure, cloud platforms, or shared compute environments.

Tier 2: certifying outputs and detecting drift

Auditing a script tells you what might change. Certification tells you what has changed.

certify() hashes a set of named outputs — model coefficients, p-values, sample sizes, summary statistics — and stores them under a named tag in a .reproducr.rds file in the project root.

model <- lm(mpg ~ wt, data = mtcars)

certify(
  outputs = list(
    coefs     = coef(model),
    r_squared = summary(model)$r.squared,
    n_obs     = nrow(mtcars)
  ),
  tag    = "submission-v1",
  script = "analysis.R"
)
#> reproducr: certified 3 output(s) [2026-07-09] under tag 'submission-v1'

After any environment change — a package upgrade, an R version update, a platform migration — check_drift() compares the current outputs against the stored baseline:

check_drift(
  outputs = list(
    coefs     = coef(model),
    r_squared = summary(model)$r.squared,
    n_obs     = nrow(mtcars)
  ),
  against = "submission-v1"
)
#>
#> -- reproducr drift check vs 'submission-v1' --
#>
#>   Verdict  : ALL OUTPUTS MATCH
#>   OK       : 3
#>   Drifted  : 0

If any output has changed — even by a single floating point digit — the verdict reports which outputs drifted and by how much. The .reproducr.rds file accumulates certifications across tags. Commit it to version control: it is your numerical audit trail.

list_certs() shows all certifications stored in the project:

list_certs()
#>   tag            certified_at  script        n_outputs
#>   submission-v1  2026-07-09    analysis.R    3

Tier 3: reports and badges

repro_report() renders an audit report from the audit_script() and risk_score() outputs. Three styles are available:

minimal — a compact summary covering environment, verdict, and risk table. Suitable for internal QC or project documentation.
academic — a ready-to-paste methods paragraph for journal submissions, including R version, package versions, audit verdict, and a note that the full report is in the supplementary materials.
pharma — a structured QC document with execution environment table, full package inventory, risk register, drift assessment, and sign-off fields for analyst and reviewer. Suitable for regulated submissions.

repro_report(
  report,
  risks,
  format      = "html",
  style       = "pharma",
  output_file = "qc_report.html"
)

repro_badge() writes a shields.io badge to your README reflecting the current audit status. In CI, this updates automatically on every push — the badge in your README reflects the state of the most recent run, not the state at the time you last remembered to update it.

CI integration

reproducr is designed to run in CI on every push. The gallery repositories — reproducr-clinical, reproducr-rwe, reproducr-ecology, and reproducr-cmc — demonstrate the complete workflow across different domains and regulatory contexts, with and without renv.

A typical CI workflow audits the scripts, checks for drift against the last certified run, and updates the badge. The DEMO.md in each gallery repository walks through the complete pipeline with real output at every step.

The breaking-changes database

The database that powers risk_score() is maintained in a separate community repository — reproducr-db. Each entry specifies a pkg::fn key, a version window, a risk level, a plain-English description of the breaking change, and a reference URL (the package NEWS.md, CRAN page, or GitHub release).

The database is checked weekly against current CRAN versions via check_db_staleness(). Community contributions are welcome — the format is a small JSON file and the contributing guide covers the version window design principles in detail.

Works with or without renv

If renv is active in the project, reproducr reads the lockfile automatically and reports package versions from there rather than from the installed library. If renv is not in use, it falls back to the installed library. No configuration is required either way.

Getting started

install.packages("reproducr")

library(reproducr)

# Scan a script
report <- audit_script("my_analysis.R")

# Score for risk
risks <- risk_score(report)

# Certify outputs
certify(
  outputs = list(primary_result = my_estimate),
  tag     = "v1"
)

# Generate a report
repro_report(
  report,
  risks,
  style  = "minimal",
  format = "html"
)

Full documentation at repro-stats.github.io/reproducr/. The breaking-changes database and gallery repositories are at github.com/repro-stats.

Questions or corrections — contact@reprostats.org.