rextendr Internals

How rextendr scaffolds R packages and drives the build process for extendr-backed crates.

This page describes various internal functions and processes used by rextendr to setup and build R packages. It is designed to get contributors up-to-speed on the internal mechanics of rextendr, so that they can more efficiently contribute to development and maintenance.

Package Setup

General R package setup is achieved with usethis::create_package(). Scaffolding files required to call Rust code in R using extendr is then added to the package directory with rextendr::use_extendr(). This involves interpolating strings into template files in inst/templates/, which are then written to the user’s package directory via the internal function use_rextendr_template():

R/use_extendr.R

use_rextendr_template <- function(
  template,        # filename in inst/templates/
  save_as,         # destination path in the user's package
  data = list(),   # named list of variables to interpolate into the template
  quiet = FALSE,
  overwrite = NULL
)

When usethis is available and overwrite = NULL, this delegates to usethis::use_template(), which handles interactive prompting if the file already exists. Otherwise it reads the template file directly, performs interpolation, and writes the result. The templates directory currently includes all of the following:

fs::dir_tree(system.file("templates", package = "rextendr"))
C:/Users/kenne/AppData/Local/R/win-library/4.5/rextendr/templates
├── Cargo.toml
├── cleanup
├── cleanup.win
├── config.R
├── configure
├── configure.win
├── document.rs
├── entrypoint.c
├── extendr-wrappers.R
├── lib.rs
├── Makevars.in
├── Makevars.win.in
├── msrv.R
├── settings.json
├── win.def
└── _gitignore

The exceptions to simple string interpolation are Cargo.toml, whose content is generated programmatically before being written to file, and Makevars.in / Makevars.win.in, which use a second layer of @PLACEHOLDER@ substitution performed at package build time by tools/config.R.

You can see the content of these template files on main in the rextendr repository here: source. For a description of what each generated file does, see the Project Structure page in the user guide.

Package Build

When a developer calls devtools::document(), they initiate a set of steps to build, document, and register functions in their R package. Here are those steps in order:

  1. Check Rust and cargo versions
  2. Generate Makevars from template
  3. Compile the Rust crate into a static library
  4. Generate R wrappers
  5. Compile entrypoint.c and link the static library
  6. Register exported routines with R on load
  7. Clean up build artifacts

These steps are driven by R CMD INSTALL, which devtools::document() calls implicitly.

Setup

The first thing that gets called is configure (configure.win on Windows). The sole purpose of the configure script is to source the R script tools/config.R, which in broad outline does two things. First, it runs tools/msrv.R to check that the Rust version is consistent across metadata files like DESCRIPTION and Cargo.toml (step 1). Second, it substitutes @PLACEHOLDER@ variables in the Makevars.in and Makevars.win.in templates (signaled by .in) with relevant data to generate the actual Makevars file (step 2).

Build

The R package build process next invokes Makevars, which was just generated by the configuration file. The Makevars in turn does two things. First, it calls cargo build to compile the static library (step 3). Then it calls cargo run to compile and execute the associated document binary, which generates all extendr wrappers (basically .Call()) and writes them into R/extendr-wrappers.R (step 4). Note that if a vendor/ directory or rust/vendor.tar.xz tarball is present, the Makevars also ensures that Cargo is configured to use it for offline compilation — this is the essential mechanism required for CRAN submissions.

Registration

Because we are building an R package with compiled code, R requires that we register all routines via a C-level initialization function named R_init_<pkgname>. This is where entrypoint.c comes in. Currently, extendr generates its own version of this function — R_init_{{{mod_name}}}_extendr — inside the Rust library. The C entrypoint then bridges the two, calling the Rust-generated function through the R-facing R_init_{{{mod_name}}}. Compiling entrypoint.c makes that symbol available on package load (step 5). When the package next gets loaded, the C function is called to register compiled routines (step 6). If you inspect R/extendr-wrappers.R, you will see where this happens. The call is @useDynLib(pkgname, .registration = TRUE).

Cleanup

To prevent the Makevars generated by configure from being committed to git, a cleanup is invoked (cleanup.win on Windows) that calls the shell command rm to remove that file (step 7).

Once these steps are taken, the compiled Rust should become available in the current R session.