Skip to contents

Writing Your Own Checks

Source: vignettes/writing-checks.qmd

checktor includes more than sixty checks, but every team has house rules too local to upstream: a function you have banned, a header you insist on, a habit you keep relapsing into. This vignette is for those. It walks through the handful of helpers in R/ast.R and R/rd.R and shows how to author a new check against the parsed syntax tree in a few lines of XPath, with the orchestrator handling the bookkeeping.

Every check walks the same road. Your sources are parsed once into a syntax tree, an XPath query picks out the nodes you object to, and each match comes back as a file:line string inside a result that knows how to print itself. The figure traces that road for one of the built-in checks, the one that flags an options() call whose enclosing function never puts the setting back.

Four stages. Line 42 of R/foo.R calls options(digits = 3) inside a function that never resets it. read_r_xml() parses the file into a syntax tree, xpath_lints() queries that tree with an XPath anchored on the SYMBOL_FUNCTION_CALL node for options, and the match comes back as the string foo.R:42 inside a checktor_check_result whose passed field is FALSE.

The shape of a check

An individual check is named lab_<name>(), where <name> is the check name that checktor() reports in tidy()$check and that Config/checktor fields refer to. Keeping those the same means a reader who sees a finding knows the function to call. The five category functions keep their own names, since each runs a whole panel rather than one test. Four are diagnose_<category>_issues(), and the policy panel is diagnose_policy_violations().

Every check follows the same contract:

lab_<name> <- function(path, verbose = TRUE, parsed = NULL) {
  if (is.null(parsed)) parsed <- read_r_xml(path)
  if (length(parsed) == 0L) {
    return(checktor_check_result(TRUE, character(0), "<message>"))
  }
  # ... XPath logic ...
  checktor_check_result(passed, issues, "<message>")
}

Why the parsed argument? It holds an optional parse-cache, so when checktor() runs all code-side checks together, it parses each file once and hands the cache to every check. Sixteen code-side checks against a 200-file package then mean 200 parses rather than 3,200.

Helpers in R/ast.R and R/rd.R

R/ast.R collects the shared machinery and R/rd.R the help-page walkers. A check leans on the handful below in roughly the order of the road above. read_r_xml() does the parsing and xpath_lints() does the querying, so between them they carry every check in the package. The rest are shorthands for patterns common enough to have earned a name. Learn the first two and you can already write a check.

read_r_xml(path)

This is what makes your sources queryable. It parses every R/*.R file in the package and returns a named list of list(file, xml, error). A parse failure becomes an error slot instead of crashing the run.

parsed <- read_r_xml(".")
str(parsed[[1]], max.level = 1)
#> List of 3
#>  $ file : chr "./R/bar.R"
#>  $ xml  :List of 2
#>   ..- attr(*, "class")= chr [1:2] "xml_document" "xml_node"
#>  $ error: NULL

The xml slot is an xml2 document produced by xmlparsedata::xml_parse_data(). Every parse-tree token is an XML element with line1, col1, line2, col2 attributes.

xpath_lints(parsed, xpath, label = NULL)

The workhorse. Give it an XPath query, get back "basename:line" strings for every match across every file, ready to hand to a check result’s $issues. The optional label appears in parens after each hit.

xpath_lints(parsed, "//SYMBOL_FUNCTION_CALL[text() = 'set.seed']")
#> [1] "bar.R:17" "foo.R:42"

undesirable_function_check(parsed, funs, label = TRUE)

The most common pattern, “flag any call to function X”, has a canned helper:

undesirable_function_check(parsed, c("install.packages", "browser"))
#> [1] "foo.R:4 (install.packages())" "foo.R:5 (browser())"

This is checktor’s equivalent of lintr::undesirable_function_linter().

not_under_fn_with_call_xpath(funs)

Returns an XPath predicate that restricts hits to nodes whose innermost enclosing function-body doesn’t also contain a call to any of funs. This is how option_changes enforces that options() is guarded by a sibling on.exit() in the same function, and the “innermost” part is what makes it correct on nested functions where on.exit in the outer function wouldn’t cover an inner one.

predicate <- not_under_fn_with_call_xpath(c("on.exit", "local_options"))
xpath <- paste0(
  "//SYMBOL_FUNCTION_CALL[text() = 'options']",
  "[", predicate, "]"
)

extract_rd_section(rd, tag) and collect_rd_text(node, skip)

Walking .Rd files structurally via tools::parse_Rd(); both live in R/rd.R. Reading man/ yourself sees every help page, including any that .Rbuildignore keeps out of the tarball. The built-in checks skip those, so leave them out too if your check should judge only what CRAN receives.

rd <- tools::parse_Rd("man/my_fn.Rd")
ex <- extract_rd_section(rd, "\\examples")
collect_rd_text(ex, skip = "\\dontrun")

Walked example: Sys.setenv() without cleanup

Suppose we want a check that flags any Sys.setenv() call whose enclosing function doesn’t also call on.exit(Sys.unsetenv(...)) or withr::local_envvar(). This is the same shape as lab_option_changes and is built into checktor as lab_sys_setenv. Here is the essential shape:

lab_sys_setenv <- function(path, verbose = TRUE, parsed = NULL) {
  # reuse the parse-cache when checktor() supplies one, else parse fresh
  if (is.null(parsed)) parsed <- read_r_xml(path)
  if (length(parsed) == 0L) {
    return(checktor_check_result(TRUE, character(0),
                                 "Sys.setenv reset check"))
  }
  xpath <- paste0(
    "//SYMBOL_FUNCTION_CALL[text() = 'Sys.setenv'][",
    # keep only calls whose enclosing function never resets the variable
    "  ", not_under_fn_with_call_xpath(c(
        "on.exit",
        "Sys.unsetenv",
        "local_envvar", "with_envvar"
      )),
    "]"
  )
  issues <- xpath_lints(parsed, xpath)
  passed <- length(issues) == 0L
  # a built-in check also calls emit_issue_summary(issues, verbose, ...) here
  # to print the cli summary when verbose = TRUE
  checktor_check_result(passed, issues, "Sys.setenv reset check")
}

Twenty lines, and the interesting one is the XPath predicate. Everything else is bookkeeping shared with every other check. The built-in version adds one refinement, exempting a setter that captures the old value and hands it back (old <- get_env(); Sys.setenv(...); invisible(old)), since that is a restore contract rather than a leak. Refinements like that are where a real check earns its keep, but the skeleton is exactly this.

The xmlparsedata XML structure

A call fn(a, b = 1) parses to:

<expr>                              <!-- call expr -->
  <expr>                            <!-- function-name expr -->
    <SYMBOL_FUNCTION_CALL>fn</SYMBOL_FUNCTION_CALL>
  </expr>
  <OP-LEFT-PAREN>(
  <expr><SYMBOL>a</SYMBOL></expr>   <!-- first positional arg -->
  <OP-COMMA>,
  <SYMBOL_SUB>b</SYMBOL_SUB>        <!-- named-arg name -->
  <EQ_SUB>=</EQ_SUB>
  <expr><NUM_CONST>1</NUM_CONST></expr>  <!-- named-arg value -->
  <OP-RIGHT-PAREN>)
</expr>

Anchor on the SYMBOL_FUNCTION_CALL and every other node is one axis away. The amber card is the one that catches people out.

The parse tree for fn(a, b = 1). Anchored on the SYMBOL_FUNCTION_CALL node, parent::expr reaches only the function-name wrapper, which is the trap. The call expr is parent::expr/parent::expr, and the first positional argument is parent::expr/following-sibling::expr[1].

  • the call expr is parent::expr/parent::expr
  • the first argument expr is parent::expr/following-sibling::expr[1]
  • a named-arg name is parent::expr/parent::expr/SYMBOL_SUB

That middle one is the first argument, not the first positional argument. For fn(b = 1, a) the same axis lands on the value 1, so reach for it only when you already know the shape of the call.

Trying it out 

# Parse a file
parsed <- read_r_xml("path/to/package")

# Find every call to install.packages()
xpath_lints(parsed,
            "//SYMBOL_FUNCTION_CALL[text() = 'install.packages']")

Wiring it into checktor()

A finished lab_* function does nothing until checktor() knows to call it. register_check() connects the two at run time, with no edit to checktor’s own source:

lab_my_check <- function(path, verbose = TRUE, parsed = NULL) {
  if (is.null(parsed)) parsed <- read_r_xml(path)
  issues <- undesirable_function_check(parsed, "banned")
  checktor_check_result(length(issues) == 0L, issues, "no banned()")
}

register_check("my_check", lab_my_check,
               category = "code", severity = "policy")

From then on every checktor() run includes my_check. It appears in issues(), tidy() and the printed report under that name, and counts toward the verdict at the tier you gave it.

A few things worth knowing:

  • The function returns a checktor_check_result(), the same shape as every built-in check, so run_checks() handles its error catching and $passed bookkeeping.
  • checktor calls it as fn(path, verbose). If the function also declares a parsed argument (for code and policy checks) or a desc argument (for description checks), checktor forwards its shared cache, so your check reuses the one parse instead of re-reading the sources.
  • category is one of code, description, documentation, general or policy; the check runs alongside that category’s built-ins.
  • severity is policy, robustness or opinion, and decides whether a finding counts against a clean bill of health under checktor()’s severity argument.
  • A check that cannot run where it is, because it needs a network, a tool or a credential that is not there, should say so rather than pass. Return checktor_check_result(TRUE, character(0), "<message>", skipped = TRUE, skip_reason = "<why>"), and it shows as skipped in tidy(), summary(), the printed report, health_report() and ci_report(), never as a pass.
  • options(checktor.disable = "my_check") or Config/checktor/disable turns a registered check off, the same as a built-in one.

registered_checks() lists what is currently registered, and unregister_check("my_check") (or unregister_check() for all) removes it. The registry is session-scoped, so call register_check() somewhere it runs before your check is needed: a project .Rprofile, your own package’s .onLoad(), or the top of a CI script.

The helpers the example leans on are exported too, so a check you register has the same toolkit the built-ins use: read_r_xml(), xpath_lints(), xpath_per_file(), undesirable_function_check(), not_under_fn_with_call_xpath(), and the .Rd walkers extract_rd_section() and collect_rd_text().

Contributing a check to checktor itself? Then skip the registry and wire it the way the built-in checks are: add lab_my_check() to the right R/diagnostics-*.R file and add a row for it to the check table in R/registry.R, which gives its category, its tier and when it runs. The category function runs every row of its category in table order, passing parsed or desc to a check whose signature takes it. A check that reads desc also gives the message it reports, so it can be shown as skipped when R cannot read the DESCRIPTION. A check that should not run every time says so in the row: console or backend when it needs the network or an external tool, in which case it returns checktor_skipped_result() when it cannot run, or request when it runs only if called, in which case it stays out of the run altogether. Then add a row for it to Where the Checks Come From, and a matching *_bad scenario under inst/diagnose/ for its example. test-severity.R holds the table, and the vignette’s copy of it, to what actually runs.

Without writing code

Some house rules need no new check at all. A package configures checktor from Config/checktor/* fields in its own DESCRIPTION. Each field is a comma-separated list, and DESCRIPTION has no comment syntax, so keep notes out of the field: anything after the last name becomes part of it.

Config/checktor/software_names: brms, cmdstanr
Config/checktor/acronyms: MCMC, GLMM
Config/checktor/allow: urls:README.md
Config/checktor/disable: news_file

software_names flags those names when they appear unquoted and acronyms accepts those domain acronyms, each extending the vocabulary its check already uses; language_names and format_names work the same way. allow mutes a specific finding, either a whole check or a check:substring, which is what a green checkup() gate needs when a finding has been reviewed and accepted. disable skips a check outright: it does not run and is not counted. To turn a check off in every package rather than one, set options(checktor.disable = "news_file") once, for example in .Rprofile; the option and the field are combined. Write a check of your own when the rule is genuinely yours, and reach for Config/checktor/* when you only need to teach or quiet a check checktor already includes.

Conclusion

Building on the parsed syntax tree buys the property that makes checktor trustworthy, because a pattern sitting in a string literal or a comment is a different kind of node than a real call, so it never false-positives.

The same three lines of R seen two ways. A regex matches the letter T three times, in a comment, inside a string literal, and as a real symbol. The parse tree instead types each one as COMMENT, STR_CONST and SYMBOL, so only the SYMBOL is a bare T and only it is flagged.

Write the XPath, let run_checks() carry the rest, and your house rule is enforced as rigorously as the built-in checks.

See also