A selective test run needs evidence. When a test runs under preloop, the system detects and stores the inputs it used. On a later commit, it compares those inputs with the current files and environment. It skips the test only when the dependency data is complete, the inputs are unchanged, and the other safety checks pass. If any condition is uncertain, the test runs.

This is different from a changed-directory rule. A test in one directory can read a configuration file or a fixture elsewhere. A test with an unchanged file path can also change behavior when a previously absent file appears. The dependency data makes the decision inspectable.

What preloop detects and stores

When a test runs under preloop, a framework adapter observes its activity. preloop has adapters for pytest, Go, Vitest, Jest, Playwright, Bun, and nextest. Pytest uses a plugin inside the session, Jest uses a transformer and custom environment, and Go uses a build-time scan of the package graph. The framework integration differs, but each adapter produces the same kinds of dependency facts.

For each test, preloop stores:

  • Files it read, with a content digest for each.
  • Files it looked for and did not find. A negative probe is a dependency: if code checks os.path.exists("config/local.toml"), creating that file later must invalidate the test.
  • Directory listings it performed. A test that globs tests/fixtures/*.json depends on the set of matching files, not only their contents.
  • Symlink chains it resolved.
  • Environment variables it actually read, rather than the whole environment.
  • Whether it hit a barrier such as a subprocess, network call, or native extension loaded in process.

A barrier means part of the test's behavior was not observable. preloop cannot prove that test unaffected, so it runs the test.

The skip rule

A test is skipped only when every condition below holds:

  1. The test was collected and mapped on a previous run.
  2. The dependency data is complete: every read the adapter saw was accounted for.
  3. Every stored dependency digest is unchanged.
  4. No barrier applied during detection or replay.
  5. The session fingerprint and collection dependencies still verify.
  6. The test's last observed outcome was a pass.
  7. The test is not on the flake list.
  8. The order guard passed; the test does not depend on another test's side effects.
  9. No sensitive path changed. Lockfiles, test configuration, package manifests, TypeScript configurations, CI definitions, and .env files select every test in the session.

The sensitive-path rule is deliberately broad. When package.json changes, preloop runs the full session instead of trying to infer which tests the dependency graph might reach. A wrong selective run would be harder to diagnose than the extra work.

The pass rule also uses the flake ledger. A test that passed once and failed once with identical inputs is a confirmed flake and always runs. One green run is not enough evidence of reliability; the ledger keeps both pass and fail outcomes.

Reason codes

Every selected test has a reason:

texttext
unaffected              nothing it read changed
dep_changed             a file it read has a different digest
module_changed          a module it imports changed shape
previous_failure        it failed the last time it ran
flaky                   it has a confirmed pass/fail pair on identical inputs
barrier                 it did something we could not fully observe
unknown_test            no dependency data exists
test_removed            it was collected before and is not now
sensitive_path_changed  a lockfile, config or manifest moved
forced_full_run         you or a policy asked for everything

The reason code answers why a test ran. The API can also return its dependency map and the digest that changed:

bashbash
$ plp ci logs 0x2f19 --job test
collected 9,579 items
selected 24, skipped 9,555

  ran     24  dep_changed (lib/sqlalchemy/sql/coercions.py)
  ran      3  previous_failure
  ran      2  unknown_test (new)
  skip 9,555  unaffected

24 passed

Illustrative output.

When selection helps

Local changes

A commit that touches one module usually reaches one part of the dependency graph. preloop selects the affected tests and skips the rest because the change does not reach them. In a codebase with clear module boundaries, the job can finish after running the tests for one file instead of the full suite.

Global changes

A lockfile, test configuration, package manifest, or module imported by every test selects the full session. Those files genuinely reach everything, so the full run is the correct result. Selection does not reduce work for a repository where most changes are global.

Detection cost

Detecting dependencies adds work to the run that observes a test. preloop keeps that cost away from the step waiting on the result. Later decisions compare digests, which is cheaper than rerunning the suite.

Run selection for a week and inspect the split. If most commits are local, most pushes run a smaller set. If most commits are global, selection exposes the repository's dependency shape without hiding it.

Stored dependencies instead of prediction

Some tools predict likely failures from historical correlations or models. Those predictions can be useful, but a missed failure is difficult to explain before the commit runs.

preloop uses a stored statement about a specific test: these are the inputs the test used, and they are unchanged now. If the statement can be verified, the test can be skipped. If the inputs are uncertain, the test runs. The decision is based on evidence from the test itself rather than a forecast.

What the selector will not do

It will not skip a new test, a test that failed last time, or a test whose inputs it could not observe. It will not treat one green run as proof of health, and it will not make a selective decision when a lockfile or other sensitive path changes.

Those rules keep an uncertain test in the run instead of turning an uncertain prediction into a missing test result.