Platform / Tests
Product 02Select affected tests, analyze results and detect flakes.
Run only the tests your change can affect. Suites finish sooner, and flaky tests are caught instead of retried.
Why it exists
Small changes can rerun large test suites even when most tests are unaffected. Sharding adds coordination and cost, while hand-maintained fast-test lists become stale. Test Intelligence selects affected tests and explains every decision.
The mechanism, in the order a job meets it.
During a run, adapters detect the files each test reads, the modules it imports, the environment it sees and any barrier it hits. Adapters cover pytest, Go, Vitest, Jest, Playwright, Bun and nextest.
A dependency map is stored per test identity and namespace with the outcome, duration and input-set digest.
On a later commit, preloop recomputes each test’s inputs. A test is skipped only when its dependency map is complete, nothing in it changed, it hit no barrier, and it last passed. New, failed, flaky and uncertain tests always run. Changes to sensitive paths — lockfiles, test configuration, package manifests, CI definitions and `.env` files — run every test in the session.
The tests API receives each result. The authenticated job supplies the repository, namespace, trust tier and commit instead of the request body.
Each decision has a reason. preloop stores every test’s outcome, duration and why it ran or was skipped, with its dependencies and last passing run.
Each capability is part of the product today.
Select only tests whose detected inputs changed. Tests without a complete dependency map or recent pass, and tests with barriers, always run.
Track per-test outcomes, durations, daily duration buckets and selection rates by job.
Confirm a flake only after a pass and a failure with identical input digests and no barrier. One green run does not establish a flake.
Quarantine is metadata. It cannot suppress a failure, skip execution or change a conclusion; it marks that the test is known-flaky.
Every test that runs or skips says why: the change did not touch it, a file or module it depends on changed, it is new, it failed last time, it is flaky, a shared config or lockfile changed, or someone asked for a full run.
Read overview, per-test history, durations, dependencies, each decision and the log for its source run.
Sample / bash
collected 9,579 items
selected 24, skipped 9,555
ran 24 a dependency changed (lib/sqlalchemy/sql/coercions.py)
ran 3 failed last time
ran 2 new tests
skip 9,555 not affected by this change
24 passedIllustrative session output: one commit that reached 24 tests out of a full suite.
It runs, and the reason says it is a new test. A test without a dependency map cannot be proven unaffected.
Yes. Lockfiles, test configuration, package manifests, TypeScript configs, CI definitions and .env files select every test.
Yes. A sensitive path such as a lockfile or shared config triggers a full run automatically, and the reason says so.
An order guard catches common cases. A test that depends on another test's side effects can still be selected alone.
No. A test that shells out is a barrier and always runs.
Results go to the tests API, scoped to the authenticated job. The dashboard and API read the same store, and each test decision includes its evidence.
Start
We will set it up with you and show you what changes.