Star
Search corralai.dev

Getting started

Install

go install github.com/pdbethke/corralai/cmd/corral@v1.0.0-rc.29

Or clone and build from source:

git clone https://github.com/pdbethke/corralai
cd corralai
go build ./...

Run it on your repo

Two commands. The first is free — no key, no sandbox, no money — and tells you in seconds which files corral can audit and which have no paired test:

cd your-repo
corral certify --repo . --dry-run

In Go, JavaScript, TypeScript, Ruby, PHP or Rust, add --evidence and your test command to also see which files a green test executes even though no filename pairs them. It runs your suite in the sandbox, with no model and no key:

corral certify --repo . --dry-run --evidence -- <your test command>

Then audit the three highest-ranked files, with a cap on what the run may spend:

export GEMINI_API_KEY=...        # and ANTHROPIC_API_KEY for the critic below
corral certify --repo . --top 3 --max-tokens 3000000 \
  --writer-model gemini-3.8-flash --mutant-model gemini-3.8-flash \
  --derive-model gemini-3.8-flash --critic-model claude-haiku-4-5 \
  -- .venv/bin/python -m pytest -q   # your own test command: npm test, go test ./..., cargo test, …
  • Those model names are an example, not a default — corral has none. Name any models you hold keys for. The one rule: the critic must be a different model from the writer. One provider is enough (for example two different Anthropic models), and --critic-model off drops the critic, which is advisory anyway.
  • No API key? Use a subscription. Any seat can be a coding-agent CLI instead of an API model: --writer-model codex, --critic-model claude-code, or claude-code:<model> to pin one. The agent uses its own login, so on a Pro/Max/Plus plan a seat has no per-token bill. Two cautions: the CLI inherits corral’s environment, so an exported vendor API key may be billed instead of the subscription; and a seat that reports no token usage is refused under --max-tokens (claude-code and codex do report it). See agent seats.
  • Your suite has to pass first, inside the sandbox. The sandbox is offline and sees /usr plus your checkout, not your home directory. Install the project’s dev dependencies into a virtualenv inside the project (./.venv), which is bound in read-only, and run the suite through it, as above: a bare python inside the sandbox is the system interpreter, which does not have your dependencies. For Python, add pytest-cov so corral can pick the tests that cover each file.
  • One language per explicit test command. A test command after -- grades every file, so it is refused when the scan spans more than one language. On a mixed repository, omit it (corral then runs each language’s default runner), or narrow the scan with --path <dir>.
  • What it costs, measured once: flask’s src/flask/app.py with these seats took about six minutes and 0.5M input tokens, and proved 9 of 9 surviving bugs catchable. Your files and suite will differ; --max-tokens is the hard cap.
  • Where the proven tests appear: at the end of the report, each with “add it to your suite”. corral scans show <id> --evidence prints them again later. corral does not edit your repository or open a pull request.

corral doctor (below) checks your sandbox, toolchain and keys for free before the first paid run. To run it on every pull request instead, see the GitHub Action.

A Rust project

Rust is audited with cargo test or cargo nextest run. Two things differ from the other languages:

  • The source’s tests must live in a separate file it declares — #[path = "foo_tests.rs"] mod tests;, or mod tests; with rustc’s default module file. A file whose tests are an inline #[cfg(test)] mod tests { … } block cannot be audited yet: corral would have to compose its test into the source file itself, and that is not built. --dry-run lists those files as excluded, with the reason test-landing-not-built.
  • The toolchain. A stock rustup install lives under $HOME, which the jail cannot see. Either install a system-wide toolchain the jail can use (RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo rustup-init --no-modify-path, with /usr/local/cargo/bin on PATH), or, on a disposable checkout such as a CI runner or a scratch clone, pass --substrate workspace, which runs your own toolchain on the checkout itself (and mutates it — see Limitations).

In a Cargo workspace, scope the scan and the command to one crate:

corral certify --repo . --path crates/<crate> --top 3 --max-tokens 3000000 \
  --writer-model gemini-3.8-flash --mutant-model gemini-3.8-flash \
  --derive-model gemini-3.8-flash --critic-model claude-haiku-4-5 \
  -- cargo test -p <crate>

Rust has no fail-fast flag verified yet, no salvage of a partly failing authored test and no coverage pre-flight; language support has the full table.

See it work first: corral demo

One command, no setup beyond a provider key:

corral demo --writer-model claude-sonnet-5 --mutant-model claude-sonnet-5 --critic-model claude-haiku-4-5

It writes a small Go package with a five-clause password rule and a test that checks only two of them, then audits it with the real certify --local. You need a Go toolchain the sandbox can see — a system-wide install under /usr, not one under your home directory or from snap (see “It always runs sandboxed” below) — and one provider key, or agent seats (--writer-model codex --mutant-model codex --critic-model claude-code) and no key at all. No venv, no database, no fixtures. The project is left on disk so you can read the test and see what it never asserts.

Then point it at your own code, with corral doctor first.

Your first audit: corral certify --local

The fastest way to see corral do the thing it’s for — certify a change by execution, not opinion — is one command, off your own key, no daemon:

export ANTHROPIC_API_KEY=sk-ant-...   # or OPENAI_/GEMINI_/OPENROUTER_API_KEY

corral certify --local --repo-dir . \
  --code path/to/your/file.go \
  --goal "what this code must guarantee" \
  --writer-model claude-sonnet-5 \
  --mutant-model claude-sonnet-5 \
  --critic-model claude-haiku-4-5 \
  --out verdict.json \
  -- go test ./...

Those model names are an example, not a default — corral has none. Every seat is yours to name, from whichever provider you hold a key for; the models above are what we run. The only rule is that the test-critic must differ from the test-writer, because that decorrelation is what the verdict rests on — a property, not a vendor, so any two distinct models satisfy it. --critic-model off drops the critic entirely (advisory, never gates the verdict). A run with an unnamed seat is refused, and the refusal reports which provider credentials it can see.

That runs the full adversarial testing pool in-process: mutant-generators seed goal-violating bugs into your code, your own test is scored against them by executing it in a jail (never a self-report), a test-writer proves any gap is real by writing and killing the test you were missing, and a decorrelated test-critic reads your suite cold. You get a signed verdict — certified or needs-review — and, because of --out, a record file you can re-check offline any time:

corral certify verify verdict.json --pubkey "$(corral certify pubkey)" --allow-unanchored

A record’s own embedded key is never a trust anchor, so verify refuses to run without --pubkey (or --brain); --allow-unanchored accepts a record that was not logged to a transparency witness.

Before you spend a run: corral doctor

An audit costs real money and real minutes, and it is almost always the environment that stops one — the sandbox won’t start, the toolchain is invisible inside it, the key for the model you assigned is missing, the file has no paired test. Discovered one at a time, each of those costs another run, and most cost money to learn. doctor checks them all up front for free — no model is ever called — in the order the audit itself would hit them, so the first FAIL is the first thing to fix:

corral doctor --code path/to/your/file.go \
  --writer-model claude-sonnet-5 --mutant-model claude-sonnet-5 \
  --critic-model claude-haiku-4-5 \
  -- go test ./...
  [ok  ] sandbox starts
  [ok  ] toolchain reachable inside the sandbox
  [FAIL] herd accepted by certify's own preflight (mutant-generator=claude-sonnet-5, test-writer=claude-sonnet-5, test-critic=claude-haiku-4-5)
         agentbackend: ForModel: model "claude-sonnet-5" needs an Anthropic key — set ANTHROPIC_API_KEY
  [ok  ] code file pairs to a test
         found path/to/your/file_test.go by convention (sibling)

1 check(s) failed — fix these before spending a run.

Every argument is optional and each unlocks more checks: --code/--test add the test-pairing check, a test command after -- adds the in-sandbox toolchain check, and --mutant-model/--writer-model/--critic-model run the same seat preflight certify runs, for exactly the models you plan to route to. An agent seat gets its own line: the binary is found and its version printed, but its login is not verified, because that would cost a call. It exits non-zero if anything failed.

One thing it deliberately does not check, because it needs a real seeded workspace: whether your suite passes on unmutated code inside the sandbox — the most common way an audit dies. Run your own test command by hand first; certify --local reports a failing baseline as COULD-NOT-GRADE, with the runner’s own output. Whether a multi-file project needs --repo-dir is checked only where the single-file form is refused outright (a Rust file is one); for anything with imports, use --repo-dir.

On a file with several functions the generation fans out — one seat per group of functions, so every function gets probed, not just whichever one a single generator happened to pick (--max-shards N bounds the width; the default is 8 — only --swarm sizes itself to your machine). Name a --shadow-model (or a --shadow-pool of them, one drawn per run) and a challenger model attacks the same regions in parallel, purely to measure one model against another on identical ground — it’s recorded for comparison and never part of the verdict (off unless named, like every seat). And a mutant that makes your suite hang is killed fast and counted as caught, so a runaway loop can’t stall the audit (--test-timeout overrides the auto-derived cap).

Corral has no default models — every seat is named by you, and a run with an unnamed seat is refused. One key can still satisfy the distinctness rule on its own: two different models from the same provider (Sonnet writing and mutating, Haiku critiquing, say) off that single ANTHROPIC_API_KEY. Naming --critic-model gemini-3.6-flash plus a Google key (GEMINI_API_KEY or GOOGLE_API_KEY) routes the critic to Gemini via the OpenAI-compatible Google endpoint — a real cross-vendor critic, writer and mutant-generator unchanged. A missing key fails the run closed instead of silently falling back. It supports seven languages — Go (go test, gotestsum), Python (pytest, unittest), Ruby (RSpec, minitest), JavaScript and TypeScript (node --test, jest, mocha, vitest), PHP (PHPUnit) and Rust (cargo test, cargo nextest) — with the language inferred from --code’s extension and the runner from your test command. What each runner can and cannot do is on language support.

It always runs sandboxed. bwrap is the Linux default; --jail container falls back to docker/podman; macOS uses sandbox-exec. There’s no unsandboxed option. On Ubuntu 24.04+, AppArmor restricts unprivileged user namespaces by default and bwrap won’t start. The error message tells you to load the profile Ubuntu ships for bwrap, bwrap-userns-restrict (on 24.04 it may still be under /usr/share/apparmor/extra-profiles):

[ -f /etc/apparmor.d/bwrap-userns-restrict ] || sudo cp /usr/share/apparmor/extra-profiles/bwrap-userns-restrict /etc/apparmor.d/
sudo apparmor_parser -r /etc/apparmor.d/bwrap-userns-restrict

Do not write your own /etc/apparmor.d/bwrap marked unconfined: it takes bwrap away from the shipped profile and leaves it without the capabilities it needs. corral printed such a profile before 1.0; if one is on your host, remove it with sudo apparmor_parser -R /etc/apparmor.d/bwrap && sudo rm /etc/apparmor.d/bwrap. Or pass --jail container, which needs docker or podman and CORRALAI_EXEC_IMAGE set to an image carrying your toolchain (for example python:3.12-bookworm). The sandbox is offline: anything your tests fetch from the network fails inside it. And the language toolchain has to be jail-visible: installed system-wide under /usr (your distro’s golang/python3 package), not a --user/snap/pyenv install — a snap go or a pip install --user pytest is invisible inside the sandboxed mount namespace and the run will fail closed looking for it.

Python: you do not have to install pytest system-wide. Plenty of developers will not contaminate their system interpreter for an audit tool, and they shouldn’t have to. The jail binds /usr but not your home directory, so what actually breaks is a runner living under $HOME — pip install --user, pyenv, or a virtualenv in the default ~/.local/share/virtualenvs. Put the virtualenv inside the project instead, at ./.venv or ./venv, and --repo-dir auto-detects it and bind-mounts it read-only into the jail (see Dependency dirs). Then run your suite through it:

python3 -m venv .venv && .venv/bin/pip install pytest   # or:
PIPENV_VENV_IN_PROJECT=1 pipenv install --dev pytest

corral certify --local --repo-dir . --code pkg/thing.py --test tests/test_thing.py \
  --goal "…" --writer-model … --mutant-model … --critic-model … \
  -- .venv/bin/python -m pytest -q

pipenv’s default venv location is outside the project, which is exactly the invisible case — PIPENV_VENV_IN_PROJECT=1 is what moves it somewhere the jail can see.

Walk through a real verdict end to end in Your first audit, in detail.

In your coding agent: corral setup

If you work in a coding agent, one command wires corral into it: the read-only findings server (corral mcp) and the corral skill, which teaches the agent to read corral’s record, check a finding by running something, and leave the verdict to you.

corral setup                    # pick your agents from a list
corral setup --dry-run --all-detected   # see the plan, write nothing
corral setup --uninstall --all-detected # remove only what setup added

Detected agents start unticked and agents already wired start ticked, and nothing is written until you have seen the plan and said yes. Proven on Linux (2026-10-25) in Claude Code, Codex CLI, Gemini CLI, Cursor, OpenCode, Antigravity and GitHub Copilot CLI; VS Code is wired but labelled (not yet proven). See agent setup for each agent’s commands and the evidence.

The brain (optional)

Everything on this page runs with no server. The brain is corral’s optional coordination daemon — the substrate under a coordinated herd, a live console and the human-gated proposal loop. Nothing in it is required for a verdict, and nothing in it is read by an audit. If you want it, start with Running it, which says what it is and is not, or read the brain, and what it is not.

Common knobs

  • MODEL_BACKEND / OPENAI_BASE_URL — pin every seat to one gateway endpoint (OpenRouter, a local daemon); without it each seat routes by the model you name
  • CORRALAI_MODELS_FILE / CORRALAI_MODELS — the model registry, so seats can be named by alias
  • CORRAL_LEDGER — where certify --repo writes its entry and reads its prior (default: the audited repo’s .corral/ledger/)
  • CORRALAI_CACHE_DB — corral’s local cache of derived goals and test selections
  • CORRALAI_CERTIFY_KEY_FILE — where the signing key seed lives (--local verdicts, --attest)

The daemon’s own knobs (CORRALAI_OIDC_ISSUER, CORRALAI_GIT_TOKEN, CORRALAI_MOTHERDUCK, …) are on Running it; every variable is documented from the binary’s own source comment on corral’s CLI page.