feat(lenses): the commit-truth lens, repo-facts, as Gate 4
Every claim in every commit message, and in the documentation the range adds or changes, proven or gone. The lens's block kinds are text, doc and slot: a message unit is keyed on its text, the base and the code up to its commit (MessageDeps); a documentation unit on its file, its normalised text, the base and the code of the whole range (DocKeyParts); a diffstat on its text alone; whole:order and whole:absent are asked in slot passes of its own, never stored. A documentation file's diff header and a part the commit writes nothing in are a mechanical n/a. Budgets 60/40 for a text or doc block, 50/35 for a slot; one read. Gate 4, after the proofs, so the messages and documents are read once the code they describe has settled; the owner's gate is 5. The goldens that follow from the lens table are derived by tests/Goldens/derive.py, which asserts that the recorded table still reproduces the recorded entries before it writes. Proven, with the opt-in commit after this one: the suite (26 passed); a dry run of a scratch range (2 commits, a README with sections) by the previous binary and this one with the three static lenses on and the rest off, 25 prompts and 4 system files byte for byte; with repo-facts on as well, those files unchanged and 10 passes and 1 system file added. The lens's firing proof in this pipeline is owed (the owner's go).
This commit is contained in:
parent
61ee4602f4
commit
febbfc3fcc
GPG key ID:
2937E59CDCC1BCFB
19 changed files with 2564 additions and 646 deletions
|
|
@ -5,7 +5,7 @@ This file is for changing crafter-review itself. `README.md` says what it is and
|
|||
## What it is
|
||||
|
||||
`crafter-review freeze` exports `base..HEAD` with `git format-patch` into `.review-gates/push/<slug>/` and hashes the export. It refuses on identity, trailer, dirty-tree, merge and fast-forward failures. `crafter-review run` partitions the export mechanically (`:Inventory`: every non-blank line in exactly one unit, fixed slots for what is absent), then runs the gates the config enables, in order.
|
||||
- Each gate is a lens; Gate 3 has two, the bug-prover and the security lens.
|
||||
- Each gate is a lens; Gate 3 has two, the bug-prover and the security lens; Gate 4 is the commit-truth lens, the messages and the documentation of the range (2026-10-31). Every lens is off until the consumer's config turns it on.
|
||||
- Each lens is one `claude --print` process per unit (block passes). Where the lens has one, a whole pass gates the blocks and hands them its summary.
|
||||
- A pass returns structured output: a verdict, findings, notes, and one coverage entry per unit. `:Normalize` derives the verdict from the findings and fails closed on a missing or blocking entry.
|
||||
- Clean results are stored per unit under a content key and reused across hashes (`:Plan`). The static lenses need three clean reads of a unit before it is reused; the functional lens needs one.
|
||||
|
|
@ -76,7 +76,7 @@ Conventions:
|
|||
- **Git, temp dirs and timeouts.** Every git call runs with `GIT_CONFIG_GLOBAL=/dev/null` and `GIT_CONFIG_SYSTEM=/dev/null`; the owner's global config signs commits. Tests work under their own temp dirs, with `REVIEW_GATES_BOARD` and `--scratch` pointing there, and each stays under crafter-build's 60 s.
|
||||
- **Subprocesses.** A test that needs one re-executes its own binary. The test binaries of Runs and Commands are crafter-review itself when given arguments, in native mode whatever builds them.
|
||||
- **Stand-ins.** The claude stand-in is a `/bin/sh` script, and the Postgres stand-in an in-process thread (`tests/Common/FakePg.hpp`).
|
||||
- **Goldens.** `tests/Goldens/fixture/` holds the recorded answers the suite compares against: the inventory, the plans, the keys, the prompts, the normalised results, the stream, the config, the SQL helper's output. A deliberate change edits them in the same commit, by hand; so do `:Unicode`'s tables (Unicode 16.0.0) and `tests/Text/fixture/vectors.json`.
|
||||
- **Goldens.** `tests/Goldens/fixture/` holds the recorded answers the suite compares against: the inventory, the plans, the keys, the prompts, the normalised results, the stream, the config, the SQL helper's output. A deliberate change edits them in the same commit, by hand; so do `:Unicode`'s tables (Unicode 16.0.0) and `tests/Text/fixture/vectors.json`. The entries that follow from the lens table and the config defaults - the config cases through `config`, the commit-truth lens's plans, the not-reviewed lists, the estimates and the gates - are re-derived by `python3 -I tests/Goldens/derive.py <checkout> <crafter-review binary>`, which asserts that the recorded table still reproduces the recorded entries before it writes; review its diff.
|
||||
- **The suite runs inside the Flatpak build too** (`crafter-build test` is a build command of the manifest), where `/.flatpak-info` exists: a test pins `Platform::SetMode(Native)`.
|
||||
|
||||
## Behaviour the checks hold
|
||||
|
|
@ -133,6 +133,8 @@ Conventions:
|
|||
- In the Flatpak, a pass's `SHELL` is `/bin/bash` (Platform).
|
||||
- **Commands beyond the gates**: `init`, `doc`, `browser install`, `stop`, and `--version`, which prints the home and the charters' sha256s (Commands).
|
||||
- **Order, not value:** `estimate.cacheWrite`'s last digit depends on the order the passes ran, because it is a float sum.
|
||||
- **Every lens is off until the consumer's config turns it on** (2026-10-31, Jorijn: "all lenses off by default and opt in only"). `Config::Defaults()` gives every row `run: false`; a config that turns on none - no `lenses` key included - has the one problem that names the lenses, so `config`, `freeze` and `run` all say so, the template ships every lens off, and `init` tells the consumer to turn lenses on (Config, Commands, ReviewFixes, the Goldens' config cases). A consumer config without a `lenses` key needs one.
|
||||
- **The commit-truth lens** (`repo-facts.md`, 2026-10-31): Gate 4, after the proofs, so the messages and documents are read once the code they describe has settled. Its block kinds are text, doc and slot: the message units keyed on their text, the base and the code up to their commit (`MessageDeps`); a documentation unit on its file, its normalised text, the base and the code of the whole range (`DocKeyParts`); a diffstat on its text alone; `whole:order` and `whole:absent` in slot passes of its own (a lens with no whole pass asks the slots named in its row; never stored). A documentation file's diff header and a part the commit writes nothing in are mechanical n/a. Budgets 60/40 for a text or doc block, 50/35 for a slot; one read. The owner's gate is 5. The other lenses' prompts, keys and plans are untouched (CommitsMode, CodeKeys, Runs, the Goldens' series and gates).
|
||||
|
||||
**Known quirks, kept:**
|
||||
- the label-to-filename collision in `out/`;
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Crafter Review
|
||||
|
||||
The pre-push review pipeline: adversarial lenses review a frozen commit range, one `claude --print` process per inventory unit, before the range is pushed. Four lenses in a fixed order: the approach, the range exercised in the running application through the Playwright MCP server, and side by side the static bug proof and the security review. The repository's config says which of them run and what each gets: the browser, the local Supabase database, both, or neither.
|
||||
The pre-push review pipeline: adversarial lenses review a frozen commit range, one `claude --print` process per inventory unit, before the range is pushed. Five lenses in a fixed order: the approach, the range exercised in the running application through the Playwright MCP server, side by side the static bug proof and the security review, and last the commit truth of the messages and the documentation. The repository's config says which of them run and what each gets: the browser, the local Supabase database, both, or neither.
|
||||
|
||||
| Gate | Lens | Question |
|
||||
|---|---|---|
|
||||
|
|
@ -8,7 +8,8 @@ The pre-push review pipeline: adversarial lenses review a frozen commit range, o
|
|||
| 2 | `repo-functional` (a whole pass gating code blocks; the `browser` or `browser+supabase` environment) | in the running application, does the range do what its code and messages say, and does it leave the flows next to it working? |
|
||||
| 3 | `repo-bug-prover` (code blocks, static; with the database a proof may also run there) | a demonstrated defect in the range's code |
|
||||
| 3 | `repo-security` (code blocks, static, beside the bug-prover) | a demonstrated gap in a protection the repository promises |
|
||||
| 4 | the owner | approves the exact frozen hash, then the push happens |
|
||||
| 4 | `repo-facts` (message blocks, documentation blocks and the `whole:order` / `whole:absent` slots, static) | every claim in every commit message, and in the documentation the range adds or changes, proven or gone |
|
||||
| 5 | the owner | approves the exact frozen hash, then the push happens |
|
||||
|
||||
crafter-review's prompts, reuse keys, exit codes and shared files are pinned byte for byte by its goldens (`tests/Goldens`): a review one driver started, another resumes and reuses, and every driver on a machine sees the others' passes on its seat board.
|
||||
|
||||
|
|
|
|||
|
|
@ -163,6 +163,16 @@ namespace Crafter::Review::Plan {
|
|||
bool Contains(std::span<const std::string> list, std::string_view item) {
|
||||
return std::ranges::contains(list, item);
|
||||
}
|
||||
|
||||
// any(l[:1] in ('+', '-') and l[1:].strip() for l in code_norm(text).split('\n')):
|
||||
// does the commit add or remove a line with something on it in this
|
||||
// part of a documentation hunk?
|
||||
bool WritesIn(std::string_view text) {
|
||||
for (const std::string& line : Dyn::Split(CodeNorm(text), "\n")) {
|
||||
if ((line.starts_with('+') || line.starts_with('-')) && !Dyn::Strip(line.substr(1)).empty()) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
std::vector<ModelTier>& Models() {
|
||||
|
|
@ -193,6 +203,7 @@ namespace Crafter::Review::Plan {
|
|||
{"Gate 1 — Approach", {"repo-better-solution.md"}},
|
||||
{"Gate 2 — Functional", {FunctionalLens}},
|
||||
{"Gate 3 — Proof", {"repo-bug-prover.md", "repo-security.md"}},
|
||||
{"Gate 4 — Commit truth", {FactsLens}},
|
||||
};
|
||||
return gates;
|
||||
}
|
||||
|
|
@ -213,8 +224,11 @@ namespace Crafter::Review::Plan {
|
|||
|
||||
Budget BudgetFor(std::string_view kind, std::string_view unitKind, std::string_view lensFile) {
|
||||
if (kind == "whole") return lensFile == FunctionalLens ? Budget{220, 120} : Budget{180, 105};
|
||||
if (unitKind != "code") throw Dyn::Error("KeyError", Dyn::Repr(std::format("block:{}", unitKind)));
|
||||
return lensFile == FunctionalLens ? Budget{90, 60} : Budget{60, 40};
|
||||
if (kind == "slot") return {50, 35};
|
||||
if (unitKind == "code") return lensFile == FunctionalLens ? Budget{90, 60} : Budget{60, 40};
|
||||
if (unitKind == "doc") return {60, 40};
|
||||
if (unitKind == "text") return lensFile == FactsLens ? Budget{60, 40} : Budget{50, 30};
|
||||
throw Dyn::Error("KeyError", Dyn::Repr(std::format("block:{}", unitKind)));
|
||||
}
|
||||
|
||||
std::string_view DefaultEffort() {
|
||||
|
|
@ -251,8 +265,9 @@ namespace Crafter::Review::Plan {
|
|||
return path && IsDocPath(*path);
|
||||
}
|
||||
|
||||
// The lens is not consulted (yet): no lens of this pipeline reads a
|
||||
// documentation file since the facts lens was removed.
|
||||
// Every lens of this pipeline is a repo lens: the units of a documentation
|
||||
// file are "doc" for each of them - the commit-truth lens reads them, the
|
||||
// code lenses do not.
|
||||
std::string_view KindFor(std::string_view, const Json& unit) {
|
||||
const std::string_view kind = UnitKind(unit);
|
||||
return kind == "code" && IsDocUnit(Dyn::At(unit, "id").get_ref<const std::string&>()) ? "doc" : kind;
|
||||
|
|
@ -361,6 +376,38 @@ namespace Crafter::Review::Plan {
|
|||
if (!unitsFilter.empty() && !Contains(unitsFilter, id)) continue;
|
||||
const std::string text = UnitText(lines, unit);
|
||||
const std::optional<std::string> path = UnitPath(id);
|
||||
if (kind == "slot") {
|
||||
// A lens with no whole pass asks each of its own slots in a
|
||||
// pass of its own; another lens's whole slot is not its.
|
||||
if (!std::ranges::contains(lens.slots, std::string_view(id))) continue;
|
||||
const Budget budget = BudgetFor("slot", "slot", lensFile);
|
||||
plan.blocks.push_back({
|
||||
{"label", std::format("{}:slot:{}", name, id)},
|
||||
{"lens", name},
|
||||
{"lensFile", lensFile},
|
||||
{"kind", "slot"},
|
||||
{"unit", id},
|
||||
{"unitKind", "slot"},
|
||||
{"expectedIds", {id}},
|
||||
{"unitSha", Sha(text)},
|
||||
{"model", ModelFor(lens.model)},
|
||||
{"maxTurns", budget.maxTurns},
|
||||
{"timeoutMin", budget.timeoutMin},
|
||||
});
|
||||
continue;
|
||||
}
|
||||
if (kind == "doc" && id.ends_with("#hdr")) {
|
||||
// The diff header of a documentation file names the file, its
|
||||
// blobs and its mode: no sentence, no claim.
|
||||
plan.blocks.push_back(Mechanical(name, lensFile, unit, text, "mechanical: diff header of a documentation file"));
|
||||
continue;
|
||||
}
|
||||
if (kind == "doc" && !WritesIn(text)) {
|
||||
// A part of a hunk that holds only context (the lines above a
|
||||
// section the commit adds): nothing was written here.
|
||||
plan.blocks.push_back(Mechanical(name, lensFile, unit, text, "mechanical: context only - the commit adds and removes no line in this part"));
|
||||
continue;
|
||||
}
|
||||
if (!lens.perCommitSlots && (id.ends_with(":outside") || id.ends_with(":absent"))) {
|
||||
// The functional lens judges adjacent flows and what is missing
|
||||
// in its whole pass, not per commit.
|
||||
|
|
@ -570,9 +617,54 @@ namespace Crafter::Review::Plan {
|
|||
std::format("functional:{}", ident));
|
||||
}
|
||||
|
||||
std::pair<std::string, std::vector<std::string>> DocKeyParts(std::string_view unitId, const CommitCode& cc) {
|
||||
const std::size_t colon = unitId.find(':');
|
||||
if (colon == std::string_view::npos) throw Dyn::Error("ValueError", "not enough values to unpack (expected 2, got 1)");
|
||||
const std::string_view rest = unitId.substr(colon + 1);
|
||||
const std::size_t hash = rest.rfind('#');
|
||||
return {std::format("doc:{}", hash == std::string_view::npos ? rest : rest.substr(0, hash)), {std::format("base={}", cc.base), std::format("range={}", cc.range)}};
|
||||
}
|
||||
|
||||
std::vector<std::string> MessageDeps(std::string_view unitId, const CommitCode& cc) {
|
||||
// ^p(\d+):(subject|msg)$; `$` also matches before a final newline.
|
||||
const std::optional<std::size_t> prefix = PkPrefix(unitId);
|
||||
if (!prefix) return {};
|
||||
std::string_view name = unitId.substr(*prefix);
|
||||
if (name.ends_with('\n')) name.remove_suffix(1);
|
||||
if (name != "subject" && name != "msg") return {};
|
||||
const std::string_view number = unitId.substr(1, *prefix - 2);
|
||||
const std::optional<std::int64_t> k = Dyn::ParseInt(number);
|
||||
if (!k) throw Dyn::Error("ValueError", Dyn::IntError(number));
|
||||
const auto upto = cc.upto.find(*k);
|
||||
if (upto == cc.upto.end()) throw Dyn::Error("KeyError", std::to_string(*k));
|
||||
return {std::format("base={}", cc.base), std::format("upto={}", upto->second)};
|
||||
}
|
||||
|
||||
std::optional<std::string> DocReuseKey(const Json& job, std::string_view lensFile, std::string_view unitText, const CommitCode& cc,
|
||||
std::span<const std::string> inForceIds, std::string_view charterSha, std::int32_t promptVersion,
|
||||
std::string_view environment) {
|
||||
auto [ident, deps] = DocKeyParts(Dyn::At(job, "unit").get_ref<const std::string&>(), cc);
|
||||
std::ranges::move(EnvDeps(lensFile, environment), std::back_inserter(deps));
|
||||
return ReuseKey(job, lensFile, CodeNorm(unitText), deps, {inForceIds.begin(), inForceIds.end()}, charterSha, promptVersion, ident);
|
||||
}
|
||||
|
||||
std::optional<std::string> TextReuseKey(const Json& job, std::string_view lensFile, std::string_view unitText, const CommitCode& cc,
|
||||
std::span<const std::string> inForceIds, std::string_view charterSha, std::int32_t promptVersion,
|
||||
std::string_view environment) {
|
||||
std::vector<std::string> deps = MessageDeps(Dyn::At(job, "unit").get_ref<const std::string&>(), cc);
|
||||
std::ranges::move(EnvDeps(lensFile, environment), std::back_inserter(deps));
|
||||
return ReuseKey(job, lensFile, unitText, deps, {inForceIds.begin(), inForceIds.end()}, charterSha, promptVersion);
|
||||
}
|
||||
|
||||
std::optional<std::string> BlockReuseKey(const Json& job, std::string_view lensFile, std::string_view unitText, const CommitCode& cc,
|
||||
std::span<const std::string> inForceIds, std::string_view charterSha, std::int32_t promptVersion,
|
||||
std::string_view environment) {
|
||||
// A slot pass and a whole pass have no block key (ReuseKey's own
|
||||
// rule, before any id is parsed: a `whole:*` id has no commit number).
|
||||
if (Dyn::At(job, "kind") != "block") return std::nullopt;
|
||||
const Json& unitKind = Dyn::Get(job, "unitKind");
|
||||
if (unitKind == "doc") return DocReuseKey(job, lensFile, unitText, cc, inForceIds, charterSha, promptVersion, environment);
|
||||
if (unitKind == "text") return TextReuseKey(job, lensFile, unitText, cc, inForceIds, charterSha, promptVersion, environment);
|
||||
if (LensOf(lensFile).key == "functional") {
|
||||
return FunctionalReuseKey(job, lensFile, unitText, cc, inForceIds, charterSha, promptVersion, environment);
|
||||
}
|
||||
|
|
@ -657,7 +749,9 @@ namespace Crafter::Review::Plan {
|
|||
const auto layer = lensLayerBytes.find(lens);
|
||||
const double lensTokens = static_cast<double>(layer == lensLayerBytes.end() ? 3000 : layer->second) / tpb;
|
||||
const double tail = static_cast<double>(tailBytes) / tpb;
|
||||
const std::int64_t turns = kind == "whole" ? 40 : (Dyn::At(job, "lensFile") == FunctionalLens ? 25 : 10);
|
||||
const Json& unitKind = Dyn::Get(job, "unitKind");
|
||||
const std::int64_t turns = kind == "whole" ? 40 : kind == "slot" ? 10 : Dyn::At(job, "lensFile") == FunctionalLens ? 25
|
||||
: unitKind == "code" ? 10 : (Dyn::At(job, "lensFile") == FactsLens ? 12 : 8);
|
||||
if (seen.insert({lens, kind}).second) cacheWrite += prefix + lensTokens;
|
||||
cacheWrite += tail + static_cast<double>(turns * 3000);
|
||||
std::vector<double> reads;
|
||||
|
|
|
|||
|
|
@ -37,6 +37,25 @@ namespace Crafter::Review::Prompt {
|
|||
"Judge the in-repository users and callers of what this commit changes, outside its own diff - at the base (`git -C <tree> show <base>:<path>`) and at HEAD.";
|
||||
const std::string_view HintAbsent =
|
||||
"Judge what a reviewer of this repository expects in this commit and does not find: a commit body at all (a body-less commit has no pK:msg unit), a test, a version or changelog bump, a doc or packaging update, an error path, a migration for existing installs.";
|
||||
const std::string_view TextBlockHintRepo =
|
||||
"Within this unit, verify every claim against this commit's diff and the repository at HEAD and at the base; a device, field or network claim against the evidence paths INPUTS names, or report it unverifiable.";
|
||||
const std::string_view DocBlockHintRepo =
|
||||
"This unit is documentation the commit adds or changes: a diff hunk of a documentation file, or one heading-delimited part of it. Judge the lines the commit adds (\"+\"): every claim in them must hold for the repository as THIS commit leaves it "
|
||||
"(`git -C <tree> show <sha>:<path>`, the sha on this patch's `From` line) and, where the document at HEAD still carries it, at HEAD too - a later commit of the range may have made it false without updating it; a device, field or network claim against the evidence paths INPUTS names, or report it unverifiable. "
|
||||
"Context lines are context and removed lines are not claims, except a context sentence this commit's own change makes false.";
|
||||
|
||||
std::string_view SlotDuty(std::string_view lensFile, std::string_view unitId) {
|
||||
if (lensFile != Plan::FactsLens) return {};
|
||||
if (unitId == "whole:order") {
|
||||
return "; for this lens that means each message describes its own commit alone - nothing a message claims is delivered by a later commit of the range, and no message describes the state a later commit creates";
|
||||
}
|
||||
if (unitId == "whole:absent") {
|
||||
return "; for this lens that means what the range as a whole implies but no message states - a behaviour change no message mentions, a version bump without its changelog line - "
|
||||
"and documentation the range leaves stale: a sentence of the repository's README or other documentation at HEAD, written by the range or older, that the range's code makes false "
|
||||
"(a behaviour, an option, a path or a command described as it was before), or a list in it that something the range adds belongs in and is missing from";
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
namespace {
|
||||
// An optional string as the prompt prints it: None when absent.
|
||||
|
|
@ -202,12 +221,14 @@ namespace Crafter::Review::Prompt {
|
|||
const std::string id = Dyn::Str(Dyn::At(u, "id"));
|
||||
if (kind == "slot") {
|
||||
const std::string description = Dyn::Replace(Text(Dyn::At(u, "excerpt")), "FIXED SLOT: ", "");
|
||||
return std::format("SCOPE - SLOT PASS, unit \"{}\": this unit has no lines; it stands for a question about the whole artifact: {}. Answer it for your lens. Coverage: exactly this one unit.\n{}",
|
||||
id, description, GenericVocab);
|
||||
return std::format("SCOPE - SLOT PASS, unit \"{}\": this unit has no lines; it stands for a question about the whole artifact: {}. Answer it for your lens{}. Coverage: exactly this one unit.\n{}",
|
||||
id, description, SlotDuty(lensFile, id), GenericVocab);
|
||||
}
|
||||
const Json& range = Dyn::At(u, "lines");
|
||||
const std::int64_t a = IntItem(range, 0);
|
||||
const std::int64_t b = IntItem(range, 1);
|
||||
const Json& unitKind = Dyn::Get(job, "unitKind");
|
||||
const bool prose = unitKind == "text" || unitKind == "doc";
|
||||
std::string_view hint;
|
||||
if (id.ends_with(":outside")) {
|
||||
hint = HintOutside;
|
||||
|
|
@ -215,10 +236,15 @@ namespace Crafter::Review::Prompt {
|
|||
hint = HintAbsent;
|
||||
} else if (lensFile == Plan::FunctionalLens) {
|
||||
hint = FunctionalBlockHint;
|
||||
} else if (unitKind == "text") {
|
||||
hint = TextBlockHintRepo;
|
||||
} else if (unitKind == "doc") {
|
||||
hint = DocBlockHintRepo;
|
||||
}
|
||||
const std::string text = Plan::UnitText(lines, u);
|
||||
const std::string scope = std::format("SCOPE - BLOCK PASS, unit \"{}\" [lines {}-{}]: judge ONLY this unit, with your full attention by your charter - a hunk against the standard - reading the rest of the artifact only as context. Do not re-litigate the approach as a whole{}. Coverage: exactly this one unit.",
|
||||
id, a, b, wholeSummary.empty() ? "" : " - the whole pass of this lens concluded as stated above");
|
||||
const std::string scope = std::format("SCOPE - BLOCK PASS, unit \"{}\" [lines {}-{}]: judge ONLY this unit, with your full attention by your charter - {} - reading the rest of the artifact only as context. Do not re-litigate the approach as a whole{}. Coverage: exactly this one unit.",
|
||||
id, a, b, prose ? "a claim at its source, a sentence against its proof obligation" : "a hunk against the standard",
|
||||
wholeSummary.empty() ? "" : " - the whole pass of this lens concluded as stated above");
|
||||
return std::format("{}{}\n\nUNIT TEXT (verbatim, lines {}-{}):\n{}\n\n{}", scope, hint.empty() ? std::string() : std::format("\n{}", hint), a, b, text, GenericVocab);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -498,8 +498,7 @@ namespace Crafter::Review::Run {
|
|||
}
|
||||
const std::vector<std::string> exclude = Strings(cfg.at("functional").at("exclude"));
|
||||
notReviewed = Plan::NotReviewed(units, lines, running, exclude);
|
||||
log(std::format("inventory: {} units ({}); {} units no lens that runs reads (text, documentation, the units of a lens the config turns off)", units.size(), Dyn::RelPath(invPath.string(), repo.string()),
|
||||
notReviewed.size()));
|
||||
log(std::format("inventory: {} units ({}); {} units no lens that runs reads", units.size(), Dyn::RelPath(invPath.string(), repo.string()), notReviewed.size()));
|
||||
if (a.units) {
|
||||
std::set<std::string> known;
|
||||
for (const Json& u : units) known.insert(Dyn::Str(u.at("id")));
|
||||
|
|
|
|||
|
|
@ -12,6 +12,9 @@ import :Json;
|
|||
// progress.json and the prompts are made from them.
|
||||
export namespace Crafter::Review::Plan {
|
||||
constexpr std::string_view FunctionalLens = "repo-functional.md";
|
||||
// The commit-truth lens (2026-10-31, the fp6 pipeline's repo-facts): the
|
||||
// messages and the documentation of the range.
|
||||
constexpr std::string_view FactsLens = "repo-facts.md";
|
||||
// The reasoning effort when --effort is not given (2026-10-02: the push
|
||||
// gates run at xhigh).
|
||||
constexpr std::string_view Effort = "xhigh";
|
||||
|
|
@ -40,16 +43,21 @@ export namespace Crafter::Review::Plan {
|
|||
constexpr std::array<std::string_view, 2> ApproachSlots = {"whole:order", "whole:absent"};
|
||||
constexpr std::array<std::string_view, 1> BehaviourSlots = {"whole:behaviour"};
|
||||
constexpr std::array<std::string_view, 1> CodeBlocks = {"code"};
|
||||
constexpr std::array<std::string_view, 3> TruthBlocks = {"text", "doc", "slot"};
|
||||
|
||||
// One row of LENSES. whole: "" (none), "first-repo" (the approach) or
|
||||
// "functional" (the range exercised in the running application); every
|
||||
// whole pass gates the lens's block passes and covers the slots named in
|
||||
// `slots`. block: the unit kinds the lens zooms into. reuse: per-unit reuse
|
||||
// across hashes; wholeReuse: the clean whole pass carries while the code
|
||||
// of the range is unchanged. key: what a block is keyed on — "code" (its
|
||||
// own hunk and the base; 2026-10-06) or "functional" (its hunk AND the
|
||||
// code of the whole range: what the browser shows for a hunk depends on
|
||||
// every other hunk). reads: the lens's code units need Reads clean reads.
|
||||
// `slots`. A lens with no whole pass and "slot" among its block kinds
|
||||
// asks each slot named in `slots` in a slot pass of its own (the
|
||||
// commit-truth lens: whole:order and whole:absent). block: the unit kinds
|
||||
// the lens zooms into. reuse: per-unit reuse across hashes; wholeReuse:
|
||||
// the clean whole pass carries while the code of the range is unchanged.
|
||||
// key: what a code block is keyed on — "code" (its own hunk and the base;
|
||||
// 2026-10-06) or "functional" (its hunk AND the code of the whole range:
|
||||
// what the browser shows for a hunk depends on every other hunk); a text
|
||||
// or doc block is keyed by its kind whatever the lens's key (TextReuseKey,
|
||||
// DocReuseKey). reads: the lens's code units need Reads clean reads.
|
||||
// empirical: the lens's evidence is observation, so it needs an
|
||||
// environment with an application under test, and a whole pass that never
|
||||
// used the instrument is not evidence. perCommitSlots false: the lens
|
||||
|
|
@ -71,7 +79,7 @@ export namespace Crafter::Review::Plan {
|
|||
std::string_view env;
|
||||
};
|
||||
// In the order the dump and the prompts list them.
|
||||
constexpr std::array<Lens, 4> Lenses = {{
|
||||
constexpr std::array<Lens, 5> Lenses = {{
|
||||
{"repo-better-solution.md", 1, "first-repo", ApproachSlots, CodeBlocks, "fable", true, true, true, "code", false, true, "none"},
|
||||
{FunctionalLens, 2, "functional", BehaviourSlots, CodeBlocks, "opus", true, true, false, "functional", true, false, "browser"},
|
||||
{"repo-bug-prover.md", 3, "", {}, CodeBlocks, "opus", true, false, true, "code", false, true, "none"},
|
||||
|
|
@ -79,6 +87,13 @@ export namespace Crafter::Review::Plan {
|
|||
// protections the repository promises? The bug-prover's shape beside
|
||||
// it in Gate 3: one gate stop for both proofs.
|
||||
{"repo-security.md", 3, "", {}, CodeBlocks, "opus", true, false, true, "code", false, true, "none"},
|
||||
// The commit-truth lens (2026-10-31): every claim in every commit
|
||||
// message, and in the documentation the range adds or changes, proven
|
||||
// or gone. The text and doc units and the two whole slots, in a gate
|
||||
// of its own after the proofs, so the messages and documents are read
|
||||
// once the code they describe has settled. One read: its units are
|
||||
// prose, keyed on their text and the code they describe.
|
||||
{FactsLens, 4, "", ApproachSlots, TruthBlocks, "opus", true, false, false, "text", false, true, "none"},
|
||||
}};
|
||||
bool IsLens(std::string_view file);
|
||||
// LENSES[file]; Dyn::Error (KeyError) for a file that is no lens.
|
||||
|
|
@ -234,7 +249,26 @@ export namespace Crafter::Review::Plan {
|
|||
std::optional<std::string> FunctionalReuseKey(const Json& job, std::string_view lensFile, std::string_view unitText, const CommitCode& cc,
|
||||
std::span<const std::string> inForceIds, std::string_view charterSha, std::int32_t promptVersion,
|
||||
std::string_view environment = {});
|
||||
// By the lens's `key` kind.
|
||||
// (ident, deps) of a doc unit - a hunk of a documentation file, or one
|
||||
// heading-delimited part of it: keyed on its file and its own normalised
|
||||
// text, like a code hunk, and on the base and the code of the whole
|
||||
// range - a sentence must hold for the code as its commit leaves it AND
|
||||
// as the range leaves it, so any code change in the range asks it again
|
||||
// (2026-10-06).
|
||||
std::pair<std::string, std::vector<std::string>> DocKeyParts(std::string_view unitId, const CommitCode& cc);
|
||||
// What a message unit (pK:subject, pK:msg) depends on beside its own text:
|
||||
// the base and the code of the range up to and including its commit - a
|
||||
// message describes its commit's change and the state it changes, so a
|
||||
// fix to that code asks the message again. Empty for every other unit (a
|
||||
// diffstat states only its own numbers).
|
||||
std::vector<std::string> MessageDeps(std::string_view unitId, const CommitCode& cc);
|
||||
std::optional<std::string> DocReuseKey(const Json& job, std::string_view lensFile, std::string_view unitText, const CommitCode& cc,
|
||||
std::span<const std::string> inForceIds, std::string_view charterSha, std::int32_t promptVersion,
|
||||
std::string_view environment = {});
|
||||
std::optional<std::string> TextReuseKey(const Json& job, std::string_view lensFile, std::string_view unitText, const CommitCode& cc,
|
||||
std::span<const std::string> inForceIds, std::string_view charterSha, std::int32_t promptVersion,
|
||||
std::string_view environment = {});
|
||||
// By the job's unit kind (doc, text), else by the lens's `key` kind.
|
||||
std::optional<std::string> BlockReuseKey(const Json& job, std::string_view lensFile, std::string_view unitText, const CommitCode& cc,
|
||||
std::span<const std::string> inForceIds, std::string_view charterSha, std::int32_t promptVersion,
|
||||
std::string_view environment = {});
|
||||
|
|
|
|||
|
|
@ -23,6 +23,13 @@ export namespace Crafter::Review::Prompt {
|
|||
// KIND_HINTS_REPO
|
||||
extern const std::string_view HintOutside;
|
||||
extern const std::string_view HintAbsent;
|
||||
// The block hints of the commit-truth lens (2026-10-31): a message unit,
|
||||
// and a unit of a documentation file.
|
||||
extern const std::string_view TextBlockHintRepo;
|
||||
extern const std::string_view DocBlockHintRepo;
|
||||
// SLOT_DUTIES: what a slot pass of a lens covers beyond the slot's
|
||||
// question; empty for every other lens and slot.
|
||||
std::string_view SlotDuty(std::string_view lensFile, std::string_view unitId);
|
||||
|
||||
// _common-repo.md with the artifact kind filled in.
|
||||
std::string SystemText();
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@
|
|||
<release version="1.0.0" date="2026-10-31">
|
||||
<description>
|
||||
<p>First release: the pre-push review gates in C++26, shipped as a Flatpak.</p>
|
||||
<p>The commit-truth lens, repo-facts, reads the commit messages and the documentation of the range as Gate 4.</p>
|
||||
</description>
|
||||
</release>
|
||||
</releases>
|
||||
|
|
|
|||
151
process/lenses/repo-facts.md
Normal file
151
process/lenses/repo-facts.md
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
# Lens: repo-facts
|
||||
|
||||
You are an adversarial fact-checker. Your job is to DISPROVE every claim made
|
||||
in the commit messages of the range under review, and in the documentation
|
||||
the range adds or changes (README, AGENTS.md: the units of a `.md` file). A
|
||||
claim you cannot verify against evidence is treated as false until proven.
|
||||
These commits go into a public repository's permanent history and into a
|
||||
release users install; its documentation is what they read.
|
||||
|
||||
Inputs: the frozen range, the repository at the range HEAD and at the base,
|
||||
its history (`git -C <tree> log`), and the evidence paths INPUTS names (the
|
||||
proof artifacts for anything tested on a device or seen on a network).
|
||||
|
||||
## Method
|
||||
|
||||
1. **Enumerate every factual claim** in each commit message — behavior
|
||||
claims ("X now does Y", "fixes Z"), claims about the previous behavior,
|
||||
claims about users, devices, carriers, networks or field observations,
|
||||
"tested" claims, version and changelog statements, references to specs,
|
||||
tickets, other projects or other people's work. For each: find the proof
|
||||
in the diff, the tree, the history, or a capture — or flag it.
|
||||
2. **Code claims are verified against the diff and the tree at HEAD** —
|
||||
what the message says the change does must be what the change does, and
|
||||
what it says about the state before must hold at the base
|
||||
(`git -C <tree> show <base>:<path>`). A message that describes the
|
||||
intended behavior rather than the implemented one is a finding.
|
||||
3. **"Tested" claims trace to a capture** naming what ran, where, and on
|
||||
which build. When INPUTS names no evidence path, a tested claim that the
|
||||
diff and tree cannot substantiate is unverifiable — a finding whose cure
|
||||
is the capture or the deletion of the claim.
|
||||
4. **Overclaims are cured by TESTING, not rewording.** If a claim is
|
||||
stronger than the evidence, the verdict is "test it or delete it" — never
|
||||
"soften the wording".
|
||||
5. **"Doesn't affect X" / "no user-visible change" requires 100% hard
|
||||
proof** — enumerate every user of the touched path in the repository and
|
||||
show each is unaffected, or the claim goes.
|
||||
6. **Negative existence claims** ("nothing else uses this", "no caller
|
||||
passes NULL", "there is no test for this", "the first X to …") are
|
||||
auditable only against a search you ran: cite the grep and its result in
|
||||
the basis.
|
||||
7. **Evidence quality rules**: exact-event match, never substring
|
||||
pattern-match; levels-vs-baseline, never mere presence; an unexplained
|
||||
nonzero baseline is evidence, not noise — attribute it before relying on
|
||||
any "nothing happens" claim.
|
||||
8. **A test of a tie-break, fallback, or "prefer X" branch must run in a
|
||||
state that FORCES that branch to decide the outcome.** "The test passed"
|
||||
and "the new code ran" are different claims. Name the state that forces
|
||||
it, and show the evidence came from that state.
|
||||
9. **References** to a spec, an RFC, a ticket, an upstream commit or another
|
||||
project: verify that what is on disk (the repository, its installed or
|
||||
vendored dependencies, the evidence paths) says what the message says it
|
||||
says; a document not on disk makes the claim unverifiable — a note naming
|
||||
what would settle it, never an assumed pass.
|
||||
10. **History claims** ("since 0.3.4", "introduced by <sha>", "reverts",
|
||||
"the previous commit"): archaeology in the repository's own history,
|
||||
cited by sha; a message in commit K that describes what commit K+1 does
|
||||
is a finding on both.
|
||||
11. **Documentation the range adds or changes** — a unit of a `.md` file: a
|
||||
diff hunk, or one heading-delimited part of it — is read like a message,
|
||||
by the rules above, with three differences. (a) The claims are the lines
|
||||
the commit adds. A context line is context and a removed line claims
|
||||
nothing — except a context sentence that this commit's own change makes
|
||||
false, which is a finding. (b) A sentence must hold for the repository
|
||||
as ITS commit leaves it (`git -C <tree> show <sha>:<path>`, the sha on
|
||||
that patch's `From` line): a later commit that makes it true does not
|
||||
cure it. Where the document at HEAD still carries the sentence it must
|
||||
hold at HEAD too: a later commit of the range that makes it false
|
||||
without updating it is a finding on the unit that wrote it. (c) An
|
||||
instruction — a command, a path, a variable, an option, a reference to a
|
||||
section — is a claim that it exists and works as written: check the
|
||||
file, the flag, the unit, the target section in the tree at that commit;
|
||||
what only a run on a device or a network can settle traces to a capture,
|
||||
or is unverifiable. This lens checks what a document says, not what it
|
||||
could also say: an omission is a finding only where the text claims
|
||||
completeness ("every", "all", "only", "never", a list given as the
|
||||
list). Documentation the range did not touch and its code makes false is
|
||||
the `whole:absent` slot's.
|
||||
|
||||
## Verdict
|
||||
|
||||
Per `prepush.md` §Verdicts — objective lens: no materiality defence. Any
|
||||
unproven or disproven claim is FIX FIRST whoever would notice — and the cure
|
||||
is evidence (a test, a capture, a read of the referenced code) or deletion of
|
||||
the claim, never a hedge word. Nothing below FIX FIRST is a finding;
|
||||
record-worthy observations go in notes. Never grade a finding.
|
||||
|
||||
## Calibration (append dated entries at the bottom)
|
||||
|
||||
- Seeded 2026-09-25 from the earlier facts lens: the 2026-07-20
|
||||
overclaim rule (Method 4), the 2026-07-24 forced-branch rule (Method 8),
|
||||
the 2026-08-30 captured-search rule for negative existence claims (Method
|
||||
6), and the 2026-09-16 coverage contract: when the prompt carries an
|
||||
INVENTORY (a mechanical partition: every non-blank line in exactly one
|
||||
unit, fixed `whole:*`/`pK:outside`/`pK:absent` slots for what is absent),
|
||||
return one `{item, verdict, basis}` per unit AFTER the review: `clear`
|
||||
only when every claim in the unit was checked at its source (basis = the
|
||||
tree ref, the command run, the capture path) — a plausible reading is not
|
||||
a check; `defect` with a finding carrying `items:[id]`; `immaterial` with
|
||||
a note carrying `items:[id]`; `n/a` with the reason (never a paragraph of
|
||||
claims). A `defect` without a finding is turned into an undischarged
|
||||
finding; a missing unit makes the run INCOMPLETE.
|
||||
- 2026-09-25 (firing proof at creation): a fixture repository with a
|
||||
planted range - commit 1 adds `median()` whose message claims an
|
||||
empty-list guard and the mean of the two middle values that the diff does
|
||||
not implement, commit 2 "fix the median crash on an empty list" adds the
|
||||
guard - and a corrected single-commit range. Single-unit `--only`/`--units`
|
||||
runs: block pass on `p1:msg` -> FIX FIRST with 3 findings: the even-count
|
||||
claim disproven against the diff, the guard claim disproven against
|
||||
`git show <sha>:stats.py` of that commit (and the next commit's message
|
||||
conceding it - cited on both units), the "as the other helpers document"
|
||||
reference unsupported (no docstring, README silent); the corrected message
|
||||
-> SEND-READY, `clear`, with an `info` note that no tested/field claim
|
||||
existed to trace. 7 / 7 turns, ~$0.45 / $0.25. No denied call.
|
||||
- 2026-10-06 (documentation is this lens's since driver 2026-10-06.3 -
|
||||
Method 11; firing proof with this charter): a fixture repository, two
|
||||
commits. Planted: commit 1's README says "`spread(xs)` returns the largest
|
||||
value minus the smallest" while spread() arrives in commit 2; commit 2's
|
||||
README says the three functions raise `TypeError` (the code raises
|
||||
ValueError); commit 2's message says "the test suite covers an unsorted
|
||||
sequence" (it does not). Single-unit `--only`/`--units` run on
|
||||
`p1:README.md#1.2`, `p2:README.md#1.2`, `p2:msg` -> FIX FIRST, three
|
||||
findings: the first disproven at commit 1's own sha (`git show
|
||||
<sha>:stats.py` has no spread; `git grep spread <sha>` hits only the
|
||||
README line) and cited on `whole:order` too, with an `info` note that the
|
||||
sentence is accurate at HEAD and only its placement is wrong; the second
|
||||
against `_require` and the tests of the same commit; the third against
|
||||
the two spread tests. The corrected range, units `p2:README.md#1.1`,
|
||||
`#1.2`, `p2:msg` -> SEND-READY, `clear` x3; that the README's list leaves
|
||||
out `is_finite()` came back as a `pre-existing` note, not a finding (the
|
||||
list does not claim to be complete). Opus 5.5 xhigh, 10 / 17 turns, $0.49
|
||||
/ $0.64, under a minute each. No denied call.
|
||||
- 2026-10-06 (Method 11(b) reworded and the `whole:absent` duty widened
|
||||
after the soundness review of the same day - driver 2026-10-06.4;
|
||||
firing proof): the same fixture repository; commit 1's README says
|
||||
median() returns "the mean of the two middle values for an even count"
|
||||
(true there), commit 2 makes median() return the lower middle value and
|
||||
touches no documentation. Run on `p1:README.md#1` and `whole:absent` ->
|
||||
FIX FIRST: the sentence flagged on the unit that wrote it ("true at the
|
||||
commit that adds it but false at HEAD ... does not update README.md"),
|
||||
the slot flagged the stale README and that spread(), which the range
|
||||
adds, is missing from the list the range itself keeps. With the sentence
|
||||
updated in commit 2, `p1:README.md#1`, `p2:README.md#1`, `whole:absent`
|
||||
-> SEND-READY: "replaced the sentence, so it does not carry to HEAD"; the
|
||||
list's old gap (is_finite) again a `pre-existing` note. Opus 5.5 xhigh,
|
||||
13 / 14 turns, $0.51 / $0.72. No denied call.
|
||||
- 2026-10-31 — carried into this pipeline with the runtime examples made
|
||||
generic (the evidence paths INPUTS names stand for a journal's ledger and
|
||||
captures; a device for a phone); the rules are the same. Here it is Gate 4, after the
|
||||
proofs, so the messages and documents are read once the code they
|
||||
describe has settled, and it is off until a repository's config turns it
|
||||
on. Its firing proof in this pipeline is still owed (the owner's go).
|
||||
|
|
@ -3,9 +3,10 @@
|
|||
Scope: every push of a commit range from a repository set up for this
|
||||
pipeline (`crafter-review init`) — a release channel with real users, where a push (or
|
||||
the pull request it opens) is the next release. The `/prepush` skill drives
|
||||
this file. The mechanics: freeze → hash → gates → the owner; four lenses,
|
||||
this file. The mechanics: freeze → hash → gates → the owner; five lenses,
|
||||
the functional one as Gate 2 (Jorijn, 2026-10-15), the security lens beside
|
||||
the bug-prover in Gate 3 (2026-10-22).
|
||||
the bug-prover in Gate 3 (2026-10-22), the commit-truth lens as Gate 4
|
||||
(2026-10-31).
|
||||
|
||||
Nothing is pushed without the owner's explicit approval of the exact frozen
|
||||
hash. Local commits parked unpushed are fine and normal.
|
||||
|
|
@ -15,7 +16,7 @@ Terms: *the pipeline* is crafter-review, the Flatpak
|
|||
`flatpak run net.catcrafts.CrafterReview <command>`, because the Bash tool sees no
|
||||
alias (a native build is run by its path instead). *The repository* is the
|
||||
consumer checkout the driver runs in; *the owner* is whoever approves the
|
||||
hash at Gate 4.
|
||||
hash at Gate 5.
|
||||
|
||||
## Inputs — what the lenses see, and nothing else
|
||||
|
||||
|
|
@ -231,7 +232,7 @@ therefore the hash: re-freeze with `--replace` into the same dir
|
|||
the new HEAD before the rerun. A fix during a run happens in another
|
||||
worktree, never in the tree the lenses read.
|
||||
|
||||
## Gates 1–3 (the driver) and Gate 4 (the owner)
|
||||
## Gates 1–4 (the driver) and Gate 5 (the owner)
|
||||
|
||||
| Gate | Lens | Question |
|
||||
|---|---|---|
|
||||
|
|
@ -239,7 +240,8 @@ therefore the hash: re-freeze with `--replace` into the same dir
|
|||
| 2 Functional | `repo-functional` (whole pass gating code blocks; the `browser` or `browser+supabase` environment; absent when the config turns it off) | in the running application, does the range do what its code and messages say, and does it leave the flows next to it working? |
|
||||
| 3 Proof | `repo-bug-prover` (code blocks, static; with the `supabase` environment a proof may also be run in the database) | a demonstrated functional defect in the range's code |
|
||||
| 3 Proof | `repo-security` (code blocks, static, beside the bug-prover; with the `supabase` or a browser environment a demonstration may also be run in the database or the application) | a demonstrated gap in a protection the repository promises: a check, policy or grant the range's code lacks, weakens or removes |
|
||||
| 4 | the owner | approves this exact hash, then the push happens |
|
||||
| 4 Commit truth | `repo-facts` (message blocks, documentation blocks and the `whole:order` / `whole:absent` slots, static) | every claim in every commit message, and in the documentation the range adds or changes, proven or gone |
|
||||
| 5 | the owner | approves this exact hash, then the push happens |
|
||||
|
||||
Every lens runs on Opus 5.5 at effort `xhigh` (`--effort` overrides).
|
||||
There is no width of the machine by default (2026-10-31, Jorijn: "there is
|
||||
|
|
@ -299,15 +301,17 @@ passes; coverage is fail-closed per inventory unit (`inventory commits`:
|
|||
outputs after a kill, budgets, `--only` / `--units` test-only runs (PARTIAL,
|
||||
never a pass), the dump at `gate-state/commits-<hash8>-run-<n>.json`.
|
||||
|
||||
**Who reads what.** The four lenses read code: hunks, file headers and (the
|
||||
static lenses) the per-commit slots `pK:outside` and `pK:absent`. The
|
||||
text units — subjects, messages, diffstats — and the units of documentation
|
||||
files (`*.md`) are read by no lens of this pipeline (the facts lens was
|
||||
removed, 2026-10-15); the dump lists them under `coverage.notReviewed` and
|
||||
the summary says so. A lens the config turns off (`lenses.<name>.run =
|
||||
false`) reads nothing: the units only it would have read (its whole slot —
|
||||
`whole:behaviour` for the functional lens) are listed there too, never as
|
||||
holes. The functional lens plans no pass on a diff header, on a
|
||||
**Who reads what.** The four code lenses read code: hunks, file headers and
|
||||
(the static lenses) the per-commit slots `pK:outside` and `pK:absent`. The
|
||||
commit-truth lens reads the prose: the text units — subjects, messages,
|
||||
diffstats — and the units of documentation files (`*.md`), each
|
||||
heading-delimited part of a hunk on its own, and it asks `whole:order` and
|
||||
`whole:absent` in slot passes of its own; a documentation file's diff header
|
||||
and a part the commit writes nothing in are a mechanical `n/a`. A lens the
|
||||
config turns off (`lenses.<name>.run = false`) reads nothing: the units only it would have read (its whole slot —
|
||||
`whole:behaviour` for the functional lens; the prose for the commit-truth
|
||||
lens) are listed under `coverage.notReviewed`, never as holes, and the
|
||||
summary says so. The functional lens plans no pass on a diff header, on a
|
||||
unit of a file matching `functional.exclude`, or on the per-commit slots (its
|
||||
whole pass judges adjacent flows and missing deliveries range-wide): each is
|
||||
a mechanical `n/a` with its basis.
|
||||
|
|
@ -334,6 +338,16 @@ the functional lens):
|
|||
range changed, or when a documentation file of their commit is added,
|
||||
deleted or renamed (`absent` also when the commit gains or loses its body).
|
||||
The commit's position is in the key;
|
||||
- a message (`pK:subject`, `pK:msg`), for the commit-truth lens: its own
|
||||
text, the base and the code of the range up to and including its commit —
|
||||
a fix to the code a message describes asks the message again; a diffstat
|
||||
its own text alone;
|
||||
- a unit of a documentation file, for the commit-truth lens: its file and
|
||||
its own text (normalised like a hunk), the base and the code of the whole
|
||||
range — a sentence must hold for the code as its commit leaves it and as
|
||||
the range leaves it;
|
||||
- the commit-truth lens's slot passes (`whole:order`, `whole:absent`): never
|
||||
stored, bought every run;
|
||||
- both whole passes: the code of every commit of the range in order, which
|
||||
commits have a body and which documentation files each touches. No message
|
||||
text and no documentation text. A whole pass that returned a finding is
|
||||
|
|
@ -371,7 +385,7 @@ from the previous rounds get run again until they reach x"; x is 3
|
|||
again after a finished run is the next round.
|
||||
- All gates clean with reads still owed is `CONFIRMING — NOT SEND-READY`
|
||||
(exit 2); run the same line again, on the same hash, until it says
|
||||
SEND-READY. Gate 4 is asked for a hash only then.
|
||||
SEND-READY. Gate 5 is asked for a hash only then.
|
||||
- Every finding in the dump carries `read`, the number of the read that
|
||||
found it.
|
||||
|
||||
|
|
@ -401,7 +415,7 @@ Each entry is dated, names what it `supersedes`, binds to a hash, records
|
|||
grounds, and lists the artifact `anchors` its grounds depend on with the
|
||||
count of occurrences at binding. The register is
|
||||
`<out>/gate-state/dispositions.json` — the authority and the record; the
|
||||
manifest's rounds table names each discharge with its grounds for Gate 4.
|
||||
manifest's rounds table names each discharge with its grounds for Gate 5.
|
||||
Carry across a hash change
|
||||
is mechanical: an entry is in force when bound to the current hash, carried
|
||||
when `carry` is true and every anchor still occurs its recorded count of
|
||||
|
|
@ -419,7 +433,7 @@ the push, the rest are banked per kind.
|
|||
A new disposition in force re-keys every unit, so a waiver costs every static
|
||||
code unit its three reads again.
|
||||
|
||||
## Gate 4 — the owner, then the push
|
||||
## Gate 5 — the owner, then the push
|
||||
|
||||
Present the frozen patches themselves (never summaries), the rounds table,
|
||||
every discharged finding with its grounds, every contest, every `unverified`
|
||||
|
|
@ -592,3 +606,6 @@ repository.
|
|||
5-hour column. Then (Jorijn: "there is no need to do a global cap anymore,
|
||||
the cap should be 128 per available account") the machine-wide width
|
||||
went: `--concurrency` caps only when given.
|
||||
- 2026-10-31 — **the commit-truth lens as Gate 4** (Jorijn: "fix that in
|
||||
Crafter.Review"). The owner's gate is 5. Its firing proof in this pipeline
|
||||
is owed (the owner's go).
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
name: prepush
|
||||
description: Pre-push gates for a repository set up for crafter-review (`crafter-review init`). Runs the pipeline's procedure end to end — Gate 0 build/tests + freeze (identity, trailers by the config's rule, clean tree, no merges, fast-forward), the application and the database under test brought up from the frozen tree, the driver run of the lenses the config enables (repo-better-solution → repo-functional with the Playwright MCP → repo-bug-prover and repo-security side by side) via crafter-review, the fix/rerun loop, the owner's approval of the hash, verify, push. Nothing is pushed unreviewed or unapproved.
|
||||
description: Pre-push gates for a repository set up for crafter-review (`crafter-review init`). Runs the pipeline's procedure end to end — Gate 0 build/tests + freeze (identity, trailers by the config's rule, clean tree, no merges, fast-forward), the application and the database under test brought up from the frozen tree, the driver run of the lenses the config enables (repo-better-solution → repo-functional with the Playwright MCP → repo-bug-prover and repo-security side by side → repo-facts, the commit truth) via crafter-review, the fix/rerun loop, the owner's approval of the hash, verify, push. Nothing is pushed unreviewed or unapproved.
|
||||
---
|
||||
|
||||
Follow the pipeline's procedure, `prepush.md`, EXACTLY — read it now, in
|
||||
|
|
@ -46,7 +46,7 @@ of running it:
|
|||
`supabase.dbUrl` does not answer or its applied migrations are not
|
||||
HEAD's: reset the stack from the frozen tree, after every
|
||||
`freeze --replace` too. Do not edit that tree while the run is on.
|
||||
4. **Gates 1–3**: run the printed command from the repository root in a
|
||||
4. **Gates 1–4**: run the printed command from the repository root in a
|
||||
detached background shell (`setsid nohup … > <log> 2>&1 < /dev/null &`);
|
||||
poll `~/.cache/review-gates/<hash8>-run-<n>/progress.json` and `log.txt`.
|
||||
Add `--context-paths` for the evidence dirs the messages' claims rest on.
|
||||
|
|
@ -68,11 +68,11 @@ of running it:
|
|||
`grounds`, `carry`). A `contests` entry is a lens claim: re-check its cited
|
||||
evidence first; a surviving contest on an owner entry goes to the owner.
|
||||
Notes never block — bank each per its kind; an `unverified` note names a
|
||||
manual check the owner sees at Gate 4. INCOMPLETE is never a pass: rerun.
|
||||
manual check the owner sees at Gate 5. INCOMPLETE is never a pass: rerun.
|
||||
6. **Rounds** go in `.review-gates/push/<slug>/manifest.md`
|
||||
(`flatpak run net.catcrafts.CrafterReview doc manifest` prints the
|
||||
template; test runs with `--only`/`--units` never).
|
||||
7. **Gate 4 — the owner**: present the frozen patches themselves, the rounds
|
||||
7. **Gate 5 — the owner**: present the frozen patches themselves, the rounds
|
||||
table, the discharged findings with grounds, every contest, all notes —
|
||||
then STOP. Nothing is pushed until they explicitly approve that exact hash.
|
||||
8. **Push**: `flatpak run net.catcrafts.CrafterReview freeze --slug <slug>
|
||||
|
|
|
|||
|
|
@ -26,6 +26,10 @@
|
|||
"repo-security": {
|
||||
"run": true,
|
||||
"environment": "none"
|
||||
},
|
||||
"repo-facts": {
|
||||
"run": true,
|
||||
"environment": "none"
|
||||
}
|
||||
},
|
||||
"supabase": {
|
||||
|
|
|
|||
|
|
@ -25,5 +25,5 @@ repository's rule).
|
|||
|
||||
-
|
||||
|
||||
approved: <date> @ <hash> (the owner, Gate 4 — saw the frozen patches, the discharged-findings list, contests and notes)
|
||||
approved: <date> @ <hash> (the owner, Gate 5 — saw the frozen patches, the discharged-findings list, contests and notes)
|
||||
pushed: <date> <remote>/<branch> @ <sha> (PR: <url>)
|
||||
|
|
|
|||
|
|
@ -35,6 +35,25 @@ namespace {
|
|||
return out;
|
||||
}
|
||||
|
||||
// {unit id: reuse key} of every text and doc unit under the commit-truth
|
||||
// lens (a doc file header and a context-only part are auto-n/a: no key).
|
||||
Keys TruthKeys(const fs::path& d, std::string_view env = {}) {
|
||||
const Inventory::Built built = Inventory::Build("commits", d.string());
|
||||
const Plan::CommitCode cc = Plan::CommitCodeOf(Artifact::PatchTexts(d));
|
||||
const Json& units = built.out.at("units");
|
||||
Keys out;
|
||||
for (const Json& job : Plan::PlanLens(Plan::FactsLens, units, built.lines).blocks) {
|
||||
if (job.at("kind") != "block") continue;
|
||||
const std::string id = job.at("unit").get<std::string>();
|
||||
const Json* u = nullptr;
|
||||
for (const Json& unit : units) {
|
||||
if (unit.at("id") == id) u = &unit;
|
||||
}
|
||||
out[id] = Plan::BlockReuseKey(job, Plan::FactsLens, Plan::UnitText(built.lines, *u), cc, {}, "c", 1, env).value_or("");
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::optional<std::string> WholeKey(const fs::path& d, const std::string& lens = "repo-better-solution.md", std::string_view env = {}) {
|
||||
return Plan::WholeCodeKey({{"kind", "whole"}, {"unit", nullptr}}, lens, Plan::CommitCodeOf(Artifact::PatchTexts(d)), {}, "c", 1, env);
|
||||
}
|
||||
|
|
@ -136,6 +155,42 @@ int main() {
|
|||
&& WholeKey(rw, "repo-better-solution.md", "supabase") == WholeKey(src, "repo-better-solution.md", "supabase")
|
||||
&& Changed(CodeKeys(code, "repo-bug-prover.md", "supabase"), sb) == expected);
|
||||
|
||||
// The commit-truth lens's keys (2026-10-31, the fp6 driver's doc_reuse_key
|
||||
// and message_deps): a doc unit on its file, its text, the base and the
|
||||
// code of the whole range; a message on its text, the base and the code
|
||||
// up to its commit; a diffstat on its text alone; a slot pass never.
|
||||
const Keys tbase = TruthKeys(src);
|
||||
std::set<std::string> tbaseValues;
|
||||
for (const auto& [id, key] : tbase) tbaseValues.insert(key);
|
||||
std::set<std::string> sharedWithCode;
|
||||
std::ranges::set_intersection(baseValues, tbaseValues, std::inserter(sharedWithCode, sharedWithCode.begin()));
|
||||
const std::set<std::string> truthUnits = {"p1:subject", "p1:diffstat", "p2:subject", "p2:msg", "p2:diffstat", "p2:README.md#1"};
|
||||
Check("fixture: the commit-truth lens keys the five text units and the one doc unit (the README header is n/a), every key its own, none a code key's, no slot key",
|
||||
std::set<std::string>(std::views::keys(tbase).begin(), std::views::keys(tbase).end()) == truthUnits && std::ranges::all_of(tbase, [](const auto& e) { return !e.second.empty(); })
|
||||
&& tbaseValues.size() == tbase.size() && sharedWithCode.empty());
|
||||
const Keys trw = TruthKeys(rw);
|
||||
Check("the rewrite (p2's message reworded, the patch counter bumped): the doc key and the diffstat keys hold, p2's body and both subjects are asked again",
|
||||
trw.at("p2:README.md#1") == tbase.at("p2:README.md#1") && trw.at("p1:diffstat") == tbase.at("p1:diffstat") && trw.at("p2:diffstat") == tbase.at("p2:diffstat")
|
||||
&& trw.at("p2:msg") != tbase.at("p2:msg") && trw.at("p1:subject") != tbase.at("p1:subject") && trw.at("p2:subject") != tbase.at("p2:subject"));
|
||||
const Keys tdocs = TruthKeys(docs);
|
||||
Check("a reworded README sentence: its doc unit is asked again, nothing else of this lens", Changed(tdocs, tbase) == std::set<std::string>{"p2:README.md#1"});
|
||||
const Keys tcode = TruthKeys(code);
|
||||
Check("one changed line in p2's code: the doc unit (the range is in its key) and p2's subject and body (the code up to p2) are asked again; p1's subject and the diffstats hold",
|
||||
Changed(tcode, tbase) == std::set<std::string>{"p2:README.md#1", "p2:subject", "p2:msg"}, Show(Changed(tcode, tbase)));
|
||||
Check("another base: every doc and message key changes; a diffstat depends on no code", Changed(TruthKeys(bse), tbase) == std::set<std::string>{"p2:README.md#1", "p1:subject", "p2:subject", "p2:msg"});
|
||||
Check("p2 drops its README hunk: no doc key is left", !TruthKeys(nodoc).contains("p2:README.md#1") && TruthKeys(nodoc).size() == 5);
|
||||
const Plan::CommitCode c0 = Plan::CommitCodeOf(Artifact::PatchTexts(src));
|
||||
const Plan::CommitCode c1 = Plan::CommitCodeOf(Artifact::PatchTexts(code));
|
||||
Check("doc key: a sentence depends on the code of the whole range - a later commit's code change asks an earlier commit's sentence again; a part is keyed on its file, not on its hunk or part number",
|
||||
Plan::DocKeyParts("p1:README.md#1", c0) != Plan::DocKeyParts("p1:README.md#1", c1) && Plan::DocKeyParts("p2:README.md#1", c0) == Plan::DocKeyParts("p2:README.md#3.12", c0)
|
||||
&& Plan::DocKeyParts("p2:README.md#1", c0).first != Plan::DocKeyParts("p2:AGENTS.md#1", c0).first && Plan::DocKeyParts("p2:README.md#1", c0).first == "doc:README.md"
|
||||
&& c0.docs.at(1).empty() && c0.docs.at(2).size() == 1 && c0.docs.at(2).at(0).starts_with("diff --git a/README.md b/README.md\nnew file mode"));
|
||||
Check("message key: a commit message depends on the code up to its own commit - p2's code change asks p2's subject and body again, not p1's; a diffstat, a slot and a hunk depend on no code",
|
||||
Plan::MessageDeps("p2:msg", c0) != Plan::MessageDeps("p2:msg", c1) && Plan::MessageDeps("p2:subject", c0) != Plan::MessageDeps("p2:subject", c1)
|
||||
&& Plan::MessageDeps("p1:subject", c0) == Plan::MessageDeps("p1:subject", c1) && !Plan::MessageDeps("p1:subject", c0).empty() && Plan::MessageDeps("p2:diffstat", c0).empty()
|
||||
&& Plan::MessageDeps("whole:order", c0).empty() && Plan::MessageDeps("p2:hello.py#1", c0).empty() && Plan::MessageDeps("p2:msg\n", c0) == Plan::MessageDeps("p2:msg", c0));
|
||||
Check("environment: a text or doc key in another environment is its own, and a slot job has no key", TruthKeys(src, "supabase") != tbase && AllChanged(TruthKeys(src, "supabase"), tbase)
|
||||
&& !Plan::BlockReuseKey({{"kind", "slot"}, {"unit", "whole:order"}, {"unitKind", "slot"}}, Plan::FactsLens, "", c0, {}, "c", 1));
|
||||
const Plan::CommitCode cc = Plan::CommitCodeOf(Artifact::PatchTexts(src));
|
||||
Check("whole key: both wholes have keys and differ; a block never; the bug-prover has none", WholeKey(src) && WholeKey(src, Fn) && WholeKey(src) != WholeKey(src, Fn)
|
||||
&& !WholeKey(src, "repo-bug-prover.md") && !Plan::WholeCodeKey({{"kind", "block"}, {"unit", "x"}}, "repo-better-solution.md", cc, {}, "c", 1));
|
||||
|
|
|
|||
|
|
@ -109,37 +109,45 @@ int main() {
|
|||
const auto g = Gates(Plan::GatesFor());
|
||||
std::vector<std::string> gateNames;
|
||||
for (const auto& [title, lenses] : g) gateNames.push_back(Dyn::Split(title, " — ")[0]);
|
||||
Check("gates: 1 approach, 2 functional, 3 proof - the bug-prover and the security lens side by side (2026-10-22), the owner stays Gate 4",
|
||||
gateNames == std::vector<std::string>{"Gate 1", "Gate 2", "Gate 3"} && g[0].second == std::vector<std::string>{"repo-better-solution.md"} && g[1].second == std::vector<std::string>{Fn}
|
||||
&& g[2].second == std::vector<std::string>{"repo-bug-prover.md", "repo-security.md"});
|
||||
const std::string Facts(Plan::FactsLens);
|
||||
Check("gates: 1 approach, 2 functional, 3 proof - the bug-prover and the security lens side by side (2026-10-22) - 4 commit truth (2026-10-31); the owner is Gate 5",
|
||||
gateNames == std::vector<std::string>{"Gate 1", "Gate 2", "Gate 3", "Gate 4"} && g[0].second == std::vector<std::string>{"repo-better-solution.md"} && g[1].second == std::vector<std::string>{Fn}
|
||||
&& g[2].second == std::vector<std::string>{"repo-bug-prover.md", "repo-security.md"} && g[3] == std::pair<std::string, std::vector<std::string>>{"Gate 4 — Commit truth", {Facts}});
|
||||
const std::vector<std::string> onlyFn = {Fn};
|
||||
const std::vector<std::string> security = {"repo-security.md"};
|
||||
std::set<std::string> lensSet;
|
||||
for (const Plan::Lens& lens : Plan::Lenses) lensSet.emplace(lens.file);
|
||||
using GateList = std::vector<std::pair<std::string, std::vector<std::string>>>;
|
||||
Check("mode_lenses = the four; --only keeps, --without drops, an emptied gate disappears; Gate 3 with one of its lenses dropped keeps the other",
|
||||
Check("mode_lenses = the five; --only keeps, --without drops, an emptied gate disappears; Gate 3 with one of its lenses dropped keeps the other",
|
||||
Set(Plan::ModeLenses()) == lensSet && Gates(Plan::GatesFor(onlyFn)) == GateList{{"Gate 2 — Functional", {Fn}}}
|
||||
&& Titles(Plan::GatesFor({}, onlyFn)) == std::vector<std::string>{"Gate 1 — Approach", "Gate 3 — Proof"} && Plan::GatesFor(onlyFn, onlyFn).empty()
|
||||
&& Gates(Plan::GatesFor({}, security)).back() == std::pair<std::string, std::vector<std::string>>{"Gate 3 — Proof", {"repo-bug-prover.md"}}
|
||||
&& Titles(Plan::GatesFor({}, onlyFn)) == std::vector<std::string>{"Gate 1 — Approach", "Gate 3 — Proof", "Gate 4 — Commit truth"} && Plan::GatesFor(onlyFn, onlyFn).empty()
|
||||
&& Gates(Plan::GatesFor({}, security))[2] == std::pair<std::string, std::vector<std::string>>{"Gate 3 — Proof", {"repo-bug-prover.md"}}
|
||||
&& Gates(Plan::GatesFor(security)) == GateList{{"Gate 3 — Proof", {"repo-security.md"}}});
|
||||
const std::vector<std::string> threeOff = {"repo-better-solution.md", "repo-bug-prover.md", "repo-security.md"};
|
||||
const std::vector<std::string> bugOff = {"repo-bug-prover.md"};
|
||||
Check("lenses off by config: gates_for drops the lens and its gate; --only cannot bring it back", Titles(Plan::GatesFor({}, {}, onlyFn)) == std::vector<std::string>{"Gate 1 — Approach", "Gate 3 — Proof"}
|
||||
&& Plan::GatesFor(onlyFn, {}, onlyFn).empty() && Gates(Plan::GatesFor({}, {}, threeOff)) == GateList{{"Gate 2 — Functional", {Fn}}}
|
||||
&& Gates(Plan::GatesFor({}, {}, bugOff)).back() == std::pair<std::string, std::vector<std::string>>{"Gate 3 — Proof", {"repo-security.md"}});
|
||||
const std::vector<std::string> onlyFacts = {Facts};
|
||||
Check("lenses off by config: gates_for drops the lens and its gate; --only cannot bring it back", Titles(Plan::GatesFor({}, {}, onlyFn)) == std::vector<std::string>{"Gate 1 — Approach", "Gate 3 — Proof", "Gate 4 — Commit truth"}
|
||||
&& Plan::GatesFor(onlyFn, {}, onlyFn).empty() && Gates(Plan::GatesFor({}, {}, threeOff)) == GateList{{"Gate 2 — Functional", {Fn}}, {"Gate 4 — Commit truth", {Facts}}}
|
||||
&& Gates(Plan::GatesFor({}, {}, bugOff))[2] == std::pair<std::string, std::vector<std::string>>{"Gate 3 — Proof", {"repo-security.md"}}
|
||||
&& Gates(Plan::GatesFor(onlyFacts)) == GateList{{"Gate 4 — Commit truth", {Facts}}} && Titles(Plan::GatesFor({}, {}, onlyFacts)).size() == 3);
|
||||
const std::vector<std::string> allLenses(lensSet.begin(), lensSet.end());
|
||||
const std::set<std::string> textAndDoc = {"p1:subject", "p1:diffstat", "p2:subject", "p2:msg", "p2:diffstat", "p2:README.md#hdr", "p2:README.md#1"};
|
||||
const std::set<std::string> nrAll = Set(Plan::NotReviewed(units, lines, Plan::ModeLenses()));
|
||||
Check("not reviewed (every lens runs): the text units and the documentation units", nrAll == std::set<std::string>{"p1:subject", "p1:diffstat", "p2:subject", "p2:msg", "p2:diffstat", "p2:README.md#hdr", "p2:README.md#1"},
|
||||
Show(nrAll));
|
||||
std::vector<std::string> withoutFacts;
|
||||
for (const Plan::Lens& lens : Plan::Lenses) {
|
||||
if (lens.file != Facts) withoutFacts.emplace_back(lens.file);
|
||||
}
|
||||
Check("not reviewed (every lens runs): only the documentation file's header, a mechanical n/a of the commit-truth lens; without that lens the text units and the documentation units",
|
||||
nrAll == std::set<std::string>{"p2:README.md#hdr"} && Set(Plan::NotReviewed(units, lines, withoutFacts)) == textAndDoc, Show(nrAll));
|
||||
std::vector<std::string> statics;
|
||||
for (const Plan::Lens& lens : Plan::Lenses) {
|
||||
if (lens.file != Fn) statics.emplace_back(lens.file);
|
||||
}
|
||||
std::set<std::string> plusBehaviour = nrAll;
|
||||
plusBehaviour.insert("whole:behaviour");
|
||||
std::set<std::string> plusStatic = nrAll;
|
||||
std::set<std::string> plusStatic = textAndDoc;
|
||||
plusStatic.insert({"whole:order", "whole:absent", "p1:outside", "p1:absent", "p2:outside", "p2:absent", "p1:hello.py#hdr", "p2:hello.py#hdr"});
|
||||
Check("not reviewed is derived from the lenses that run: the functional lens off adds whole:behaviour; only the functional lens adds the static slots and the file headers (a mechanical n/a is no reading)",
|
||||
Check("not reviewed is derived from the lenses that run: the functional lens off adds whole:behaviour; only the functional lens adds the static slots, the file headers and the prose (a mechanical n/a is no reading)",
|
||||
Set(Plan::NotReviewed(units, lines, statics)) == plusBehaviour && Set(Plan::NotReviewed(units, lines, onlyFn)) == plusStatic, Show(Set(Plan::NotReviewed(units, lines, onlyFn))));
|
||||
std::set<std::string> codeIds;
|
||||
for (const std::string& id : ids) {
|
||||
|
|
@ -166,6 +174,36 @@ int main() {
|
|||
Check("repo-security: the bug-prover's shape - no whole pass, the same code units, static blocks keyed on the code, three reads, none by default, not empirical",
|
||||
sec.whole.is_null() && unitsOf(sec.blocks) == codeIds && sameShape && s.gate == 3
|
||||
&& std::ranges::all_of(sec.blocks, [](const Json& j) { return j.at("maxTurns") == 60 && j.at("timeoutMin") == 40 && j.at("model") == Plan::ModelFor("opus"); }));
|
||||
// The commit-truth lens (2026-10-31): the prose of the range and its two
|
||||
// whole slots, in passes of their own.
|
||||
const Plan::LensPlan truth = Plan::PlanLens(Facts, units, lines);
|
||||
std::map<std::string, std::string> truthKinds;
|
||||
for (const Json& j : truth.blocks) truthKinds[j.at("unit").get<std::string>()] = j.at("kind").get<std::string>();
|
||||
const std::map<std::string, std::string> wantTruth = {{"p1:subject", "block"}, {"p1:diffstat", "block"}, {"p2:subject", "block"}, {"p2:msg", "block"}, {"p2:diffstat", "block"},
|
||||
{"p2:README.md#hdr", "auto-na"}, {"p2:README.md#1", "block"}, {"whole:order", "slot"}, {"whole:absent", "slot"}};
|
||||
const Json& truthMsg = Job(truth.blocks, "block", "p2:msg");
|
||||
const Json& truthDoc = Job(truth.blocks, "block", "p2:README.md#1");
|
||||
const Json& truthOrder = Job(truth.blocks, "slot", "whole:order");
|
||||
const Plan::Lens& t = Plan::LensOf(Facts);
|
||||
Check("repo-facts: no whole pass; the text units, the documentation units and its two whole slots (whole:behaviour is the functional lens's); a doc file header is a mechanical n/a with its own basis; "
|
||||
"text and doc blocks 60/40, a slot 50/35, one read, opus, Gate 4, none by default",
|
||||
truth.whole.is_null() && truthKinds == wantTruth && Job(truth.blocks, "auto-na", "p2:README.md#hdr").at("basis") == "mechanical: diff header of a documentation file"
|
||||
&& truthMsg.at("unitKind") == "text" && truthMsg.at("maxTurns") == 60 && truthMsg.at("timeoutMin") == 40 && truthDoc.at("unitKind") == "doc" && truthDoc.at("maxTurns") == 60
|
||||
&& truthOrder.at("unitKind") == "slot" && truthOrder.at("maxTurns") == 50 && truthOrder.at("timeoutMin") == 35 && truthOrder.at("label") == "repo-facts:slot:whole:order"
|
||||
&& truthOrder.at("expectedIds") == Json::array({"whole:order"}) && !t.reads && t.gate == 4 && t.model == "opus" && t.env == "none" && !t.empirical && t.whole.empty()
|
||||
&& std::ranges::all_of(truth.blocks, [](const Json& j) { return j.at("kind") == "auto-na" || j.at("model") == Plan::ModelFor("opus"); }),
|
||||
Dyn::ReprValue(Json(truthKinds)));
|
||||
const Plan::Budget textElsewhere = Plan::BudgetFor("block", "text", "repo-bug-prover.md");
|
||||
const Plan::Budget docBudget = Plan::BudgetFor("block", "doc", Facts);
|
||||
Check("budgets: a text block of another lens 50/30, a doc block 60/40, a slot 50/35; an unknown unit kind is still a KeyError", textElsewhere.maxTurns == 50 && textElsewhere.timeoutMin == 30
|
||||
&& docBudget.maxTurns == 60 && docBudget.timeoutMin == 40 && Plan::BudgetFor("slot", "slot", Facts).maxTurns == 50 && [] {
|
||||
try {
|
||||
Plan::BudgetFor("block", "nope", "repo-bug-prover.md");
|
||||
return false;
|
||||
} catch (const Dyn::Error&) {
|
||||
return true;
|
||||
}
|
||||
}());
|
||||
const std::vector<std::string> exclude = {"**/*.test.*", "docs/**"};
|
||||
const Plan::LensPlan functional = Plan::PlanLens(Fn, units, lines, {}, exclude);
|
||||
std::map<std::string, std::string> kinds;
|
||||
|
|
@ -223,9 +261,22 @@ int main() {
|
|||
const std::string tailFunctional = Prompt::Tail(functional.whole, units, lines);
|
||||
Check("whole tails: the approach text lists order+absent; the functional text lists behaviour and names the APPLICATION UNDER TEST", tailApproach.contains("for THIS repository")
|
||||
&& tailApproach.contains("whole:order") && tailFunctional.contains("as a USER") && tailFunctional.contains("whole:behaviour") && !tailFunctional.contains("whole:order"));
|
||||
const std::string hunkTail = Prompt::Tail(Job(prover.blocks, "block", "p2:hello.py#1"), units, lines);
|
||||
Check("block tails: the functional hint on a functional block, none on a static hunk, the repo hints on the slots",
|
||||
Prompt::Tail(functionalBlock, units, lines).contains("judges ONE hunk in the running application") && !Prompt::Tail(Job(prover.blocks, "block", "p2:hello.py#1"), units, lines).contains("judges ONE hunk")
|
||||
Prompt::Tail(functionalBlock, units, lines).contains("judges ONE hunk in the running application") && !hunkTail.contains("judges ONE hunk")
|
||||
&& Prompt::Tail(Job(prover.blocks, "block", "p1:absent"), units, lines).contains("body-less commit") && Prompt::Tail(Job(prover.blocks, "block", "p1:outside"), units, lines).contains("users and callers"));
|
||||
const std::string msgTail = Prompt::Tail(truthMsg, units, lines);
|
||||
const std::string docTail = Prompt::Tail(truthDoc, units, lines);
|
||||
const std::string orderTail = Prompt::Tail(truthOrder, units, lines);
|
||||
const std::string absentTail = Prompt::Tail(Job(truth.blocks, "slot", "whole:absent"), units, lines);
|
||||
Check("repo-facts tails: a message block judges a claim at its source with the text hint, a doc block with the documentation hint, each slot pass the slot's question with this lens's duty; "
|
||||
"a code block's tail stays a hunk against the standard, and a slot of another lens would carry no duty",
|
||||
msgTail.contains("a claim at its source, a sentence against its proof obligation") && msgTail.contains("verify every claim against this commit's diff") && !msgTail.contains("a hunk against the standard")
|
||||
&& docTail.contains("a claim at its source") && docTail.contains("This unit is documentation the commit adds or changes") && docTail.contains("UNIT TEXT (verbatim")
|
||||
&& orderTail.starts_with("SCOPE - SLOT PASS, unit \"whole:order\"") && orderTail.contains("Answer it for your lens; for this lens that means each message describes its own commit alone")
|
||||
&& absentTail.contains("documentation the range leaves stale") && absentTail.ends_with(Prompt::GenericVocab)
|
||||
&& hunkTail.contains("a hunk against the standard") && !hunkTail.contains("a claim at its source") && Prompt::SlotDuty("repo-bug-prover.md", "whole:order").empty()
|
||||
&& Prompt::SlotDuty(Facts, "whole:behaviour").empty());
|
||||
const std::string ll = Prompt::LensLayer(Fn, "CHARTER TEXT", "DISPOSITIONS BLOCK", ab, "SUMMARY");
|
||||
const std::string sp = Prompt::SystemPrompt(st, rl, ll);
|
||||
const std::string up = Prompt::UserPrompt(tailFunctional, {{"maxTurns", 220}});
|
||||
|
|
|
|||
|
|
@ -86,8 +86,8 @@ int main() {
|
|||
const Json defaults = Config::Defaults();
|
||||
std::set<std::string> defaultNames;
|
||||
for (auto it = defaults.at("lenses").begin(); it != defaults.at("lenses").end(); ++it) defaultNames.insert(it.key());
|
||||
const Json environments = {{"repo-better-solution.md", "none"}, {fn, "browser"}, {"repo-bug-prover.md", "none"}, {"repo-security.md", "none"}};
|
||||
Check("lenses: the defaults run all four with none/browser/none/none - plan.LENSES' env; the table names every lens of plan.LENSES", loaded.problems.empty() && allRun
|
||||
const Json environments = {{"repo-better-solution.md", "none"}, {fn, "browser"}, {"repo-bug-prover.md", "none"}, {"repo-security.md", "none"}, {Plan::FactsLens, "none"}};
|
||||
Check("lenses: the defaults run all five with none/browser/none/none/none - plan.LENSES' env; the table names every lens of plan.LENSES", loaded.problems.empty() && allRun
|
||||
&& Config::LensesOff(loaded.config).empty() && Dyn::Equal(Config::Environments(loaded.config), environments) && defaultNames == names && defaultsMatch, Show(loaded.problems));
|
||||
put({{"lenses", {{"repo-functional", {{"run", false}}}, {"repo-bug-prover", {{"environment", "browser"}}}}}});
|
||||
loaded = load();
|
||||
|
|
|
|||
166
tests/Goldens/derive.py
Normal file
166
tests/Goldens/derive.py
Normal file
|
|
@ -0,0 +1,166 @@
|
|||
#!/usr/bin/env python3
|
||||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
|
||||
"""Re-derive the goldens that follow from the lens table and the config defaults.
|
||||
|
||||
tests/Goldens/fixture/pipeline.json holds the recorded answers the suite
|
||||
compares against. Where an entry follows from the lens table or the config
|
||||
defaults, this script derives it from the rules instead of a hand edit: the
|
||||
config cases through the binary's own `config` command (what Load, Validate,
|
||||
LensesOff and Environments answer), the commit-truth lens's plans, the
|
||||
not-reviewed lists and the estimates from the recorded units by the lens's
|
||||
rules, and the gates from the table. Before it writes, it asserts that the
|
||||
recorded table still reproduces the recorded entries, so a drift in the
|
||||
derivation itself shows up as a failure here.
|
||||
|
||||
python3 -I tests/Goldens/derive.py <checkout> <crafter-review binary>
|
||||
|
||||
Run from anywhere; review the diff of pipeline.json. The suite never runs it.
|
||||
"""
|
||||
import sys, json, hashlib, subprocess, pathlib, tempfile, os, shutil
|
||||
|
||||
W = pathlib.Path(sys.argv[1]); EXE = sys.argv[2]
|
||||
G = W / 'tests/Goldens/fixture/pipeline.json'
|
||||
g = json.loads(G.read_text())
|
||||
TABLE = [('repo-better-solution.md', 'none'), ('repo-functional.md', 'browser'), ('repo-bug-prover.md', 'none'), ('repo-security.md', 'none'), ('repo-facts.md', 'none')]
|
||||
GATES = [('Gate 1 — Approach', ['repo-better-solution.md']), ('Gate 2 — Functional', ['repo-functional.md']),
|
||||
('Gate 3 — Proof', ['repo-bug-prover.md', 'repo-security.md']), ('Gate 4 — Commit truth', ['repo-facts.md'])]
|
||||
name_of = lambda f: f.replace('.md', '')
|
||||
sha = lambda s: hashlib.sha256(s.encode()).hexdigest()
|
||||
|
||||
# ---- config: every case through the binary --------------------------------------------
|
||||
c = g['config']
|
||||
scratch = pathlib.Path(tempfile.mkdtemp(prefix='goldens-patch-'))
|
||||
def run_config(repo, extra=()):
|
||||
r = subprocess.run([EXE, *extra, 'config'], cwd=repo, capture_output=True, text=True, env={**os.environ, 'GIT_CONFIG_GLOBAL': '/dev/null', 'GIT_CONFIG_SYSTEM': '/dev/null'})
|
||||
lines = r.stdout.split('\n'); body, warnings, problems = [], [], []
|
||||
tail = False
|
||||
for l in lines:
|
||||
if l.startswith('warning: '): warnings.append(l[9:]); tail = True
|
||||
elif l.startswith('problem: '): problems.append(l[9:]); tail = True
|
||||
elif not tail: body.append(l)
|
||||
shown = json.loads('\n'.join(body))
|
||||
return shown, warnings, problems, r.returncode
|
||||
def spelled(text, repo):
|
||||
text = text.replace(str(repo), '<repo>')
|
||||
return text.replace(f'`{EXE} slots`', '`<invocation> slots`')
|
||||
# LensEntry: the config's entry for the lens, else its default one - the
|
||||
# defaults as the binary answers them, so this holds whatever they are.
|
||||
default_dir = scratch / 'defaults'; default_dir.mkdir()
|
||||
DEFAULT_LENSES = run_config(default_dir)[0]['lenses']
|
||||
def entry_of(lenses, name):
|
||||
e = lenses.get(name) if isinstance(lenses, dict) else None
|
||||
return e if isinstance(e, dict) else DEFAULT_LENSES[name]
|
||||
changed_problems = {}
|
||||
for case in c['cases']:
|
||||
repo = scratch / case['name']; repo.mkdir(parents=True)
|
||||
if case['body'] is not None:
|
||||
(repo / '.review-gates').mkdir()
|
||||
(repo / '.review-gates' / 'config.json').write_text(case['body'], encoding='utf-8', newline='')
|
||||
shown, warnings, problems, code = run_config(repo)
|
||||
path = shown.pop('_path')
|
||||
old_problems = case['problems']
|
||||
case['problems'] = [spelled(x, repo) for x in problems]
|
||||
case['warnings'] = [spelled(x, repo) for x in warnings]
|
||||
case['config'] = json.loads(spelled(json.dumps(shown), repo))
|
||||
case['path'] = None if path is None else spelled(path, repo)
|
||||
lenses = shown.get('lenses')
|
||||
case['lensesOff'] = [f for f, _ in TABLE if entry_of(lenses, name_of(f)).get('run') is False]
|
||||
case['environments'] = {f: entry_of(lenses, name_of(f)).get('environment', 'none') for f, _ in TABLE}
|
||||
if old_problems != case['problems']: changed_problems[case['name']] = (old_problems, case['problems'])
|
||||
missing = scratch / 'missing'; missing.mkdir()
|
||||
_, _, mprob, _ = run_config(missing, ['--config', str(missing / 'none.json')])
|
||||
c['missing'] = [spelled(x, missing) for x in mprob]
|
||||
# The defaults as the binary answers them: the 'none' case is Defaults() with
|
||||
# _path null and the database password masked, which the record keeps unmasked.
|
||||
none = next(case for case in c['cases'] if case['name'] == 'none')
|
||||
defaults = json.loads(json.dumps(none['config']))
|
||||
defaults['supabase']['dbUrl'] = c['defaults']['supabase']['dbUrl']
|
||||
c['defaults'] = defaults
|
||||
c['example'] = json.loads((W / 'process/templates/config.json').read_text())
|
||||
ex = scratch / 'example'; ex.mkdir()
|
||||
_, _, eprob, _ = run_config(ex, ['--config', str(W / 'process/templates/config.json')])
|
||||
c['exampleProblems'] = [spelled(x, ex) for x in eprob]
|
||||
shutil.rmtree(scratch)
|
||||
print('config cases with changed problems:', len(changed_problems))
|
||||
for n, (o, new) in changed_problems.items(): print(' ', n, '\n -', o, '\n +', new)
|
||||
|
||||
# ---- series: the facts lens's plans, not_reviewed, the estimate ------------------------
|
||||
OPUS = 'claude-opus-5-5'
|
||||
def facts_plan(entry):
|
||||
inv = entry['inventory']['units']; info = entry['units']
|
||||
blocks = []
|
||||
for u, meta in zip(inv, info):
|
||||
uid = u['id']; kind = meta['kind']
|
||||
if kind == 'code' and meta['doc']: kind = 'doc'
|
||||
common = {'lens': 'repo-facts', 'lensFile': 'repo-facts.md'}
|
||||
if kind == 'slot':
|
||||
if uid not in ('whole:order', 'whole:absent'): continue
|
||||
blocks.append({'label': f'repo-facts:slot:{uid}', **common, 'kind': 'slot', 'unit': uid, 'unitKind': 'slot', 'expectedIds': [uid], 'unitSha': sha(meta['text']),
|
||||
'model': OPUS, 'maxTurns': 50, 'timeoutMin': 35})
|
||||
elif kind == 'doc':
|
||||
if uid.endswith('#hdr'):
|
||||
blocks.append({'label': f'repo-facts:na:{uid}', **common, 'kind': 'auto-na', 'unit': uid, 'expectedIds': [uid], 'unitSha': sha(meta['text']), 'basis': 'mechanical: diff header of a documentation file'})
|
||||
elif not any(l[:1] in ('+', '-') and l[1:].strip() for l in meta['norm'].split('\n')):
|
||||
blocks.append({'label': f'repo-facts:na:{uid}', **common, 'kind': 'auto-na', 'unit': uid, 'expectedIds': [uid], 'unitSha': sha(meta['text']), 'basis': 'mechanical: context only - the commit adds and removes no line in this part'})
|
||||
else:
|
||||
blocks.append({'label': f'repo-facts:block:{uid}', **common, 'kind': 'block', 'unit': uid, 'unitKind': 'doc', 'expectedIds': [uid], 'unitSha': sha(meta['text']), 'model': OPUS, 'maxTurns': 60, 'timeoutMin': 40})
|
||||
elif kind == 'text':
|
||||
blocks.append({'label': f'repo-facts:block:{uid}', **common, 'kind': 'block', 'unit': uid, 'unitKind': 'text', 'expectedIds': [uid], 'unitSha': sha(meta['text']), 'model': OPUS, 'maxTurns': 60, 'timeoutMin': 40})
|
||||
return {'whole': None, 'blocks': blocks}
|
||||
def estimate(jobs, prefix_bytes=123457, lens_layer_bytes={'repo-better-solution': 4100, 'repo-functional': 9000}, tail_bytes=1800, tpb=3.7, sys_tokens=32000, out_per_turn=1200):
|
||||
P = prefix_bytes / tpb + sys_tokens
|
||||
total = {'cacheRead': 0.0, 'cacheWrite': 0.0, 'output': 0.0, 'processes': 0}
|
||||
seen = set()
|
||||
for j in jobs:
|
||||
if j['kind'] == 'auto-na': continue
|
||||
L = lens_layer_bytes.get(j['lens'], 3000) / tpb; T = tail_bytes / tpb
|
||||
k = 40 if j['kind'] == 'whole' else 10 if j['kind'] == 'slot' else 25 if j['lensFile'] == 'repo-functional.md' else 10 if j.get('unitKind') == 'code' else (12 if j['lensFile'] == 'repo-facts.md' else 8)
|
||||
key = (j['lens'], j['kind'])
|
||||
if key not in seen: seen.add(key); total['cacheWrite'] += P + L
|
||||
total['cacheWrite'] += T + k * 3000
|
||||
total['cacheRead'] += sum(P + L + T + 3000 * i for i in range(k))
|
||||
total['output'] += k * out_per_turn
|
||||
total['processes'] += 1
|
||||
total['costUnits'] = 1.25 * total['cacheWrite'] + 0.1 * total['cacheRead'] + 5 * total['output']
|
||||
total['listUsd'] = round((6.25 * total['cacheWrite'] + 0.5 * total['cacheRead'] + 25 * total['output']) / 1e6, 2)
|
||||
return total
|
||||
for name, entry in g['series'].items():
|
||||
if 'plans' not in entry: continue
|
||||
jobs = []
|
||||
recorded = []
|
||||
for f, _ in TABLE:
|
||||
plan = entry['plans'].get(f)
|
||||
if plan is None: continue
|
||||
here = ([plan['whole']] if plan['whole'] is not None else []) + plan['blocks']
|
||||
recorded += here
|
||||
if f != 'repo-facts.md': jobs += here
|
||||
assert estimate(recorded) == entry['estimate'], f'{name}: the estimate formula does not reproduce the golden'
|
||||
fp = facts_plan(entry)
|
||||
entry['plans']['repo-facts.md'] = fp
|
||||
entry['plans']['repo-facts.md:exclude'] = fp
|
||||
covered = {b['unit'] for b in fp['blocks'] if b['kind'] != 'auto-na'}
|
||||
nr = entry['notReviewed']
|
||||
nr['all'] = [i for i in nr['all'] if i not in covered]
|
||||
nr['static'] = [i for i in nr['static'] if i not in covered]
|
||||
entry['estimate'] = estimate(jobs + fp['blocks'])
|
||||
print(f"series {name}: facts plan {len(fp['blocks'])} jobs ({sum(b['kind']=='auto-na' for b in fp['blocks'])} n/a), not reviewed all -> {nr['all']}")
|
||||
|
||||
# ---- gates ---------------------------------------------------------------------------------
|
||||
def gates_for(table, only, without, off):
|
||||
out = []
|
||||
for title, lenses in table:
|
||||
kept = [l for l in lenses if (not only or l in only) and l not in without and l not in off]
|
||||
if kept: out.append([title, kept])
|
||||
return out
|
||||
for args, old in g['gates']:
|
||||
only, without, off = args.get('only', []), args.get('without', []), args.get('off', [])
|
||||
assert old in (gates_for(GATES[:3], only, without, off), gates_for(GATES, only, without, off)), f'gates {args}: neither table reproduces the golden'
|
||||
new_gates = [[args, gates_for(GATES, args.get('only', []), args.get('without', []), args.get('off', []))] for args, _ in g['gates']]
|
||||
print('gates entries changed:', sum(1 for (a, o), (_, n) in zip(g['gates'], new_gates) if o != n), 'of', len(new_gates))
|
||||
g['gates'] = new_gates
|
||||
|
||||
# ---- the charter ----------------------------------------------------------------------------
|
||||
g['prompts']['charterSha']['repo-facts.md'] = sha((W / 'process/lenses/repo-facts.md').read_text())
|
||||
G.write_text(json.dumps(g, indent=1, ensure_ascii=False) + '\n', encoding='utf-8')
|
||||
print('written', G)
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -61,7 +61,7 @@ for a in "$@"; do
|
|||
prev=$a
|
||||
done
|
||||
lens=$(sed -n 's/^# Lens: //p' "$sys" | head -n 1)
|
||||
ids=$(sed -n -e 's/^SCOPE - BLOCK PASS, unit "\([^"]*\)".*/\1/p' -e 's/^- \(whole:[a-z]*\) \[.*/\1/p')
|
||||
ids=$(sed -n -e 's/^SCOPE - BLOCK PASS, unit "\([^"]*\)".*/\1/p' -e 's/^SCOPE - SLOT PASS, unit "\([^"]*\)".*/\1/p' -e 's/^- \(whole:[a-z]*\) \[.*/\1/p')
|
||||
echo "$$ $lens" $ids >> "$FAKE_DIR/calls"
|
||||
echo "{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"$sid\",\"model\":\"$model\",\"mcp_servers\":[],\"tools\":[\"Read\"]}"
|
||||
echo '{"type":"assistant","message":{"role":"assistant","content":[{"type":"text","text":"reading"}]}}'
|
||||
|
|
@ -197,13 +197,14 @@ echo "{\"type\":\"result\",\"subtype\":\"success\",\"is_error\":false,\"result\"
|
|||
{"REVIEW_GATES_BOARD", (base / "board.json").string()}, {"FAKE_DIR", fakeDir.string()}});
|
||||
|
||||
// A repository with one commit to review, the functional lens off (it
|
||||
// needs a running application).
|
||||
// needs a running application) and the commit-truth lens off (it gets
|
||||
// the last scenario).
|
||||
fs::create_directories(repo);
|
||||
Git(repo, {"init", "-q", "-b", "main"});
|
||||
Git(repo, {"config", "user.email", "me@example.com"});
|
||||
Git(repo, {"config", "user.name", "Me"});
|
||||
Write(repo / ".gitignore", ".review-gates/push/\n.review-gates/trees/\n");
|
||||
Write(repo / ".review-gates" / "config.json", R"({"lenses": {"repo-functional": {"run": false}}})");
|
||||
Write(repo / ".review-gates" / "config.json", R"({"lenses": {"repo-functional": {"run": false}, "repo-facts": {"run": false}}})");
|
||||
Write(repo / "README.md", "a greeter\n");
|
||||
Git(repo, {"add", "-A"});
|
||||
Git(repo, {"commit", "-q", "-m", "base"});
|
||||
|
|
@ -420,6 +421,32 @@ echo "{\"type\":\"result\",\"subtype\":\"success\",\"is_error\":false,\"result\"
|
|||
Check("an error escaping a full run (a reserve that is no number) ends it with its ValueError, exit 1",
|
||||
badReserve.returnCode == 1 && badReserve.err.contains("crafter-review: ValueError: could not convert string to float: 'x'"), badReserve.err);
|
||||
|
||||
// The commit-truth lens (2026-10-31), alone: a config that turns off
|
||||
// every other lens (through --config: the tracked config file must
|
||||
// stay as committed, or the tree is dirty) runs Gate 4 - its three text
|
||||
// blocks and its two slot passes (the stand-in answers a SLOT PASS
|
||||
// too) - push-ready, with the code units, the per-commit slots and
|
||||
// the functional lens's slot listed as not reviewed.
|
||||
{
|
||||
Write(base / "facts.json", R"({"lenses": {"repo-better-solution": {"run": false}, "repo-functional": {"run": false}, "repo-bug-prover": {"run": false}, "repo-security": {"run": false}}})");
|
||||
const std::int64_t before = calls();
|
||||
std::vector<std::string> truthArgs = runArgs({"--no-reuse", "--no-salvage"});
|
||||
truthArgs.insert(truthArgs.begin() + 1, {"--config", (base / "facts.json").string()});
|
||||
const Process::Captured truth = Process::Run(truthArgs, repo, std::chrono::seconds(60), {}, Env);
|
||||
const Json dumpT = LatestDump(gateDir, h8);
|
||||
const Json& processesT = dumpT.is_object() ? dumpT.at("processes") : Json::array();
|
||||
std::set<std::string> labels;
|
||||
for (const Json& p : processesT) labels.insert(Text(p.at("label")));
|
||||
const Json notReviewedT = dumpT.is_object() ? dumpT.at("coverage").at("notReviewed") : Json::array();
|
||||
auto notReviewed = [&](std::string_view id) { return std::ranges::any_of(notReviewedT, [&](const Json& x) { return x == id; }); };
|
||||
Check("the commit-truth lens alone: Gate 4 runs its three text blocks and its two slot passes (5 calls) - SEND-READY, exit 0; the code units, the per-commit slots and whole:behaviour are not reviewed",
|
||||
truth.returnCode == 0 && calls() - before == 5 && dumpT.is_object() && dumpT.at("overall") == "SEND-READY" && dumpT.at("stoppedAt").is_null()
|
||||
&& labels == std::set<std::string>{"repo-facts:block:p1:subject", "repo-facts:block:p1:msg", "repo-facts:block:p1:diffstat", "repo-facts:slot:whole:order", "repo-facts:slot:whole:absent"}
|
||||
&& Count(processesT, "kind", "slot") == 2 && dumpT.at("lensesOff") == Json::array({"repo-better-solution", "repo-functional", "repo-bug-prover", "repo-security"})
|
||||
&& notReviewed("p1:hello.py#1") && notReviewed("p1:absent") && notReviewed("whole:behaviour") && !notReviewed("whole:order") && !notReviewed("p1:msg"),
|
||||
std::format("exit {}, {} call(s), {}", truth.returnCode, calls() - before, dumpT.is_object() ? Text(dumpT.at("overall")) : truth.out + truth.err));
|
||||
}
|
||||
|
||||
// A stop between two gates: the next gate's pool starts aborted and
|
||||
// runs nothing of it.
|
||||
{
|
||||
|
|
|
|||
Loading…
Reference in a new issue