Catcrafts/Crafter.Review
Watch
2
0
Fork
You've already forked Crafter.Review
 0
Code Issues 7 Pull requests Projects Releases 3 Packages Wiki Activity Actions

feat(modes): the kernel modes - a patch series and a mailing-list reply

Browse source 
Beside the push gates, crafter-review runs the pre-send review of a kernel
contributor's workbench: --mode series over a format-patch export with its
cover letter, --mode reply over a mail draft. The inventory partitions a
cover letter, a plain mail and the camera review's shape; ten kernel lenses
join the table with their gates (structure, code, artifact truth) and the
persona gate - one maintainer-persona instance per --reviewers entry, its
whole pass gating its blocks and pruning the units that are not theirs. A
series' code units are keyed on the code like the push gates'; every text
unit on its text, the references it cites and the anchor when it is dated.
The references an artifact cites are prefetched through utilities/lore.py
into the gate-state, with a digest in the run layer, and a persona's
calibration map is its lore listing. The kernel schema carries the persona
vocabulary, the kernel note kinds, the predicted review and the not-mine
list; the matrix the persona holes, the thin and the modelled units; gates
clean without a persona pass is GATES CLEAN. A reply's gate-state sits
beside the draft and is read once; a kernel mode needs --subsystem,
defaults the effort to max and the trees to work/linux,work/milos-linux,
and excuses no unit from the holes.

The kernel lenses and the common text ship verbatim; the table lists them
after the push gates' lenses and the config template holds them off, so a
repository that never turns them on changes nothing. Prompts, keys and plans
were checked byte for byte against the pipeline these modes replace, on its
own artifacts: two replies and a series, every system prompt, user prompt,
job and reuse key identical.
This commit is contained in:
Jorijn van der Graaf 2026-10-31 22:56:12 +01:00
commit 339f4b9b40
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
46 changed files with 5732 additions and 362 deletions
Download patch file Download diff file Expand all files Collapse all files

38
implementations/Crafter.Review-Dispositions.cpp
View file

@ -6,6 +6,7 @@ import :Dispositions;
import :Json;
import :Text;
import :Platform;
import :Plan;
import std;
namespace fs = std::filesystem;
@ -48,6 +49,21 @@ namespace Crafter::Review::Dispositions {
"If the grounds do not survive, return the finding WITHOUT dischargedBy and set contests {id, why} with the evidence - a contest is a claim and will itself be verified. "
"Record that verification inside the finding's evidence field. For a refuted entry you agree with, return a note kind=refutation-verified with its id instead of a finding. "
"An entry with no matching finding of yours: ignore it - nothing is owed for it.";
// The kernel pipeline's rules, where the owner is Gate 5 and the
// evidence is the ledger, the captures, lore and the tree.
constexpr std::string_view KernelRules =
"Rules: if you detect a finding one of these entries names, STILL REPORT IT and set dischargedBy to its id and its unit id in items:[] - detection never changes, only disposition, and Gate 5 must see it. "
"Before setting dischargedBy, VERIFY that entry's grounds against its cited evidence (the ledger, captures/, lore via utilities/lore.py, the tree). "
"If the grounds do not survive, return the finding WITHOUT dischargedBy and set contests {id, why} with the evidence - a contest is a claim and will itself be verified. "
"Record that verification inside the finding's evidence field - do NOT emit disposition-review notes (that kind belongs to the persona lens). "
"For a refuted entry you agree with, return a note kind=refutation-verified with its id instead of a finding. An entry with no matching finding of yours: ignore it - nothing is owed for it.";
constexpr std::string_view PersonaRules =
"Rules: for EVERY entry above except kind gate0, return one note kind=disposition-review with its id, wouldComment (would this reviewer comment on the item as the artifact stands?) and why. "
"If one of your own findings is the item an entry names, report it with dischargedBy set to that id and its unit id in items:[] - it does not block.";
bool IsKernelLens(std::string_view lensFile) {
return Plan::IsLens(lensFile) && Plan::LensOf(lensFile).kernel;
}
}
std::vector<Json> Load(std::string_view path) {
@ -67,7 +83,7 @@ namespace Crafter::Review::Dispositions {
return out;
}
std::vector<Json> Classify(std::span<const Json> entries, std::string_view hash, const fs::path& artifact) {
std::vector<Json> Classify(std::span<const Json> entries, std::string_view hash, const fs::path& artifact, std::string_view mode) {
std::vector<Json> rows;
for (const Json& entry : entries) {
if (Dyn::Equal(Dyn::Get(entry, "hash"), Json(hash))) {
@ -75,7 +91,7 @@ namespace Crafter::Review::Dispositions {
continue;
}
const Json& carry = Dyn::Get(entry, "carry");
if (carry.is_boolean() && carry.get<bool>() && Dyn::Truthy(Dyn::Get(entry, "anchors"))) {
if (mode != "reply" && carry.is_boolean() && carry.get<bool>() && Dyn::Truthy(Dyn::Get(entry, "anchors"))) {
bool ok = true;
std::vector<std::string> details;
for (const Json& anchor : Dyn::At(entry, "anchors")) {
@ -96,7 +112,8 @@ namespace Crafter::Review::Dispositions {
}
rows.push_back({{"id", Dyn::At(entry, "id")}, {"status", ok ? "carried" : "lapsed"}, {"detail", Dyn::Join(details, "; ")}});
} else {
rows.push_back({{"id", Dyn::At(entry, "id")}, {"status", "lapsed"}, {"detail", "hash mismatch; carry disabled or no anchors"}});
rows.push_back({{"id", Dyn::At(entry, "id")}, {"status", "lapsed"},
{"detail", mode == "reply" ? "hash mismatch; no carry in reply mode - re-bind the entry to this hash" : "hash mismatch; carry disabled or no anchors"}});
}
}
return rows;
@ -112,13 +129,18 @@ namespace Crafter::Review::Dispositions {
return out;
}
std::string Block(std::span<const Json> inForce, std::span<const Json> rows, std::string_view lensFile, std::string_view hash) {
std::string Block(std::span<const Json> inForce, std::span<const Json> rows, std::string_view lensFile, std::string_view hash, std::string_view subsystem) {
const bool isPersona = lensFile == Plan::PersonaLens;
const bool kernel = IsKernelLens(lensFile);
const std::string_view ownerGate = kernel ? "Gate 5" : "Gate 4";
std::vector<const Json*> visible;
for (const Json& entry : inForce) {
const Json& lenses = Dyn::Get(entry, "lenses");
if (!Dyn::Equal(Dyn::Get(entry, "kind"), Json("refuted")) || (Dyn::Truthy(lenses) && Dyn::In(Json(lensFile), lenses))) visible.push_back(&entry);
if (isPersona || !Dyn::Equal(Dyn::Get(entry, "kind"), Json("refuted")) || (Dyn::Truthy(lenses) && Dyn::In(Json(lensFile), lenses))) visible.push_back(&entry);
}
const std::string head = std::format("DISPOSITIONS IN FORCE at hash {} - the register is the authority. Nothing outside this list discharges anything.", hash);
const std::string head = std::format("DISPOSITIONS IN FORCE at hash {} - the register is the authority.{}", hash,
subsystem.empty() ? std::string(" Nothing outside this list discharges anything.")
: std::format(" A waiver row in journal/{}/ledger.md for your lens that is NOT listed here is LAPSED for this hash and discharges nothing.", subsystem));
if (visible.empty()) return std::format("{}\n(none)", head);
const std::map<std::string, std::string> status = Statuses(rows);
std::vector<std::string> lines;
@ -145,7 +167,7 @@ namespace Crafter::Review::Dispositions {
if (Dyn::Truthy(Dyn::Get(d, "supersedes"))) line += std::format("\n supersedes: {}", Text(Dyn::At(d, "supersedes")));
line += std::format("\n anchors: {}", anchors);
if (Dyn::Truthy(Dyn::Get(d, "reaffirmed"))) {
line += std::format("\n REAFFIRMED by the owner {}x - final for this binding; a further contest is recorded for Gate 4, it does not block", Len(Dyn::At(d, "reaffirmed")));
line += std::format("\n REAFFIRMED by the owner {}x - final for this binding; a further contest is recorded for {}, it does not block", Len(Dyn::At(d, "reaffirmed")), ownerGate);
}
if (Dyn::Truthy(Dyn::Get(d, "contestRejected"))) {
line += std::format("\n contest REJECTED {}x after re-checking the cited fact - a repeat contest without new evidence is recorded, it does not block",
@ -153,6 +175,6 @@ namespace Crafter::Review::Dispositions {
}
lines.push_back(std::move(line));
}
return std::format("{}\n{}\n{}", head, Dyn::Join(lines, "\n"), Rules);
return std::format("{}\n{}\n{}", head, Dyn::Join(lines, "\n"), isPersona ? PersonaRules : kernel ? KernelRules : Rules);
}
}

25
implementations/Crafter.Review-Driver.cpp
View file

@ -267,8 +267,9 @@ inventory unit, against a frozen commit range.)";
{"--verify", false, "", "push-time check: HEAD, a clean tree and a fresh export still match freeze.json"},
}};
constexpr std::array<std::string_view, 1> Modes = {"commits"};
constexpr std::array<std::string_view, 3> Modes = {"reply", "series", "commits"};
constexpr std::array<std::string_view, 2> Harnesses = {"claude", "opencode"};
constexpr std::array<std::string_view, 2> PersonaBlocks = {"pruned", "all"};
const std::vector<OptionSpec>& RunOptions() {
static const std::string effort = std::format("reasoning effort of every lens process (default {})", Plan::Effort);
@ -282,12 +283,14 @@ inventory unit, against a frozen commit range.)";
static const std::string variant = std::format("opencode harness: the model's reasoning variant (default {})", OpenCode::DefaultVariant);
static const std::vector<OptionSpec> specs = {
{"--mode", true, "MODE", "", Kind::String, Modes, false, true},
{"--artifact", true, "ARTIFACT", "the frozen artifact dir (.review-gates/push/<slug>/)", Kind::String, {}, true},
{"--subsystem", true, "SUBSYSTEM", "journal subsystem: required for reply and series; optional in commits mode (names the ledger and captures the lenses may read)"},
{"--artifact", true, "ARTIFACT", "the frozen artifact: a dir of *.patch (.review-gates/push/<slug>/, a series), or the reply draft file"},
{"--hash", true, "HASH", ""},
{"--reviewers", true, "REVIEWERS", "reply and series: the reviewers a persona reads as, comma-separated (the persona gate runs after the others)"},
{"--dispositions", true, "DISPOSITIONS", ""},
{"--base", true, "BASE", ""},
{"--date", true, "DATE", ""},
{"--trees", true, "TREES", "the checkout the lenses read, at the range HEAD: the repository itself (`.`, default) or the frozen worktree"},
{"--trees", true, "TREES", "the checkout the lenses read, at the range HEAD: the repository itself (`.`, default) or the frozen worktree; reply and series default work/linux,work/milos-linux"},
{"--context-paths", true, "CONTEXT_PATHS", "comma-separated paths of material the artifact discusses (captures, evidence) - mechanical pointers, no narrative"},
{"--only", true, "ONLY", "comma-separated lens files (test only)"},
{"--without", true, "WITHOUT", "comma-separated lens files to leave out of this run by owner ruling (logged; the run stays a full run)"},
@ -302,6 +305,7 @@ inventory unit, against a frozen commit range.)";
{"--reads", true, "READS", reads, Kind::Int},
{"--no-reuse", false, "", ""},
{"--no-salvage", false, "", "do not reuse finished process outputs of an earlier run on this hash"},
{"--persona-blocks", true, "{pruned,all}", "reply and series: pruned (default) skips the block passes of the units a persona's whole pass marked not-mine", Kind::String, PersonaBlocks},
{"--dry-run", false, "", ""},
{"--claude-bin", true, "CLAUDE_BIN", ""},
{"--models", true, "MODELS", models},
@ -332,8 +336,12 @@ inventory unit, against a frozen commit range.)";
Run::Arguments a;
a.config = configPath;
a.mode = parsed.Value("mode").value_or("commits");
a.subsystem = parsed.Value("subsystem");
a.artifact = parsed.Value("artifact").value_or("");
if (a.artifact.empty()) return UsageError(UsageLine(Prog, specs), Prog, "the following arguments are required: --artifact");
a.hash = parsed.Value("hash");
a.reviewers = parsed.Value("reviewers").value_or("");
a.personaBlocks = parsed.Value("persona-blocks").value_or("pruned");
a.dispositions = parsed.Value("dispositions");
a.base = parsed.Value("base").value_or("");
a.date = parsed.Value("date").value_or(Platform::TodayIso());
@ -361,7 +369,13 @@ inventory unit, against a frozen commit range.)";
if (*a.concurrency < 1) return UsageError(UsageLine(Prog, specs), Prog, "argument --concurrency: a number from 1 up (leave it out for no cap)");
}
a.effort = parsed.Value("effort").value_or("");
if (a.effort.empty()) a.effort = std::string(Plan::DefaultEffort());
if (a.effort.empty()) a.effort = std::string(Plan::DefaultEffort(a.mode));
if (a.mode == "commits") {
if (!a.reviewers.empty()) return Die("commits mode has no persona gate (the owner is the last gate): drop --reviewers");
} else {
if (!a.subsystem || a.subsystem->empty()) return Die(std::format("{} mode: --subsystem is required", a.mode));
if (!a.trees || a.trees->empty()) a.trees = "work/linux,work/milos-linux";
}
a.noReuse = parsed.Flag("no-reuse");
a.noSalvage = parsed.Flag("no-salvage");
a.dryRun = parsed.Flag("dry-run");
@ -377,9 +391,10 @@ inventory unit, against a frozen commit range.)";
a.variant = parsed.Value("variant");
if (!a.variant || a.variant->empty()) a.variant = a.harness == "opencode" ? std::optional<std::string>(std::string(OpenCode::DefaultVariant)) : std::nullopt;
a.opencodeBin = parsed.Value("opencode-bin");
a.reads = Plan::ReadsFor(a.mode);
if (const std::optional<std::string> reads = parsed.Value("reads")) {
const std::int64_t n = *Dyn::ParseInt(*reads);
if (n < 1) return Die("--reads: a number from 1 up");
if (n < 1 || (a.mode == "reply" && n != 1)) return Die("--reads: a number from 1 up; a reply has no code units and is read once");
a.reads = n;
}
for (const std::string& part : Dyn::Split(a.models, ",")) {

289
implementations/Crafter.Review-Inventory.cpp
View file

@ -13,15 +13,21 @@ namespace fs = std::filesystem;
namespace Crafter::Review::Inventory {
namespace {
constexpr std::string_view Usage = R"(Partition a frozen commit range (a `git format-patch` export) into review units for the coverage contract.
constexpr std::string_view Usage = R"(Partition a frozen artifact into review units for the coverage contract.
The inventory is MECHANICAL: no agent decides what the units are. Every non-blank line of the artifact belongs to
exactly one unit, and the tool fails if that invariant does not hold. Fixed slots are added so that what is NOT in
the text still has a row in the coverage matrix.
crafter-review inventory commits <dir-with-*.patch> > gate-state/inventory-<hash8>.json
crafter-review inventory reply <draft.txt> > gate-state/inventory-<hash8>.json
crafter-review inventory series <dir-with-*.patch> > gate-state/inventory-<hash8>.json
crafter-review inventory commits <dir-with-*.patch> > gate-state/inventory-<hash8>.json (own-repo push: git format-patch of a range)
Units (ids):
Reply mode units: nl:intro, nl:<file>, anchor, RN, <sec>:N, <sec>:Na, x:N, refs, whole:* - or, for a plain mail
without the dated anchor: hdr, quote:N, para:N, trailers, whole:*. Series mode: cover:header, cover:N and the
units below (every hunk whole), whole:* from the series slots.
Commits mode units (ids):
pK:subject commit K's subject
pK:msg commit K's message body (before the trailers) - only when non-empty
pK:trailers commit K's trailer block - only when non-empty
@ -79,6 +85,74 @@ Units (ids):
return {{"id", std::move(id)}, {"lines", {first, last}}, {"excerpt", std::move(excerpt)}};
}
// ^On .+ wrote:$ on a stripped line: an attribution.
bool IsAttribution(std::string_view stripped) {
return stripped.starts_with("On ") && stripped.ends_with(" wrote:") && stripped.size() > 10 && !stripped.contains('\n');
}
// ^[A-Z][A-Za-z-]+: \S : a trailer line of a mail (Tested-by: ...).
bool IsMailTrailer(std::string_view line) {
if (line.empty() || line[0] < 'A' || line[0] > 'Z') return false;
std::size_t at = 1;
while (at < line.size() && ((line[at] >= 'A' && line[at] <= 'Z') || (line[at] >= 'a' && line[at] <= 'z') || line[at] == '-')) ++at;
if (at < 2 || !line.substr(at).starts_with(": ")) return false;
return at + 2 < line.size() && !Dyn::IsSpace(Dyn::CodePointAt(line, at + 2).first);
}
// ^[\w.-]+/\d{4}-[^\s]+\.patch$ : a bare patch path, the header of a
// per-file group of the camera review.
bool IsPatchPath(std::string_view text) {
std::size_t at = 0;
std::size_t head = 0;
while (at < text.size()) {
const auto [c, used] = Dyn::CodePointAt(text, at);
if (!(Dyn::IsWord(c) || c == '.' || c == '-')) break;
at += used;
++head;
}
if (head == 0 || at >= text.size() || text[at] != '/') return false;
++at;
if (Dyn::DecimalRun(text, at) != 4) return false;
at += 4;
if (at >= text.size() || text[at] != '-') return false;
++at;
const std::size_t rest = Dyn::NonSpaceRun(text, at);
if (rest <= 6 || at + rest != text.size()) return false;
return text.ends_with(".patch");
}
// ^R\d+\. and ^\d+\. : a reversal or a numbered item; the id is the
// text before the dot.
std::optional<std::string> NumberedHead(std::string_view line, bool reversal) {
std::size_t at = 0;
if (reversal) {
if (!line.starts_with('R')) return std::nullopt;
at = 1;
}
const std::size_t digits = Dyn::DecimalRun(line, at);
if (digits == 0 || !line.substr(at + digits).starts_with(". ")) return std::nullopt;
return std::string(line.substr(0, at + digits));
}
// ^\s+[a-z]\. \S : a lettered sub-item; the letter.
std::optional<char> LetteredHead(std::string_view line) {
const std::size_t spaces = Dyn::SpaceRun(line, 0);
if (spaces == 0 || spaces + 3 >= line.size()) return std::nullopt;
const char letter = line[spaces];
if (letter < 'a' || letter > 'z' || !line.substr(spaces + 1).starts_with(". ")) return std::nullopt;
if (Dyn::IsSpace(Dyn::CodePointAt(line, spaces + 3).first)) return std::nullopt;
return letter;
}
// ^\[1\] https?://
bool IsFirstReference(std::string_view line) {
return line.starts_with("[1] http://") || line.starts_with("[1] https://");
}
std::span<const std::string> Range(const std::vector<std::string>& lines, std::size_t from, std::size_t to) {
return std::span<const std::string>(lines).subspan(from, to - from);
}
std::string ReprInts(std::span<const std::int64_t> values) {
std::string out = "[";
for (std::size_t i = 0; i < values.size(); ++i) out += std::format("{}{}", i ? ", " : "", values[i]);
@ -119,7 +193,7 @@ Units (ids):
return "";
}
Series PartitionSeries(const fs::path& dir, bool splitDocs) {
Series PartitionSeries(const fs::path& dir, bool splitDocs, bool kernel) {
Series series;
series.units = Json::array();
Json& units = series.units;
@ -129,7 +203,8 @@ Units (ids):
const std::vector<std::string> lines = Dyn::Split(text, "\n");
const auto count = static_cast<std::int64_t>(lines.size());
const std::optional<std::int64_t> number = Artifact::PatchNumber(name);
const std::string pid = std::format("p{}", number ? *number : -1);
const std::int64_t k = number ? *number : -1;
const std::string pid = k == 0 ? std::string("cover") : std::format("p{}", k);
const std::int64_t base = offset;
std::optional<std::int64_t> subject;
for (std::int64_t i = 0; i < count; ++i) {
@ -155,6 +230,26 @@ Units (ids):
break;
}
}
if (pid == "cover") {
// The header as one unit, the paragraphs of the body.
units.push_back(Unit("cover:header", base + 1, base + body0, subject ? Dyn::Slice(Dyn::Strip(lines[static_cast<std::size_t>(*subject)]), 0, 90) : std::string()));
std::int64_t i = body0;
std::int64_t pn = 0;
while (i < count) {
if (Dyn::Strip(lines[static_cast<std::size_t>(i)]).empty()) {
++i;
continue;
}
std::int64_t j = i;
while (j < count && !Dyn::Strip(lines[static_cast<std::size_t>(j)]).empty()) ++j;
++pn;
units.push_back(Unit(std::format("cover:{}", pn), base + i + 1, base + j, Excerpt(Lines(lines, i, j))));
i = j;
}
series.lines.insert(series.lines.end(), lines.begin(), lines.end());
offset += count;
continue;
}
units.push_back(Unit(std::format("{}:subject", pid), base + 1, base + body0,
subject ? Dyn::Slice(Dyn::Strip(lines[static_cast<std::size_t>(*subject)]), 0, 90) : std::string()));
std::int64_t t = diff0;
@ -216,19 +311,189 @@ Units (ids):
++i;
}
closeHunk(count);
units.push_back(Unit(std::format("{}:outside", pid), 0, 0, "FIXED SLOT: in-repository users/callers of what this commit changes, outside its diff"));
units.push_back(Unit(std::format("{}:absent", pid), 0, 0, "FIXED SLOT: what a reviewer expects in this commit and does not find"));
units.push_back(Unit(std::format("{}:outside", pid), 0, 0,
kernel ? "FIXED SLOT: in-tree users/callers of what this patch changes, outside its diff" : "FIXED SLOT: in-repository users/callers of what this commit changes, outside its diff"));
units.push_back(Unit(std::format("{}:absent", pid), 0, 0,
kernel ? "FIXED SLOT: what a reviewer expects in this patch and does not find" : "FIXED SLOT: what a reviewer expects in this commit and does not find"));
series.lines.insert(series.lines.end(), lines.begin(), lines.end());
offset += count;
}
return series;
}
Series PartitionMail(std::string_view text) {
Series out;
out.lines = Dyn::Split(text, "\n");
out.units = Json::array();
const std::vector<std::string>& lines = out.lines;
const std::size_t n = lines.size();
std::size_t hdrEnd = n;
for (std::size_t i = 0; i < n; ++i) {
if (Dyn::Strip(lines[i]).empty()) {
hdrEnd = i;
break;
}
}
if (hdrEnd > 0) out.units.push_back(Unit("hdr", 1, static_cast<std::int64_t>(hdrEnd), Excerpt(Range(lines, 0, hdrEnd))));
std::size_t i = hdrEnd;
std::int64_t q = 0;
std::int64_t pn = 0;
std::int64_t tr = 0;
while (i < n) {
if (Dyn::Strip(lines[i]).empty()) {
++i;
continue;
}
std::size_t j = i;
std::string uid;
if (lines[i].starts_with('>') || IsAttribution(Dyn::Strip(lines[i]))) {
// A quoted block: the attribution line (if any) and every
// following quoted line; a blank line inside the quote is
// part of it only when a quoted line follows.
while (j < n && (lines[j].starts_with('>') || IsAttribution(Dyn::Strip(lines[j])) || (Dyn::Strip(lines[j]).empty() && j + 1 < n && lines[j + 1].starts_with('>')))) ++j;
++q;
uid = std::format("quote:{}", q);
} else {
while (j < n && !Dyn::Strip(lines[j]).empty() && !lines[j].starts_with('>')) ++j;
if (std::all_of(lines.begin() + static_cast<std::ptrdiff_t>(i), lines.begin() + static_cast<std::ptrdiff_t>(j), [](const std::string& l) { return IsMailTrailer(l); })) {
++tr;
uid = tr == 1 ? std::string("trailers") : std::format("trailers:{}", tr);
} else {
++pn;
uid = std::format("para:{}", pn);
}
}
out.units.push_back(Unit(uid, static_cast<std::int64_t>(i) + 1, static_cast<std::int64_t>(j), Excerpt(Range(lines, i, j))));
i = j;
}
return out;
}
Series PartitionReply(std::string_view text) {
Series out;
out.lines = Dyn::Split(text, "\n");
out.units = Json::array();
const std::vector<std::string>& lines = out.lines;
const std::size_t n = lines.size();
// The block starts at the dated anchor line; a mail without one is a
// plain reply.
std::optional<std::size_t> anchor;
for (std::size_t i = 0; i < n; ++i) {
if (lines[i].starts_with("Dates: every")) {
anchor = i;
break;
}
}
if (!anchor) return PartitionMail(text);
const std::size_t blk = *anchor;
// The author's part: paragraphs grouped by the patch file they
// discuss - a group starts at a paragraph that is a bare path ending
// in .patch; everything before the first such header is nl:intro.
struct Group {
std::string id;
std::size_t start;
std::size_t end;
};
std::vector<Group> groups;
std::size_t i = 0;
while (i < blk) {
if (Dyn::Strip(lines[i]).empty()) {
++i;
continue;
}
std::size_t j = i;
while (j < blk && !Dyn::Strip(lines[j]).empty()) ++j;
const std::string first = Dyn::Strip(lines[i]);
if (IsPatchPath(first) && j - i == 1) {
groups.push_back({std::format("nl:{}", first), i, j});
} else if (!groups.empty()) {
groups.back().end = j;
} else {
groups.push_back({"nl:intro", i, j});
}
i = j;
}
for (const Group& g : groups) out.units.push_back(Unit(g.id, static_cast<std::int64_t>(g.start) + 1, static_cast<std::int64_t>(g.end), Excerpt(Range(lines, g.start, g.end))));
// The anchor paragraph.
std::size_t j = blk;
while (j < n && !Dyn::Strip(lines[j]).empty()) ++j;
out.units.push_back(Unit("anchor", static_cast<std::int64_t>(blk) + 1, static_cast<std::int64_t>(j), Excerpt(Range(lines, blk, j))));
// The rest of the block: reversals, sections, items, sub-items,
// bullets, refs.
std::optional<std::size_t> refsStart;
for (std::size_t r = 0; r < n; ++r) {
if (IsFirstReference(lines[r])) {
refsStart = r;
break;
}
}
if (!refsStart) throw InvariantError("StopIteration");
static const std::map<std::string, std::string> Sections = {{"camss-milos", "camss"}, {"ov13b10-of", "ov13b10"}, {"milos-dts-camss", "dts"}, {"across the three series", "x"}};
std::optional<std::string> sec;
std::optional<std::string> item;
std::int64_t bullet = 0;
std::vector<std::pair<std::size_t, std::string>> starts;
i = j;
while (i < *refsStart) {
const std::string& l = lines[i];
const std::string s = Dyn::Strip(l);
if (s.empty()) {
++i;
continue;
}
const std::string low = Dyn::Lower(s);
const bool indented = l.starts_with(' ');
if (Sections.contains(low) && !indented) {
sec = Sections.at(low);
item.reset();
bullet = 0;
starts.emplace_back(i, std::format("{}:header", *sec));
} else if (const std::optional<std::string> r = NumberedHead(l, true)) {
starts.emplace_back(i, *r);
} else if (l.starts_with("Reversals of the")) {
starts.emplace_back(i, "reversals:header");
} else if (const std::optional<std::string> num = NumberedHead(l, false); num && sec) {
item = *num;
starts.emplace_back(i, std::format("{}:{}", *sec, *item));
} else if (const std::optional<char> letter = LetteredHead(l); letter && sec && item) {
starts.emplace_back(i, std::format("{}:{}{}", *sec, *item, *letter));
} else if (l.starts_with("- ") && sec == "x") {
++bullet;
starts.emplace_back(i, std::format("x:{}", bullet));
} else if (!indented && starts.empty()) {
starts.emplace_back(i, std::format("para:{}", i + 1));
} else if (!indented && !NumberedHead(l, false) && !NumberedHead(l, true) && Dyn::Strip(lines[i - 1]).empty() && !sec) {
starts.emplace_back(i, std::format("para:{}", i + 1));
}
++i;
}
for (std::size_t k = 0; k < starts.size(); ++k) {
const auto& [a, uid] = starts[k];
std::size_t b = k + 1 < starts.size() ? starts[k + 1].first : *refsStart;
while (b > a && Dyn::Strip(lines[b - 1]).empty()) --b;
out.units.push_back(Unit(uid, static_cast<std::int64_t>(a) + 1, static_cast<std::int64_t>(b), Excerpt(Range(lines, a, b))));
}
std::size_t b = n;
while (b > *refsStart && Dyn::Strip(lines[b - 1]).empty()) --b;
out.units.push_back(Unit("refs", static_cast<std::int64_t>(*refsStart) + 1, static_cast<std::int64_t>(b), Excerpt(Range(lines, *refsStart, b))));
return out;
}
Built Build(std::string_view mode, std::string_view artifact) {
if (mode != "commits") throw InvariantError(std::format("inventory: mode {} is not supported here (commits only)", Dyn::Repr(mode)));
Series series = PartitionSeries(fs::path(artifact), true);
if (mode != "reply" && mode != "series" && mode != "commits") throw InvariantError(std::format("inventory: mode {} is not reply, series or commits", Dyn::Repr(mode)));
Series series;
std::string h8;
std::span<const WholeSlot> whole;
if (mode == "reply") {
series = PartitionReply(Dyn::ReadText(fs::path(artifact)));
h8 = Artifact::Sha256File(fs::path(artifact)).substr(0, 8);
whole = WholeReply;
} else {
series = PartitionSeries(fs::path(artifact), mode == "commits", mode == "series");
h8 = Artifact::ArtifactHash("commits", fs::path(artifact)).substr(0, 8);
whole = mode == "commits" ? std::span<const WholeSlot>(WholeCommits) : std::span<const WholeSlot>(WholeSeries);
}
Json& units = series.units;
const std::string h8 = Artifact::ArtifactHash("commits", fs::path(artifact)).substr(0, 8);
std::map<std::int64_t, std::string> owner;
for (const Json& unit : units) {
const std::int64_t a = Dyn::At(unit, "lines")[0].get<std::int64_t>();
@ -251,7 +516,7 @@ Units (ids):
const std::span<const std::int64_t> first(uncovered.data(), std::min<std::size_t>(uncovered.size(), 10));
throw InvariantError(std::format("INVARIANT: {} non-blank lines in no unit, first: {}", uncovered.size(), ReprInts(first)));
}
for (const WholeSlot& slot : WholeCommits) units.push_back(Unit(std::string(slot.id), 0, 0, std::format("FIXED SLOT: {}", slot.description)));
for (const WholeSlot& slot : whole) units.push_back(Unit(std::string(slot.id), 0, 0, std::format("FIXED SLOT: {}", slot.description)));
std::map<std::string, std::int32_t> seen;
for (const Json& unit : units) ++seen[Dyn::At(unit, "id").get<std::string>()];
Json duplicates = Json::array();
@ -274,7 +539,7 @@ Units (ids):
}
std::int32_t Main(std::span<const std::string_view> args) {
if (args.size() != 2 || args[0] != "commits") {
if (args.size() != 2 || (args[0] != "reply" && args[0] != "series" && args[0] != "commits")) {
std::print(std::cerr, "{}", Usage);
return 1;
}

50
implementations/Crafter.Review-Normalize.cpp
View file

@ -6,6 +6,7 @@ import :Normalize;
import :Json;
import :Text;
import :Schema;
import :Plan;
import std;
namespace Crafter::Review::Normalize {
@ -208,13 +209,16 @@ namespace Crafter::Review::Normalize {
for (const Json& n : notes) {
for (const Json& item : Dyn::Or(Dyn::Get(n, "items"), Dyn::EmptyList())) noted.insert(KeyOf(item));
}
const std::string vocabulary = Dyn::Join(std::vector<std::string>(Schema::CoverageVerdicts.begin(), Schema::CoverageVerdicts.end()), "/");
// A persona speaks its own vocabulary; every other lens the generic one.
const bool isPersona = lensFile == Plan::PersonaLens;
const std::span<const std::string_view> allowed = isPersona ? std::span<const std::string_view>(Schema::PersonaVerdicts) : std::span<const std::string_view>(Schema::GenericVerdicts);
const std::string vocabulary = Dyn::Join(std::vector<std::string>(allowed.begin(), allowed.end()), "/");
for (const std::string& id : ids) {
const Json* c = byItem.Find(Json(id));
if (!c || !Dyn::Truthy(*c)) continue;
const Json& verdict = Dyn::At(*c, "verdict");
const std::string key = KeyOf(Json(id));
if (!InVocabulary(verdict, Schema::CoverageVerdicts)) {
if (!InVocabulary(verdict, allowed)) {
logs.push_back(std::format("{}: coverage verdict \"{}\" on {} is not in the vocabulary ({}) - kept as given", lensFile, Dyn::Str(verdict), id, vocabulary));
}
if (verdict == "immaterial" && !noted.contains(key)) {
@ -260,7 +264,7 @@ namespace Crafter::Review::Normalize {
{"kind", "FIX FIRST declared with every finding discharged and no blocking contest (silent veto) - derived pass stands"}});
}
}
return {
Json out = {
{"verdict", derived},
{"declaredVerdict", declared},
{"findings", std::move(findings)},
@ -274,6 +278,13 @@ namespace Crafter::Review::Normalize {
{"normalizeLogs", std::move(logs)},
{"summary", Dyn::Get(v, "summary")},
};
if (lensFile == Plan::PersonaLens) {
// The review this person would send, and the units that are not
// theirs: what the persona's whole pass hands its block passes.
out["predictedReview"] = Dyn::Get(v, "predictedReview");
out["notMine"] = Dyn::Get(v, "notMine");
}
return out;
}
std::vector<Json> Dedupe(std::span<const Json> items) {
@ -312,6 +323,9 @@ namespace Crafter::Review::Normalize {
if (!previous || !Dyn::Truthy(*previous) || rank(Dyn::At(c, "verdict")) >= rank(Dyn::At(*previous, "verdict"))) merged.Set(item, c);
}
}
// A persona's blocking verdict is would-object; every other lens's
// defect.
const std::string blocking = lensFile == Plan::PersonaLens ? "would-object" : "defect";
Json crossLogs = Json::array();
for (const Json& f : findings) {
if (Dyn::Truthy(Dyn::Get(f, "discharged"))) continue;
@ -321,8 +335,8 @@ namespace Crafter::Review::Normalize {
const Json& finding = Dyn::Get(f, "finding");
const std::string cited = Dyn::Truthy(finding) ? Dyn::Slice(finding.get<std::string>(), 0, 80) : std::string();
const std::string was = Dyn::Str(Dyn::At(*entry, "verdict"));
merged.Set(item, {{"item", item}, {"verdict", "defect"}, {"basis", std::format("cited by finding: {}", cited)}});
crossLogs.push_back(std::format("{}: {} raised to defect - cited by an undischarged finding while its own entry was {}", lens, Dyn::Str(item), was));
merged.Set(item, {{"item", item}, {"verdict", blocking}, {"basis", std::format("cited by finding: {}", cited)}});
crossLogs.push_back(std::format("{}: {} raised to {} - cited by an undischarged finding while its own entry was {}", lens, Dyn::Str(item), blocking, was));
}
}
}
@ -426,6 +440,29 @@ namespace Crafter::Review::Normalize {
const Json& row = mx[id];
if (!std::ranges::any_of(row, owned)) holes.push_back(id);
}
// The persona columns: a unit no persona owns is a persona hole, one
// a single persona owns is thin, and one whose every persona entry
// rests on "no record" is modelled, not calibrated.
std::vector<std::string> personaColumns;
for (const Json& column : columns) {
if (column.is_string() && column.get_ref<const std::string&>().starts_with("persona:")) personaColumns.push_back(column.get<std::string>());
}
Json personaHoles = Json::array();
Json thin = Json::array();
Json modelled = Json::array();
for (const Json& unit : inventory) {
const std::string id = Dyn::At(unit, "id").get<std::string>();
std::vector<const Json*> entries;
for (const std::string& column : personaColumns) {
const Json& entry = Dyn::Get(mx[id], column);
if (Dyn::Truthy(entry) && owned(entry)) entries.push_back(&entry);
}
if (personaColumns.empty()) continue;
if (entries.empty()) personaHoles.push_back(id);
if (entries.size() == 1) thin.push_back(id);
const bool noRecord = std::ranges::all_of(entries, [](const Json* e) { return Dyn::Lower(Dyn::Strip(Dyn::Str(Dyn::Get(*e, "basis")))) == "no record"; });
if (!entries.empty() && noRecord) modelled.push_back(id);
}
Json defects = Json::object();
for (const Json& unit : inventory) {
const std::string id = Dyn::At(unit, "id").get<std::string>();
@ -442,6 +479,9 @@ namespace Crafter::Review::Normalize {
{"units", inventory.size()},
{"lenses", std::move(columns)},
{"holes", std::move(holes)},
{"personaHoles", std::move(personaHoles)},
{"thin", std::move(thin)},
{"modelled", std::move(modelled)},
{"notReviewed", std::move(sortedExcused)},
{"defects", std::move(defects)},
{"matrix", std::move(mx)},

223
implementations/Crafter.Review-Plan.cpp
View file

@ -187,6 +187,14 @@ namespace Crafter::Review::Plan {
throw Dyn::Error("KeyError", Dyn::Repr(tier));
}
bool IsKernelMode(std::string_view mode) {
return mode == "reply" || mode == "series";
}
std::int32_t ReadsFor(std::string_view mode) {
return mode == "reply" ? ReplyReads : Reads;
}
bool IsLens(std::string_view file) {
return std::ranges::any_of(Lenses, [&](const Lens& lens) { return lens.file == file; });
}
@ -198,14 +206,25 @@ namespace Crafter::Review::Plan {
throw Dyn::Error("KeyError", Dyn::Repr(file));
}
const std::vector<Gate>& Gates() {
static const std::vector<Gate> gates = {
const std::vector<Gate>& Gates(std::string_view mode) {
static const std::vector<Gate> commits = {
{"Gate 1 — Approach", {"repo-better-solution.md"}},
{"Gate 2 — Functional", {FunctionalLens}},
{"Gate 3 — Proof", {"repo-bug-prover.md", "repo-security.md"}},
{"Gate 4 — Commit truth", {FactsLens}},
};
return gates;
static const std::vector<Gate> reply = {
{"Gate 3 — Artifact truth", {"facts.md", "etiquette.md", "consistency.md", "conciseness.md"}},
};
static const std::vector<Gate> series = {
{"Gate 1 — Structure", {"better-solution.md"}},
{"Gate 2 — Code", {"code-quality.md", "minimalism.md", "bug-prover.md"}},
{"Gate 3 — Artifact truth", {"facts.md", "consistency.md", "process-docs.md", "etiquette.md", "conciseness.md"}},
};
if (mode == "reply") return reply;
if (mode == "series") return series;
if (mode == "commits") return commits;
throw Dyn::Error("KeyError", Dyn::Repr(mode));
}
const Environment* FindEnvironment(std::string_view name) {
@ -227,7 +246,10 @@ namespace Crafter::Review::Plan {
if (kind == "slot") return {50, 35};
if (unitKind == "code") return lensFile == FunctionalLens ? Budget{90, 60} : Budget{60, 40};
if (unitKind == "doc") return {60, 40};
if (unitKind == "text") return lensFile == FactsLens ? Budget{60, 40} : Budget{50, 30};
// The facts lens of the kernel modes gets the most (2026-09-28: a
// facts pass on the densest unit used 58 of 60 calls and left one
// claim unverified).
if (unitKind == "text") return lensFile == "facts.md" ? Budget{80, 50} : lensFile == FactsLens ? Budget{60, 40} : Budget{50, 30};
throw Dyn::Error("KeyError", Dyn::Repr(std::format("block:{}", unitKind)));
}
@ -235,6 +257,16 @@ namespace Crafter::Review::Plan {
return Effort;
}
std::string_view DefaultEffort(std::string_view mode) {
return IsKernelMode(mode) ? KernelEffort : Effort;
}
bool IsBareHeader(const Json& unit, std::string_view text) {
constexpr std::array<std::string_view, 4> Bare = {"camss-milos", "ov13b10-of", "milos-dts-camss", "across the three series"};
const std::string& id = Dyn::At(unit, "id").get_ref<const std::string&>();
return id.ends_with(":header") && std::ranges::contains(Bare, std::string_view(Dyn::Lower(Dyn::Strip(text))));
}
std::string_view UnitKindOf(std::string_view unitId) {
if (unitId.starts_with("whole:")) return "slot";
const std::optional<std::size_t> prefix = PkPrefix(unitId);
@ -265,11 +297,12 @@ namespace Crafter::Review::Plan {
return path && IsDocPath(*path);
}
// Every lens of this pipeline is a repo lens: the units of a documentation
// file are "doc" for each of them - the commit-truth lens reads them, the
// code lenses do not.
std::string_view KindFor(std::string_view, const Json& unit) {
// For the repo lenses the units of a documentation file are "doc" - the
// commit-truth lens reads them, the code lenses do not; for the kernel
// lenses every unit of a diff stays code.
std::string_view KindFor(std::string_view lensFile, const Json& unit) {
const std::string_view kind = UnitKind(unit);
if (IsLens(lensFile) && LensOf(lensFile).kernel) return kind;
return kind == "code" && IsDocUnit(Dyn::At(unit, "id").get_ref<const std::string&>()) ? "doc" : kind;
}
@ -319,21 +352,27 @@ namespace Crafter::Review::Plan {
return Sha256Hex(text);
}
std::string LensName(std::string_view lensFile) {
std::string LensName(std::string_view lensFile, std::string_view persona) {
if (!persona.empty()) return std::format("persona:{}", persona);
return Dyn::Replace(lensFile, ".md", "");
}
std::vector<std::string> ModeLenses() {
std::vector<std::string> ModeLenses(std::string_view mode) {
std::vector<std::string> out;
for (const Gate& gate : Gates()) {
for (const Gate& gate : Gates(mode)) {
for (const std::string_view lens : gate.lenses) out.emplace_back(lens);
}
if (IsKernelMode(mode)) out.emplace_back(PersonaLens);
return out;
}
std::vector<Gate> GatesFor(std::span<const std::string> only, std::span<const std::string> without, std::span<const std::string> off) {
return GatesFor("commits", only, without, off);
}
std::vector<Gate> GatesFor(std::string_view mode, std::span<const std::string> only, std::span<const std::string> without, std::span<const std::string> off) {
std::vector<Gate> out;
for (const Gate& gate : Gates()) {
for (const Gate& gate : Gates(mode)) {
Gate kept{gate.title, {}};
for (const std::string_view lens : gate.lenses) {
if (!only.empty() && !Contains(only, lens)) continue;
@ -346,18 +385,30 @@ namespace Crafter::Review::Plan {
}
LensPlan PlanLens(std::string_view lensFile, const Json& units, std::span<const std::string> lines, std::span<const std::string> unitsFilter,
std::span<const std::string> exclude) {
std::span<const std::string> exclude, std::string_view persona, std::span<const std::string> excludeUnits) {
const Lens& lens = LensOf(lensFile);
const std::string name = LensName(lensFile);
const std::string name = LensName(lensFile, persona);
// A persona's jobs carry its name, after the lens file.
auto stamp = [&](Json job) {
if (persona.empty()) return job;
Json out = Json::object();
for (auto it = job.begin(); it != job.end(); ++it) {
out[it.key()] = it.value();
if (it.key() == "lensFile") out["persona"] = persona;
}
return out;
};
LensPlan plan;
if (!lens.whole.empty()) {
const Budget budget = BudgetFor("whole", "", lensFile);
Json slots = Json::array();
for (const Json& unit : units) {
const std::string& id = Dyn::At(unit, "id").get_ref<const std::string&>();
if (UnitKind(unit) == "slot" && std::ranges::contains(lens.slots, std::string_view(id))) slots.push_back(id);
// A kernel lens's whole pass covers every whole:* slot of the
// artifact; a repo lens's the slots the table names.
if (UnitKind(unit) == "slot" && (lens.kernel || std::ranges::contains(lens.slots, std::string_view(id)))) slots.push_back(id);
}
plan.whole = {
plan.whole = stamp({
{"label", std::format("{}:whole", name)},
{"lens", name},
{"lensFile", lensFile},
@ -367,21 +418,23 @@ namespace Crafter::Review::Plan {
{"model", ModelFor(lens.model)},
{"maxTurns", budget.maxTurns},
{"timeoutMin", budget.timeoutMin},
};
});
}
for (const Json& unit : units) {
const std::string_view kind = KindFor(lensFile, unit);
if (!std::ranges::contains(lens.block, kind)) continue;
const std::string& id = Dyn::At(unit, "id").get_ref<const std::string&>();
if (Contains(excludeUnits, id)) continue;
if (!unitsFilter.empty() && !Contains(unitsFilter, id)) continue;
const std::string text = UnitText(lines, unit);
const std::optional<std::string> path = UnitPath(id);
if (kind == "slot") {
// A lens with no whole pass asks each of its own slots in a
// pass of its own; another lens's whole slot is not its.
if (!std::ranges::contains(lens.slots, std::string_view(id))) continue;
// pass of its own; another lens's whole slot is not its. A
// kernel lens asks every whole:* slot.
if (!lens.kernel && !std::ranges::contains(lens.slots, std::string_view(id))) continue;
const Budget budget = BudgetFor("slot", "slot", lensFile);
plan.blocks.push_back({
plan.blocks.push_back(stamp({
{"label", std::format("{}:slot:{}", name, id)},
{"lens", name},
{"lensFile", lensFile},
@ -393,7 +446,19 @@ namespace Crafter::Review::Plan {
{"model", ModelFor(lens.model)},
{"maxTurns", budget.maxTurns},
{"timeoutMin", budget.timeoutMin},
});
}));
continue;
}
if (kind == "text" && IsBareHeader(unit, text)) {
plan.blocks.push_back(stamp({
{"label", std::format("{}:na:{}", name, id)},
{"lens", name},
{"lensFile", lensFile},
{"kind", "auto-na"},
{"unit", id},
{"expectedIds", {id}},
{"unitSha", Sha(text)},
}));
continue;
}
if (kind == "doc" && id.ends_with("#hdr")) {
@ -426,7 +491,7 @@ namespace Crafter::Review::Plan {
}
}
const Budget budget = BudgetFor("block", kind, lensFile);
plan.blocks.push_back({
plan.blocks.push_back(stamp({
{"label", std::format("{}:block:{}", name, id)},
{"lens", name},
{"lensFile", lensFile},
@ -438,7 +503,7 @@ namespace Crafter::Review::Plan {
{"model", ModelFor(lens.model)},
{"maxTurns", budget.maxTurns},
{"timeoutMin", budget.timeoutMin},
});
}));
}
return plan;
}
@ -450,7 +515,7 @@ namespace Crafter::Review::Plan {
std::ranges::sort(deps);
std::ranges::sort(inForceIds);
const Json& model = Dyn::Get(job, "model");
const std::vector<std::string> parts = {
std::vector<std::string> parts = {
std::string(lensFile),
ident.empty() ? Dyn::At(job, "unit").get<std::string>() : std::string(ident),
Sha(unitText),
@ -461,6 +526,7 @@ namespace Crafter::Review::Plan {
Dyn::Has(job, "model") ? model.get<std::string>() : std::string(),
std::string(ReuseKeyEffortSlot),
};
if (Dyn::Truthy(Dyn::Get(job, "persona"))) parts.push_back(std::format("persona={}", Dyn::Str(job.at("persona"))));
return Sha(Dyn::Join(parts, "|")).substr(0, 16);
}
@ -481,7 +547,7 @@ namespace Crafter::Review::Plan {
return Dyn::Join(lines, "\n");
}
PatchParts PartsOf(std::string_view patchText) {
PatchParts PartsOf(std::string_view patchText, bool docFiles) {
const std::vector<std::string> lines = Dyn::Split(patchText, "\n");
std::optional<std::size_t> subject;
for (std::size_t i = 0; i < lines.size(); ++i) {
@ -524,7 +590,7 @@ namespace Crafter::Review::Plan {
for (const std::string& section : SplitBeforeLines(CodeNorm(Dyn::Join(code, "\n")), "diff --git ")) {
const std::string firstLine = Dyn::Split(section, "\n", 1)[0];
const std::string path = Dyn::Strip(Dyn::Split(firstLine, " b/").back());
if (section.starts_with("diff --git ") && IsDocPath(path)) {
if (docFiles && section.starts_with("diff --git ") && IsDocPath(path)) {
// re.split(r'(?m)^@@', section, maxsplit=1)[0].rstrip('\n')
std::size_t header = section.size();
for (std::size_t at = 0; at < section.size();) {
@ -556,12 +622,13 @@ namespace Crafter::Review::Plan {
return parts;
}
CommitCode CommitCodeOf(const std::map<std::int64_t, std::string>& patches) {
CommitCode CommitCodeOf(const std::map<std::int64_t, std::string>& patches, bool kernel) {
CommitCode cc;
cc.kernel = kernel;
std::set<std::string> bases;
std::vector<std::string> codes;
for (const auto& [k, text] : patches) {
PatchParts parts = PartsOf(text);
PatchParts parts = PartsOf(text, !kernel);
cc.body[k] = parts.hasBody;
cc.code[k] = Sha(parts.code);
cc.docs[k] = std::move(parts.docs);
@ -590,7 +657,10 @@ namespace Crafter::Review::Plan {
deps.push_back(std::format("commit={}:{}", *k, code->second));
deps.push_back(std::format("range={}", cc.range));
deps.push_back(std::format("docs={}", Dyn::Join(cc.docs.at(*k), "|")));
if (rest == "absent") deps.push_back(std::format("body={}", cc.body.at(*k) ? "yes" : "no"));
// A kernel reviewer looks for a trailer, a Cc, a test statement:
// the patch's message is what `absent` depends on there; in an
// own repository it looks for a body at all.
if (rest == "absent") deps.push_back(cc.kernel ? std::format("msg={}", cc.message.at(*k)) : std::format("body={}", cc.body.at(*k) ? "yes" : "no"));
return {std::format("code:{}", rest), deps};
}
const std::size_t hash = rest.rfind('#');
@ -703,6 +773,96 @@ namespace Crafter::Review::Plan {
return Sha(Dyn::Join(parts, "|")).substr(0, 16);
}
namespace {
// \b before `at` and after `end` in the decoded text.
bool Bounded(const std::u32string& text, std::size_t at, std::size_t end) {
const bool before = at == 0 || !Dyn::IsWord(text[at - 1]);
const bool after = end >= text.size() || !Dyn::IsWord(text[end]);
return before && after;
}
// A phrase at `at`, letters compared without case.
bool PhraseAt(const std::u32string& text, std::size_t at, std::string_view phrase) {
if (at + phrase.size() > text.size()) return false;
for (std::size_t i = 0; i < phrase.size(); ++i) {
const char32_t c = text[at + i];
const char p = phrase[i];
if (p >= 'a' && p <= 'z') {
if (Dyn::IgnoreCaseLetter(c) != p) return false;
} else if (c != static_cast<char32_t>(p)) {
return false;
}
}
return true;
}
// re.search(r'\d{4}-\d{2}-\d{2}|\bv\d+\b|\bas of\b|...', text, re.I)
bool Dated(std::string_view unitText) {
constexpr std::array<std::string_view, 9> Words = {"as of", "still", "unmerged", "in review", "no reply", "latest", "merged", "posted", "applied"};
const std::u32string text = Dyn::Decode(unitText);
auto digits = [&](std::size_t at, std::size_t n) {
for (std::size_t i = 0; i < n; ++i) {
if (at + i >= text.size() || !Dyn::IsDecimal(text[at + i])) return false;
}
return true;
};
for (std::size_t at = 0; at < text.size(); ++at) {
if (digits(at, 4) && at + 4 < text.size() && text[at + 4] == '-' && digits(at + 5, 2) && at + 7 < text.size() && text[at + 7] == '-' && digits(at + 8, 2)) return true;
if (Dyn::IgnoreCaseLetter(text[at]) == 'v' && at + 1 < text.size() && Dyn::IsDecimal(text[at + 1])) {
std::size_t end = at + 1;
while (end < text.size() && Dyn::IsDecimal(text[end])) ++end;
if (Bounded(text, at, end)) return true;
}
for (const std::string_view word : Words) {
if (PhraseAt(text, at, word) && Bounded(text, at, at + word.size())) return true;
}
}
return false;
}
// re.fullmatch(r'R\d+', id)
bool IsReversalId(std::string_view id) {
return id.size() > 1 && id[0] == 'R' && Dyn::DecimalRun(id, 1) == id.size() - 1;
}
// re.search(r'\b' + re.escape(id) + r'\b', text)
bool NamesUnit(std::string_view unitText, std::string_view id) {
const std::u32string text = Dyn::Decode(unitText);
const std::u32string needle = Dyn::Decode(id);
for (std::size_t at = 0; at + needle.size() <= text.size(); ++at) {
if (std::u32string_view(text).substr(at, needle.size()) == needle && Bounded(text, at, at + needle.size())) return true;
}
return false;
}
}
std::vector<std::string> TextDeps(std::string_view unitText, const Json& units, std::span<const std::string> lines, std::span<const std::pair<std::string, std::string>> refs,
std::string_view anchorText) {
std::vector<std::string> deps;
// sorted(set(int(x) for x in re.findall(r'\[(\d+)\]', text)))
std::set<std::int64_t> cited;
for (std::size_t at = 0; at < unitText.size(); ++at) {
if (unitText[at] != '[') continue;
const std::size_t digits = Dyn::DecimalRun(unitText, at + 1);
if (digits == 0 || at + 1 + digits >= unitText.size() || unitText[at + 1 + digits] != ']') continue;
if (const std::optional<std::int64_t> n = Dyn::ParseInt(unitText.substr(at + 1, digits))) cited.insert(*n);
at += digits + 1;
}
for (const std::int64_t n : cited) {
std::string url = "?";
for (const auto& [key, value] : refs) {
if (key == std::to_string(n)) url = value;
}
deps.push_back(std::format("ref[{}]={}", n, url));
}
if (Dated(unitText)) deps.push_back(std::format("anchor={}", Sha(anchorText)));
for (const Json& unit : units) {
const std::string& id = Dyn::At(unit, "id").get_ref<const std::string&>();
if (IsReversalId(id) && NamesUnit(unitText, id)) deps.push_back(std::format("unit[{}]={}", id, Sha(UnitText(lines, unit))));
}
return deps;
}
std::int64_t ReadsOf(const Json& entry) {
if (!Dyn::Truthy(entry)) return 0;
const Json& reads = Dyn::Get(entry, "reads");
@ -750,8 +910,9 @@ namespace Crafter::Review::Plan {
const double lensTokens = static_cast<double>(layer == lensLayerBytes.end() ? 3000 : layer->second) / tpb;
const double tail = static_cast<double>(tailBytes) / tpb;
const Json& unitKind = Dyn::Get(job, "unitKind");
const std::int64_t turns = kind == "whole" ? 40 : kind == "slot" ? 10 : Dyn::At(job, "lensFile") == FunctionalLens ? 25
: unitKind == "code" ? 10 : (Dyn::At(job, "lensFile") == FactsLens ? 12 : 8);
const Json& lensFile = Dyn::At(job, "lensFile");
const std::int64_t turns = kind == "whole" ? 40 : kind == "slot" ? 10 : lensFile == FunctionalLens ? 25
: unitKind == "code" ? 10 : (lensFile == FactsLens || lensFile == "facts.md" ? 12 : 8);
if (seen.insert({lens, kind}).second) cacheWrite += prefix + lensTokens;
cacheWrite += tail + static_cast<double>(turns * 3000);
std::vector<double> reads;

10
implementations/Crafter.Review-Pool.cpp
View file

@ -161,9 +161,17 @@ namespace Crafter::Review::Pool {
return std::format("{}://{}:{}{}", p.scheme, host, portText, p.path);
}
std::vector<std::string> AllowedTools(std::string_view, std::string_view, std::span<const std::string> trees, std::span<const std::string> extra,
std::vector<std::string> AllowedTools(std::string_view lensFile, std::string_view, std::span<const std::string> trees, std::span<const std::string> extra,
const std::optional<std::string>& loginFixture, std::string_view environment) {
std::vector<std::string> allow = Tools::BaseAllow();
// The kernel lenses read lore through the helper: every one fetches;
// the lenses that judge against the list search too (2026-09-17: a
// facts block without search reported a posted series as
// unfindable).
const bool kernel = Plan::IsLens(lensFile) && Plan::LensOf(lensFile).kernel;
if (kernel) allow.emplace_back("Bash(python3 utilities/lore.py fetch:*)");
constexpr std::array<std::string_view, 6> LoreSearch = {"facts.md", "consistency.md", "etiquette.md", "process-docs.md", "better-solution.md", Plan::PersonaLens};
if (std::ranges::contains(LoreSearch, lensFile)) allow.emplace_back("Bash(python3 utilities/lore.py search:*)");
const std::string cwd = fs::current_path().string();
for (const std::string& t : trees) {
// Both spellings: lenses write absolute -C paths too (sabotage

152
implementations/Crafter.Review-Prompt.cpp
View file

@ -16,6 +16,33 @@ namespace Crafter::Review::Prompt {
"defect (a finding with items:[id] exists), immaterial (a note kind=immaterial with items:[id] exists), n/a (the unit carries nothing your charter judges - the basis says why). "
"For the functional lens \"clear\" means the behaviour of the unit was exercised in the running application and observed as the code intends, never read as plausible; a unit with no reachable behaviour is n/a with the reason, and a behaviour you could not reach for a reason that is not by design is an `unverified` note with n/a, never clear.";
const std::string_view GenericVocabKernel =
"Coverage vocabulary for this lens: clear (you checked this unit by your charter and found nothing - the basis says what you checked it against: the source fetched, the tree ref, the units you crossed it with), "
"defect (a finding with items:[id] exists), immaterial (a note kind=immaterial with items:[id] exists), n/a (the unit carries nothing your charter judges - the basis says why; a paragraph of claims is never n/a). "
"For the facts lens \"clear\" means every claim in the unit was checked at its source, not read as plausible; for the consistency lens the basis names the unit ids crossed; for the conciseness lens the basis names the class that keeps each sentence.";
const std::string_view PersonaVocab =
"Coverage vocabulary for a maintainer or reviewer: would-require (you would ask for this yourself), no-comment, would-object (you would push back - a finding with items:[id] must exist), incomplete (the unit is right but misses what a fix must contain - finding required), not-mine (not your tree, file or decision). "
"For the recipient persona: can-act, ambiguous (finding required), would-push-back (finding required), already-known, not-mine. basis = the message-id of YOUR OWN mail, or a tree fact (file:line), that grounds the verdict - or the literal string \"no record\"; never a paraphrase of reputation. "
"A no-comment on a unit that changes your file needs a message where you accepted the same construct; otherwise the basis is \"no record\".";
const std::string_view WholeFirst =
"SCOPE - WHOLE PASS, and it gates the block passes of this lens: judge the artifact as an APPROACH - is this the right solution at all, architecturally and at the subsystem level (the alternatives, the maintainers' stated direction, what upstream already has)? "
"Do not review individual hunks or sentences; per-block passes of this lens run only if you pass. Return `summary` (5-10 sentences: the approach as you judged it and the constraints the block passes must respect). Cover exactly the units listed below (the whole:* slots).";
const std::string_view WholeAll =
"SCOPE - WHOLE PASS of the consistency lens, and it gates the block passes: the artifact as a whole - the route it recommends and whether every dependent item carries it through (whole:route), the total set of asks and whether they are legible and free of contradictions (whole:asks), what a reader would expect and does not find (whole:absent), the ledger collision check of your charter, and contradictions between the parts of the artifact. "
"Per-unit consistency is checked by parallel block passes of this lens that run only if you pass - do not duplicate them. Return `summary` (5-10 sentences the block passes rest on). Cover exactly the units listed below (the whole:* slots).";
const std::string_view WholePersona =
"SCOPE - WHOLE PASS as this person, and it gates the block passes: read the whole artifact as they would and write the review they would actually send (predictedReview), the disposition-review notes your charter requires, and cover exactly the units listed below (the whole:* slots). "
"Return `summary` as a calibration digest for the block passes (this reviewer's pet issues, phrasing and imperatives, found in their own mail - name the message-ids). Return in `notMine` every unit id that is not this person's tree, file or decision - those units get no block pass from this persona; leave it empty for a recipient persona.";
const std::string_view HintAnchor =
"This unit fixes the as-of date for every state statement in the artifact. Sweep lore for movement since that date in the threads the artifact cites (the prefetched references give the message-ids; use utilities/lore.py search for newer versions and replies) and report every statement the movement falsifies as a finding; a statement still true is part of your basis.";
const std::string_view HintRefs =
"Check that every [n] line resolves (the prefetched index), that the numbering is contiguous and matches the body, and that no entry is unused or orphaned; whether each citation is apt is judged by the unit that cites it.";
const std::string_view HintOutsideKernel = "Judge the in-tree users and callers of what this patch changes, at the submission base, outside its own diff.";
const std::string_view HintAbsentKernel = "Judge what a reviewer expects in this patch and does not find: a trailer, a test statement, a Cc, a binding property, an error path.";
const std::string_view TextBlockHint = "Within this unit, verify its own dated statements against the anchor date and its own [n] citations against the prefetched raw files.";
const std::string_view WholeFirstRepo =
"SCOPE - WHOLE PASS, and it gates the block passes of this lens: judge the range as an APPROACH for THIS repository - is this the right solution at all (an existing mechanism in the repository or its platform, the right layer or component, the right commit shape, compatibility with what users already have installed)? "
"Do not review individual hunks; per-block passes of this lens run only if you pass. Return `summary` (5-10 sentences: the approach as you judged it and the constraints the block passes must respect). Cover exactly the units listed below (the whole:* slots).";
@ -45,6 +72,19 @@ namespace Crafter::Review::Prompt {
"Context lines are context and removed lines are not claims, except a context sentence this commit's own change makes false.";
std::string_view SlotDuty(std::string_view lensFile, std::string_view unitId) {
if (lensFile == "etiquette.md" && unitId == "whole:asks") {
return "; for this lens that includes Step 0 and the point map: every point the mail being answered raised - is each answered somewhere, none silently skipped? List them with where each is answered";
}
if (lensFile == "conciseness.md" && unitId == "whole:absent") {
return "; for this lens that includes the growth calibration against the first reviewed version and cross-unit duplication (the same point made in two units)";
}
if (lensFile == "process-docs.md" && unitId == "whole:asks") return "; for this lens that includes recipients (get_maintainer), subject and cover mechanics";
if (lensFile == "minimalism.md" && unitId == "whole:absent") {
return "; for this lens that includes series-level minimalism: patches or hunks that are unnecessary as a whole, split or merged wrongly, or belong in another series";
}
if (lensFile == "facts.md" && unitId == "whole:route") {
return "; for this lens that means the claims that span units - does every item that depends on the recommended route state the same facts about it";
}
if (lensFile != Plan::FactsLens) return {};
if (unitId == "whole:order") {
return "; for this lens that means each message describes its own commit alone - nothing a message claims is delivered by a later commit of the range, and no message describes the state a later commit creates";
@ -82,16 +122,56 @@ namespace Crafter::Review::Prompt {
}
}
std::string SystemText() {
std::string SystemText(std::string_view mode) {
if (mode == "reply" || mode == "series") return Dyn::Replace(Dyn::ReadText(LensDir() / "_common.md"), "{{ARTIFACT_KIND}}", mode == "reply" ? "mailing-list reply" : "patch series");
return Dyn::Replace(Dyn::ReadText(LensDir() / "_common-repo.md"), "{{ARTIFACT_KIND}}", "commit range");
}
namespace {
std::string ArtifactLines(std::span<const std::string> lines) {
std::vector<std::string> artifact;
artifact.reserve(lines.size());
for (std::size_t i = 0; i < lines.size(); ++i) artifact.push_back(std::format("{:04d}| {}", i + 1, lines[i]));
return Dyn::Join(artifact, "\n");
}
std::string InventoryLines(const Json& units) {
std::vector<std::string> inventory;
for (const Json& unit : units) {
const Json& range = Dyn::At(unit, "lines");
inventory.push_back(std::format("- {} [{}-{}] {}", Dyn::Str(Dyn::At(unit, "id")), Dyn::Str(range.at(0)), Dyn::Str(range.at(1)), Dyn::Str(Dyn::At(unit, "excerpt"))));
}
return Dyn::Join(inventory, "\n");
}
}
std::string RunLayerKernel(std::string_view mode, std::string_view artifactRel, std::string_view subsystem, std::string_view base, std::span<const std::string> reviewers, std::string_view date,
std::span<const std::string> trees, std::string_view hash, std::span<const std::string> lines, const Json& units, std::string_view refDigest,
std::span<const std::string> contextPaths) {
std::vector<std::string> inputs;
inputs.push_back(std::format("- The frozen artifact (review THIS, at hash {}): {}", hash,
mode == "reply" ? std::format("the reply draft file {} - inlined below; nothing else is an input", artifactRel)
: std::format("the *.patch files in {} - inlined below in sorted order; nothing else in that dir is an input", artifactRel)));
inputs.push_back(std::format("- Ledger: journal/{}/ledger.md (the only mutable journal file you may read)", subsystem));
inputs.push_back(std::format("- Thread mboxes: journal/{}/captures/threads/", subsystem));
inputs.push_back(std::format("- Evidence/proof artifacts: journal/{}/captures/ (index in the ledger)", subsystem));
inputs.push_back(base.empty() ? std::string("- Target tree at the submission base: see the artifact") : std::format("- Target tree at the submission base: {}", base));
if (!trees.empty()) inputs.push_back(std::format("- Trees: {}", Dyn::Join(trees, ", ")));
if (!contextPaths.empty()) inputs.push_back(std::format("- Material the artifact discusses (read-only evidence; do not spend turns locating it): {}", Dyn::Join(contextPaths, ", ")));
inputs.emplace_back("- History archaeology ONLY in work/milos-linux (work/linux is shallow; its history queries lie)");
inputs.emplace_back("- lore.kernel.org ONLY via utilities/lore.py (an Anubis PoW wall blocks curl/WebFetch)");
if (!reviewers.empty()) inputs.push_back(std::format("- Reviewers with a persona in this run: {}", Dyn::Join(reviewers, ", ")));
inputs.push_back(std::format("- Run date: {}", date));
return std::format("INPUTS:\n{}\n\nARTIFACT (frozen, hash {}), inlined with line numbers:\n{}\n\nINVENTORY - {} units, a MECHANICAL partition of the artifact (utilities/inventory.py: every non-blank line belongs to exactly one unit; fixed slots with lines 0-0 stand for what is NOT in the text):\n{}\n\n{}",
Dyn::Join(inputs, "\n"), hash, ArtifactLines(lines), units.size(), InventoryLines(units), refDigest);
}
std::string Charter(std::string_view lensFile) {
return Dyn::ReadText(LensDir() / std::string(lensFile));
}
std::string RunLayer(std::string_view artifactRel, std::string_view base, std::string_view date, std::span<const std::string> trees, std::string_view hash,
std::span<const std::string> lines, const Json& units, std::span<const std::string> contextPaths, std::string_view repoName) {
std::span<const std::string> lines, const Json& units, std::span<const std::string> contextPaths, std::string_view repoName, std::string_view subsystem) {
std::vector<std::string> inputs;
inputs.push_back(std::format("- The frozen artifact (review THIS, at hash {}): the *.patch files in {} - `git format-patch` of the commit range under review, inlined below in sorted order; nothing else in that dir is an input",
hash, artifactRel));
@ -99,6 +179,10 @@ namespace Crafter::Review::Prompt {
repoName, Dyn::Join(trees, ", "), !trees.empty() && trees.front() == "." ? " (`.` is the working directory)" : ""));
inputs.push_back(base.empty() ? std::string("- Range base: the base-commit line of the last patch")
: std::format("- Range base (the state before the first commit; `git -C <tree> show <base>:<path>`): {}", base));
if (!subsystem.empty()) {
inputs.push_back(std::format("- Ledger: journal/{}/ledger.md (the only mutable journal file you may read)", subsystem));
inputs.push_back(std::format("- Evidence/proof artifacts: journal/{}/captures/ (index in the ledger)", subsystem));
}
if (!contextPaths.empty()) {
inputs.push_back(std::format("- Material the artifact discusses (read-only evidence; do not spend turns locating it): {}", Dyn::Join(contextPaths, ", ")));
}
@ -185,8 +269,13 @@ namespace Crafter::Review::Prompt {
}
std::string LensLayer(std::string_view lensFile, std::string_view charterText, std::string_view dispositionsBlock, std::string_view app,
std::string_view wholeSummary, std::string_view db) {
std::vector<std::string> parts = {std::format("YOUR CHARTER (process/lenses/{}), verbatim:\n\n{}", lensFile, Dyn::Strip(charterText))};
std::string_view wholeSummary, std::string_view db, std::string_view persona, std::string_view corpus) {
std::vector<std::string> parts;
if (!persona.empty()) {
parts.push_back(std::format("Your persona: {}. You ARE this person, reviewing what they will actually read. Calibrate ONLY on mail they actually wrote (their replies to us and to recent similar series - fetch via utilities/lore.py).", persona));
if (!corpus.empty()) parts.push_back(std::format("CALIBRATION MAP - a lore search listing of this person's recent mail (fetch the bodies you need):\n{}", Dyn::Slice(corpus, 0, 6000)));
}
parts.push_back(std::format("YOUR CHARTER (process/lenses/{}), verbatim:\n\n{}", lensFile, Dyn::Strip(charterText)));
if (!app.empty()) parts.emplace_back(app);
if (!db.empty()) parts.emplace_back(db);
parts.emplace_back(dispositionsBlock);
@ -199,15 +288,30 @@ namespace Crafter::Review::Prompt {
std::string Tail(const Json& job, const Json& units, std::span<const std::string> lines, std::string_view wholeSummary) {
const std::string lensFile = Dyn::At(job, "lensFile").get<std::string>();
const Json& kind = Dyn::At(job, "kind");
const Plan::Lens& lens = Plan::LensOf(lensFile);
const std::string_view vocab = lensFile == Plan::PersonaLens ? PersonaVocab : lens.kernel ? GenericVocabKernel : GenericVocab;
if (kind == "whole") {
const std::string_view whole = Plan::LensOf(lensFile).whole;
if (whole != "first-repo" && whole != "functional") throw Dyn::Error("KeyError", Dyn::ReprValue(whole.empty() ? Json() : Json(whole)));
const std::string_view whole = lens.whole;
std::string_view text;
if (whole == "first-repo") {
text = WholeFirstRepo;
} else if (whole == "functional") {
text = WholeFunctional;
} else if (whole == "first") {
text = WholeFirst;
} else if (whole == "all") {
text = WholeAll;
} else if (whole == "persona") {
text = WholePersona;
} else {
throw Dyn::Error("KeyError", Dyn::ReprValue(whole.empty() ? Json() : Json(whole)));
}
std::vector<std::string> listed;
const Json& expected = Dyn::At(job, "expectedIds");
for (const Json& unit : units) {
if (Dyn::In(Dyn::At(unit, "id"), expected)) listed.push_back(std::format("- {} [0-0] {}", Dyn::Str(Dyn::At(unit, "id")), Dyn::Str(Dyn::At(unit, "excerpt"))));
}
return std::format("{}\n{}\nUnits to cover:\n{}", whole == "first-repo" ? WholeFirstRepo : WholeFunctional, GenericVocab, Dyn::Join(listed, "\n"));
return std::format("{}\n{}\nUnits to cover:\n{}", text, vocab, Dyn::Join(listed, "\n"));
}
const Json* found = nullptr;
for (const Json& unit : units) {
@ -222,7 +326,7 @@ namespace Crafter::Review::Prompt {
if (kind == "slot") {
const std::string description = Dyn::Replace(Text(Dyn::At(u, "excerpt")), "FIXED SLOT: ", "");
return std::format("SCOPE - SLOT PASS, unit \"{}\": this unit has no lines; it stands for a question about the whole artifact: {}. Answer it for your lens{}. Coverage: exactly this one unit.\n{}",
id, description, SlotDuty(lensFile, id), GenericVocab);
id, description, SlotDuty(lensFile, id), vocab);
}
const Json& range = Dyn::At(u, "lines");
const std::int64_t a = IntItem(range, 0);
@ -230,36 +334,44 @@ namespace Crafter::Review::Prompt {
const Json& unitKind = Dyn::Get(job, "unitKind");
const bool prose = unitKind == "text" || unitKind == "doc";
std::string_view hint;
if (id.ends_with(":outside")) {
hint = HintOutside;
if (id == "anchor") {
hint = HintAnchor;
} else if (id == "refs") {
hint = HintRefs;
} else if (id.ends_with(":outside")) {
hint = lens.kernel ? HintOutsideKernel : HintOutside;
} else if (id.ends_with(":absent")) {
hint = HintAbsent;
hint = lens.kernel ? HintAbsentKernel : HintAbsent;
} else if (lensFile == Plan::FunctionalLens) {
hint = FunctionalBlockHint;
} else if (unitKind == "text") {
hint = TextBlockHintRepo;
hint = lens.kernel ? TextBlockHint : TextBlockHintRepo;
} else if (unitKind == "doc") {
hint = DocBlockHintRepo;
}
const std::string text = Plan::UnitText(lines, u);
// The kernel modes' block judges a claim, a sentence or a hunk in one
// phrase; the push gates' says which the unit is.
const std::string_view against = lens.kernel ? "a claim at its source, a sentence against its proof obligation, a hunk against the standard"
: prose ? "a claim at its source, a sentence against its proof obligation"
: "a hunk against the standard";
const std::string scope = std::format("SCOPE - BLOCK PASS, unit \"{}\" [lines {}-{}]: judge ONLY this unit, with your full attention by your charter - {} - reading the rest of the artifact only as context. Do not re-litigate the approach as a whole{}. Coverage: exactly this one unit.",
id, a, b, prose ? "a claim at its source, a sentence against its proof obligation" : "a hunk against the standard",
wholeSummary.empty() ? "" : " - the whole pass of this lens concluded as stated above");
return std::format("{}{}\n\nUNIT TEXT (verbatim, lines {}-{}):\n{}\n\n{}", scope, hint.empty() ? std::string() : std::format("\n{}", hint), a, b, text, GenericVocab);
id, a, b, against, wholeSummary.empty() ? "" : " - the whole pass of this lens concluded as stated above");
return std::format("{}{}\n\nUNIT TEXT (verbatim, lines {}-{}):\n{}\n\n{}", scope, hint.empty() ? std::string() : std::format("\n{}", hint), a, b, text, vocab);
}
std::string BudgetLine(const Json& job) {
std::string BudgetLine(const Json& job, bool kernel) {
return std::format("\n\nTool-call budget for this pass: {} (every tool call is one request; the pipeline stops the process at that count and a stopped pass "
"returns NOTHING - the unit is then unaddressed and the lens NOT RETURNED). Batch independent reads into one call and return the structured output with margin to spare.",
Dyn::Str(Dyn::Get(job, "maxTurns")));
"returns NOTHING - the unit is then unaddressed and the lens NOT RETURNED). Batch independent reads into one call{} and return the structured output with margin to spare.",
Dyn::Str(Dyn::Get(job, "maxTurns")), kernel ? ", read the prefetched raw files instead of fetching," : "");
}
std::string SystemPrompt(std::string_view systemText, std::string_view run, std::string_view lens) {
return std::format("{}{}{}{}{}", systemText, SepRun, run, SepLens, lens);
}
std::string UserPrompt(std::string_view tail, const Json& job, std::string_view stack) {
std::string UserPrompt(std::string_view tail, const Json& job, std::string_view stack, bool kernel) {
return std::format("{}{}{}\n\nReturn the structured output described in your instructions.\n", tail, stack.empty() ? std::string() : std::format("\n\n{}", stack),
Dyn::Truthy(job) ? BudgetLine(job) : std::string());
Dyn::Truthy(job) ? BudgetLine(job, kernel) : std::string());
}
}

325
implementations/Crafter.Review-Refs.cpp Normal file
View file

@ -0,0 +1,325 @@
// SPDX-License-Identifier: AGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
module Crafter.Review:Refs_impl;
import :Refs;
import :Json;
import :Text;
import :Platform;
import :Process;
import std;
namespace fs = std::filesystem;
namespace Crafter::Review::Refs {
namespace {
constexpr std::string_view LoreHosts[] = {"https://lore.kernel.org/", "http://lore.kernel.org/"};
bool IsListChar(char c) {
return (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '-';
}
// LORE_RE at `at`: https?://lore\.kernel\.org/(?:r|all|[a-z0-9-]+)/([^/\s>]+)/?
// -> (the end of the match, the message id), or nullopt.
std::optional<std::pair<std::size_t, std::string>> LoreAt(std::string_view text, std::size_t at) {
std::size_t after = 0;
for (const std::string_view host : LoreHosts) {
if (text.substr(at).starts_with(host)) {
after = at + host.size();
break;
}
}
if (after == 0) return std::nullopt;
std::size_t list = after;
while (list < text.size() && IsListChar(text[list])) ++list;
if (list == after || list >= text.size() || text[list] != '/') return std::nullopt;
std::size_t id = list + 1;
std::size_t end = id;
while (end < text.size()) {
const auto [c, used] = Dyn::CodePointAt(text, end);
if (c == '/' || c == '>' || Dyn::IsSpace(c)) break;
end += used;
}
if (end == id) return std::nullopt;
const std::string msgid(text.substr(id, end - id));
if (end < text.size() && text[end] == '/') ++end;
return std::pair(end, msgid);
}
Fetcher& TheFetcher() {
static Fetcher fetcher;
return fetcher;
}
Searcher& TheSearcher() {
static Searcher searcher;
return searcher;
}
std::string Tail(std::string_view text, std::size_t n) {
return Dyn::Slice(text, -static_cast<std::int64_t>(n));
}
}
Table ParseRefs(std::string_view text) {
Table refs;
auto set = [&](const std::string& key, const std::string& url) {
for (auto& [k, u] : refs) {
if (k == key) {
u = url;
return;
}
}
refs.emplace_back(key, url);
};
// ^\[(\d+)\] (\S+), multiline.
for (const std::string& line : Dyn::Split(text, "\n")) {
if (!line.starts_with('[')) continue;
const std::size_t digits = Dyn::DecimalRun(line, 1);
if (digits == 0 || !std::string_view(line).substr(1 + digits).starts_with("] ")) continue;
const std::size_t urlAt = 1 + digits + 2;
const std::size_t urlLen = Dyn::NonSpaceRun(line, urlAt);
if (urlLen == 0) continue;
const std::optional<std::int64_t> n = Dyn::ParseInt(line.substr(1, digits));
if (!n) continue;
set(std::to_string(*n), line.substr(urlAt, urlLen));
}
if (!refs.empty()) return refs;
std::vector<std::string> seen;
for (std::size_t at = 0; at < text.size();) {
const std::optional<std::pair<std::size_t, std::string>> found = LoreAt(text, at);
if (!found) {
++at;
continue;
}
const std::string whole(text.substr(at, found->first - at));
if (!std::ranges::contains(seen, whole)) seen.push_back(whole);
at = found->first;
}
for (std::size_t k = 0; k < seen.size(); ++k) refs.emplace_back(std::format("L{}", k + 1), seen[k]);
return refs;
}
std::optional<std::string> UrlOf(const Table& refs, std::int64_t n) {
const std::string key = std::to_string(n);
for (const auto& [k, url] : refs) {
if (k == key) return url;
}
return std::nullopt;
}
std::optional<std::string> MsgidOf(std::string_view url) {
for (std::size_t at = 0; at < url.size(); ++at) {
if (const std::optional<std::pair<std::size_t, std::string>> found = LoreAt(url, at)) return found->second;
}
return std::nullopt;
}
std::string SafeName(std::string_view msgid) {
std::u32string out;
for (const char32_t c : Dyn::Decode(msgid)) {
const bool kept = (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '.' || c == '_' || c == '@' || c == '-';
out.push_back(kept ? c : U'_');
}
if (out.size() > 200) out.resize(200);
return Dyn::Encode(out);
}
std::map<std::string, std::string> Headers(std::string_view raw) {
std::map<std::string, std::string> out;
const std::string head = Dyn::Split(raw, "\n\n", 1).front();
for (const std::string& line : Dyn::SplitLines(head)) {
for (const std::string_view key : {"From", "Date", "Subject"}) {
const std::string prefix = std::format("{}:", key);
if (line.starts_with(prefix) && !out.contains(std::string(key))) out[std::string(key)] = Dyn::Strip(std::string_view(line).substr(prefix.size()));
}
}
return out;
}
void SetFetcher(Fetcher fetcher) {
TheFetcher() = std::move(fetcher);
}
std::pair<std::string, std::string> Fetch(const fs::path& repo, const std::string& url, const fs::path& path) {
if (TheFetcher()) return TheFetcher()(repo, url, path);
std::string last;
for (std::int32_t attempt = 0; attempt < 3; ++attempt) {
Process::Captured p;
try {
p = Process::Run({"python3", "utilities/lore.py", "fetch", std::format("{}/raw", Dyn::RStrip(url, "/"))}, repo, std::chrono::milliseconds(180000));
} catch (const std::exception& e) {
last = Dyn::Slice(std::format("{}: {}", "RuntimeError", e.what()), 0, 200);
std::this_thread::sleep_for(std::chrono::seconds(2));
continue;
}
if (p.timedOut) {
last = Dyn::Slice(std::format("TimeoutExpired: Command '{}' timed out after 180 seconds", url), 0, 200);
std::this_thread::sleep_for(std::chrono::seconds(2));
continue;
}
if (p.returnCode == 0 && !Dyn::Strip(p.out).empty()) {
Platform::WriteAtomic(path, p.out);
return {"fetched", ""};
}
last = Tail(Dyn::DecodeReplace(p.err.empty() ? p.out : p.err), 200);
std::this_thread::sleep_for(std::chrono::seconds(2));
}
return {"failed", last};
}
Json Prefetch(const fs::path& repo, const Table& refs, const fs::path& refsDir, const fs::path& indexPath, const Log& log, std::int32_t workers) {
fs::create_directories(refsDir);
Json index = Json::object();
struct Todo {
std::string key;
std::string url;
fs::path path;
};
std::vector<Todo> todo;
for (const auto& [n, url] : refs) {
const std::optional<std::string> mid = MsgidOf(url);
if (!mid) {
index[n] = {{"url", url}, {"status", "not-prefetched (not a lore URL)"}};
continue;
}
const fs::path path = refsDir / std::format("{}.raw", SafeName(*mid));
index[n] = {{"url", url}, {"msgid", *mid}, {"path", Dyn::RelPath(path.string(), repo.string())}};
std::error_code error;
if (fs::exists(path, error) && fs::file_size(path, error) > 0) {
index[n]["status"] = "cached";
} else {
todo.push_back({n, url, path});
}
}
std::vector<std::pair<std::string, std::string>> results(todo.size());
{
std::atomic<std::size_t> next = 0;
std::vector<std::jthread> pool;
const std::size_t width = std::min<std::size_t>(todo.size(), static_cast<std::size_t>(std::max(1, workers)));
for (std::size_t w = 0; w < width; ++w) {
pool.emplace_back([&] {
for (std::size_t i = next.fetch_add(1); i < todo.size(); i = next.fetch_add(1)) results[i] = Fetch(repo, todo[i].url, todo[i].path);
});
}
}
for (std::size_t i = 0; i < todo.size(); ++i) {
index[todo[i].key]["status"] = results[i].first;
if (!results[i].second.empty()) index[todo[i].key]["error"] = results[i].second;
}
for (auto it = index.begin(); it != index.end(); ++it) {
Json& e = it.value();
const Json& status = Dyn::Get(e, "status");
if (!Dyn::Truthy(Dyn::Get(e, "path")) || !(status == "cached" || status == "fetched")) continue;
const std::optional<std::string> bytes = Platform::ReadFile(repo / Dyn::Str(e.at("path")));
if (!bytes) {
e["status"] = "failed";
e["error"] = std::format("[Errno 2] No such file or directory: {}", Dyn::Repr((repo / Dyn::Str(e.at("path"))).string()));
continue;
}
const std::string raw = Dyn::DecodeReplace(*bytes);
for (const auto& [k, v] : Headers(raw)) e[k] = v;
e["bytes"] = Dyn::Length(raw);
}
Platform::WriteAtomic(indexPath, DumpFile(index));
std::int64_t ok = 0;
std::int64_t fetched = 0;
for (auto it = index.begin(); it != index.end(); ++it) {
const Json& status = Dyn::Get(it.value(), "status");
if (status == "cached" || status == "fetched") ++ok;
if (status == "fetched") ++fetched;
}
log(std::format("refs: {}/{} lore references available locally ({} fetched now)", ok, index.size(), fetched));
return index;
}
std::string Digest(const Json& index) {
std::vector<std::string> lines = {
"REFERENCES prefetched (read the raw file instead of fetching; a FETCH FAILED line means the reference must be fetched by you or reported as unverifiable):"};
std::vector<std::string> keys;
for (auto it = index.begin(); it != index.end(); ++it) keys.push_back(it.key());
auto numeric = [](const std::string& k) { return !k.empty() && Dyn::DecimalRun(k, 0) == k.size(); };
std::ranges::sort(keys, [&](const std::string& a, const std::string& b) {
const bool na = numeric(a);
const bool nb = numeric(b);
if (na != nb) return na;
if (na) return *Dyn::ParseInt(a) < *Dyn::ParseInt(b);
return a < b;
});
const Json unknown = "?";
for (const std::string& n : keys) {
const Json& e = index.at(n);
const Json& status = Dyn::Get(e, "status");
if (status == "cached" || status == "fetched") {
const Json& subject = Dyn::Has(e, "Subject") ? e.at("Subject") : unknown;
lines.push_back(std::format("[{}] {} | From: {} | Date: {} | Subject: {} | {} ({} B)", n, Dyn::Str(Dyn::Get(e, "msgid")), Dyn::Str(Dyn::Has(e, "From") ? e.at("From") : unknown),
Dyn::Str(Dyn::Has(e, "Date") ? e.at("Date") : unknown), Dyn::Slice(Dyn::Str(subject), 0, 100), Dyn::Str(e.at("path")),
Dyn::Str(Dyn::Has(e, "bytes") ? e.at("bytes") : Json(0))));
} else if (status == "failed") {
lines.push_back(std::format("[{}] {} | FETCH FAILED | {}", n, Dyn::Str(Dyn::Get(e, "msgid")), Dyn::Str(e.at("url"))));
} else {
lines.push_back(std::format("[{}] {} | {}", n, Dyn::Str(e.at("url")), Dyn::Str(status)));
}
}
return Dyn::Join(lines, "\n");
}
void SetSearcher(Searcher searcher) {
TheSearcher() = std::move(searcher);
}
std::pair<std::optional<fs::path>, std::string> PersonaCorpus(const fs::path& repo, std::string_view name, const fs::path& refsDir, std::string_view since, const Log& log) {
// re.sub(r'[^a-z0-9]+', '-', name.lower()).strip('-')
std::u32string slug;
bool dash = false;
for (const char32_t c : Dyn::Decode(Dyn::Lower(name))) {
if ((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9')) {
slug.push_back(c);
dash = false;
} else if (!dash) {
slug.push_back(U'-');
dash = true;
}
}
const std::string slugText = Dyn::Strip(Dyn::Encode(slug), "-");
const fs::path path = refsDir / std::format("persona-{}.txt", slugText);
std::error_code error;
if (fs::exists(path, error) && fs::file_size(path, error) > 0) return {path, Dyn::ReadText(path)};
const std::string q = std::format("f:\"{}\" AND d:{}..", name, since);
std::string out;
std::string err;
if (TheSearcher()) {
std::tie(out, err) = TheSearcher()(repo, q);
} else {
try {
const Process::Captured p = Process::Run({"python3", "utilities/lore.py", "search", "-n", "100", q}, repo, std::chrono::milliseconds(180000));
out = Dyn::DecodeReplace(p.out);
err = Dyn::DecodeReplace(p.err);
} catch (const std::exception& e) {
err = e.what();
}
}
const std::string text = Dyn::Strip(out);
if (text.empty() || Dyn::Lower(text).starts_with("no results")) {
log(std::format("persona corpus: {} -> NO RESULTS for {} (stderr: {}) - the whole pass calibrates by its own searches", name, Dyn::Repr(q), Dyn::Slice(Dyn::Strip(err), 0, 120)));
return {std::nullopt, ""};
}
fs::create_directories(refsDir);
Platform::WriteAtomic(path, std::format("lore.py search {} ({})\n{}\n", Dyn::Repr(q), Platform::LocalTime("%Y-%m-%d"), text));
log(std::format("persona corpus: {} -> {} ({} chars)", name, Dyn::RelPath(path.string(), repo.string()), Dyn::Length(text)));
return {path, Dyn::ReadText(path)};
}
std::string SinceOf(std::string_view isoDate) {
const std::vector<std::string> parts = Dyn::Split(isoDate, "-");
if (parts.size() != 3) throw Dyn::Error("ValueError", std::format("Invalid isoformat string: {}", Dyn::Repr(isoDate)));
const std::optional<std::int64_t> y = Dyn::ParseInt(parts[0]);
const std::optional<std::int64_t> m = Dyn::ParseInt(parts[1]);
const std::optional<std::int64_t> d = Dyn::ParseInt(parts[2]);
if (!y || !m || !d) throw Dyn::Error("ValueError", std::format("Invalid isoformat string: {}", Dyn::Repr(isoDate)));
const std::chrono::year_month_day date{std::chrono::year(static_cast<int>(*y)), std::chrono::month(static_cast<unsigned>(*m)), std::chrono::day(static_cast<unsigned>(*d))};
if (!date.ok()) throw Dyn::Error("ValueError", std::format("Invalid isoformat string: {}", Dyn::Repr(isoDate)));
const std::chrono::year_month_day since{std::chrono::sys_days(date) - std::chrono::days(365)};
return std::format("{:04}{:02}{:02}", static_cast<std::int32_t>(since.year()), static_cast<std::uint32_t>(since.month()), static_cast<std::uint32_t>(since.day()));
}
}

331
implementations/Crafter.Review-Run.cpp
View file

@ -173,8 +173,10 @@ namespace Crafter::Review::Run {
return {
{"config", opt(a.config)},
{"mode", a.mode},
{"subsystem", opt(a.subsystem)},
{"artifact", a.artifact},
{"hash", opt(a.hash)},
{"reviewers", a.reviewers},
{"dispositions", opt(a.dispositions)},
{"base", a.base},
{"date", a.date},
@ -189,6 +191,7 @@ namespace Crafter::Review::Run {
{"reads", a.reads},
{"no_reuse", a.noReuse},
{"no_salvage", a.noSalvage},
{"persona_blocks", a.personaBlocks},
{"dry_run", a.dryRun},
{"claude_bin", opt(a.claudeBin)},
{"models", a.models},
@ -332,6 +335,10 @@ namespace Crafter::Review::Run {
environments = Config::Environments(cfg);
}
bool Run::Kernel() const {
return Plan::IsKernelMode(a.mode);
}
Run::~Run() {
// atexit's board.leave: everything of this driver off the board.
if (board) board->Leave();
@ -373,21 +380,29 @@ namespace Crafter::Review::Run {
// ---- Gate 0 -------------------------------------------------------------------------
void Run::Gate0() {
if (!fs::exists(a.artifact)) Die(std::format("artifact {} does not exist", a.artifact));
hash = Artifact::ArtifactHash("commits", a.artifact);
hash = Artifact::ArtifactHash(a.mode, a.artifact);
if (a.hash && !a.hash->empty() && *a.hash != hash) Die(std::format("--hash {} does not match the artifact ({})", Dyn::Slice(*a.hash, 0, 12), hash.substr(0, 12)));
h8 = hash.substr(0, 8);
const std::optional<std::string> patchHead = Artifact::PatchHead(a.artifact);
if (!patchHead || patchHead->empty()) Die("the last patch has no `From <sha>` line - not a git format-patch export (use `freeze`)");
head = *patchHead;
for (const std::string& t : trees) {
if (!Artifact::IsToplevel(t)) Die(std::format("--trees: {} is not the toplevel of a git checkout", t));
if (a.mode == "commits") {
// The lenses read the live checkout named first in --trees: it
// must be the range HEAD the artifact was cut from, and clean -
// anything else is evidence the review never saw (re-checked
// before every launch).
const std::optional<std::string> patchHead = Artifact::PatchHead(a.artifact);
if (!patchHead || patchHead->empty()) Die("the last patch has no `From <sha>` line - not a git format-patch export (use `freeze`)");
head = *patchHead;
for (const std::string& t : trees) {
if (!Artifact::IsToplevel(t)) Die(std::format("--trees: {} is not the toplevel of a git checkout", t));
}
if (const std::optional<std::string> drift = treeDrift()) Die(std::format("{} - re-freeze, or check out the frozen worktree (freeze --worktree)", *drift));
}
if (const std::optional<std::string> drift = treeDrift()) Die(std::format("{} - re-freeze, or check out the frozen worktree (freeze --worktree)", *drift));
gateDir = Dyn::JoinPath(a.artifact, "gate-state");
// A reply's gate-state sits beside the draft; a directory artifact's
// inside it.
gateDir = Dyn::JoinPath(a.mode == "reply" ? Dyn::DirName(a.artifact) : a.artifact, "gate-state");
fs::create_directories(gateDir);
Inventory::Built built;
try {
built = Inventory::Build("commits", a.artifact);
built = Inventory::Build(a.mode, a.artifact);
} catch (const Inventory::InvariantError& e) {
Die(e.what());
} catch (const Dyn::Error& e) {
@ -418,16 +433,16 @@ namespace Crafter::Review::Run {
Json holder;
if (!a.dryRun && !testMode) {
try {
runLock = std::make_unique<State::RunLock>(gateDir, "commits", h8);
runLock = std::make_unique<State::RunLock>(gateDir, a.mode, h8);
} catch (const State::RunLocked& e) {
const std::string how = Platform::InSandbox() ? std::format("wait for it or stop it (`{} stop <its scratch dir>`)", Invocation()) : std::string("wait for it or kill it");
Die(std::format("a full run on this hash is already live in {}: {} - {}; a dead holder releases the lock", Dyn::RelPath(gateDir.string(), repo.string()), e.what(), how));
}
} else {
const std::optional<Json> found = State::LockHolder(gateDir, "commits", h8);
const std::optional<Json> found = State::LockHolder(gateDir, a.mode, h8);
if (found) holder = *found;
}
std::int64_t runN = State::NextRunNumber(gateDir, "commits", h8);
std::int64_t runN = State::NextRunNumber(gateDir, a.mode, h8);
if (Dyn::Truthy(holder) && Dyn::Truthy(Dyn::Get(holder, "run_n"))) runN = std::max(runN, Int(holder.at("run_n")) + 1);
const fs::path scratchRoot = ScratchRoot(a.scratch);
// A run killed before its first checkpoint left no dump but did leave
@ -445,7 +460,7 @@ namespace Crafter::Review::Run {
// run that bought nothing (no outputs in its dir) would otherwise have
// its number taken again and its dump written over.
auto dumped = [&](std::int64_t n) {
return fs::exists(gateDir / std::format("commits-{}-run-{}.json", h8, n)) || fs::exists(gateDir / std::format("commits-{}-run-{}-test.json", h8, n));
return fs::exists(gateDir / std::format("{}-{}-run-{}.json", a.mode, h8, n)) || fs::exists(gateDir / std::format("{}-{}-run-{}-test.json", a.mode, h8, n));
};
while (!a.dryRun) {
const fs::path candidate = scratchRoot / std::format("{}-run-{}", h8, runN);
@ -473,7 +488,7 @@ namespace Crafter::Review::Run {
const std::lock_guard guard(shared);
stopFile = scratch / "stop";
}
dumpPath = a.dryRun ? scratch / "dry-run.json" : gateDir / std::format("commits-{}-run-{}{}.json", h8, runN, testMode ? "-test" : "");
dumpPath = a.dryRun ? scratch / "dry-run.json" : gateDir / std::format("{}-{}-run-{}{}.json", a.mode, h8, runN, testMode ? "-test" : "");
if (runLock) {
runLock->WriteInfo({{"pid", Platform::PublishedPid()},
{"since", Platform::LocalTime("%Y-%m-%dT%H:%M:%S")},
@ -482,25 +497,33 @@ namespace Crafter::Review::Run {
{"run_n", runN}});
}
logFile = scratch / "log.txt";
log(std::format("crafter-review {} prompt v{}: commits {} @ {}", Version(), PromptVersion, Dyn::RelPath(a.artifact, repo.string()), hash));
log(std::format("crafter-review {} prompt v{}: {} {} @ {}", Version(), PromptVersion, a.mode, Dyn::RelPath(a.artifact, repo.string()), hash));
std::error_code canonical;
if (fs::weakly_canonical(reviewed, canonical) != fs::weakly_canonical(repo, canonical)) log(std::format("repository under review: {} (read at {})", reviewed.string(), trees.front()));
if (a.mode == "commits" && fs::weakly_canonical(reviewed, canonical) != fs::weakly_canonical(repo, canonical)) log(std::format("repository under review: {} (read at {})", reviewed.string(), trees.front()));
log(std::format("config: {}", Dyn::Truthy(Dyn::Get(cfg, "_path")) ? Dyn::Str(cfg.at("_path")) : std::string("defaults (no .review-gates/config.json)")));
for (const Json& w : Dyn::Or(Dyn::Get(cfg, "_warnings"), Dyn::EmptyList())) log(std::format("config: warning: {}", Dyn::Str(w)));
// The mode's lenses: the others are another mode's and say nothing
// about this run.
const std::vector<std::string> modeLenses = Plan::ModeLenses(a.mode);
std::vector<std::string> lensParts;
for (const Plan::Lens& lens : Plan::Lenses) {
const std::string lf(lens.file);
for (const std::string& lf : modeLenses) {
const std::string name = Plan::LensName(lf);
lensParts.push_back(std::ranges::contains(off, lf) ? std::format("{} OFF (lenses.{}.run = false)", name, name) : std::format("{} ({})", name, Dyn::Str(Dyn::At(environments, lf))));
}
log(std::format("lenses: {}", Dyn::Join(lensParts, ", ")));
std::vector<std::string> running;
for (const Plan::Lens& lens : Plan::Lenses) {
if (!std::ranges::contains(off, std::string(lens.file))) running.emplace_back(lens.file);
for (const std::string& lf : modeLenses) {
if (!std::ranges::contains(off, lf)) running.push_back(lf);
}
const std::vector<std::string> exclude = Strings(cfg.at("functional").at("exclude"));
notReviewed = Plan::NotReviewed(units, lines, running, exclude);
log(std::format("inventory: {} units ({}); {} units no lens that runs reads", units.size(), Dyn::RelPath(invPath.string(), repo.string()), notReviewed.size()));
if (Kernel()) {
// The kernel modes excuse nothing: a unit no lens owns is a hole.
notReviewed.clear();
log(std::format("inventory: {} units ({})", units.size(), Dyn::RelPath(invPath.string(), repo.string())));
} else {
notReviewed = Plan::NotReviewed(units, lines, running, exclude);
log(std::format("inventory: {} units ({}); {} units no lens that runs reads", units.size(), Dyn::RelPath(invPath.string(), repo.string()), notReviewed.size()));
}
if (a.units) {
std::set<std::string> known;
for (const Json& u : units) known.insert(Dyn::Str(u.at("id")));
@ -511,11 +534,49 @@ namespace Crafter::Review::Run {
if (!unknown.empty()) Die(std::format("--units names ids not in the inventory: {}", Dyn::Join(unknown, ", ")));
}
dispEntries = Dispositions::Load(a.dispositions.value_or(""));
dispRows = Dispositions::Classify(dispEntries, hash, a.artifact);
dispRows = Dispositions::Classify(dispEntries, hash, a.artifact, a.mode);
inForce = Dispositions::InForce(dispEntries, dispRows);
std::vector<std::string> rowParts;
for (const Json& r : dispRows) rowParts.push_back(std::format("{}:{}", Dyn::Str(r.at("id")), Dyn::Str(r.at("status"))));
log(std::format("dispositions: {} in force of {} ({})", inForce.size(), dispEntries.size(), rowParts.empty() ? std::string("none") : Dyn::Join(rowParts, ", ")));
if (Kernel()) {
// The references the artifact cites, prefetched into the
// gate-state (a dry run reads an existing index and fetches
// nothing), and each reviewer's calibration map.
const std::string text = a.mode == "reply" ? Dyn::ReadText(fs::path(a.artifact)) : Dyn::Join(lines, "\n");
refUrls = Refs::ParseRefs(text);
const fs::path refsDir = gateDir / "refs";
const fs::path indexPath = gateDir / std::format("refs-{}.json", h8);
if (a.dryRun && fs::exists(indexPath)) {
refIndex = Dyn::LoadJsonFile(indexPath);
log(std::format("refs: {} entries loaded from the existing index (dry run, no fetch)", refIndex.size()));
} else {
refIndex = Refs::Prefetch(repo, refUrls, refsDir, indexPath, log);
}
refDigest = Dyn::Truthy(refIndex) ? Refs::Digest(refIndex) : std::string("REFERENCES prefetched: (none - the artifact cites no lore URLs)");
const std::string since = Refs::SinceOf(a.date);
corpora.clear();
for (const std::string& name : reviewers) {
auto [path, corpus] = Refs::PersonaCorpus(repo, name, refsDir, since, log);
if (corpus.empty()) {
// A private contributor (no lore record): their mail is
// indexed in the ledger's live-threads table.
const std::string s = a.subsystem.value_or("");
corpus = std::format("lore.py search for mail from {} since {}: NO RESULTS - this person has no mail on lore in the last year. "
"Their mail is PRIVATE: the Live threads table in journal/{}/ledger.md indexes every message with its capture path "
"under journal/{}/captures/, and their submissions are frozen under journal/{}/patch/. Calibrate on those, verbatim.",
name, since, s, s, s);
}
corpora[name] = corpus;
}
anchorText.clear();
for (const Json& u : units) {
if (u.at("id") == "anchor") {
anchorText = Plan::UnitText(lines, u);
break;
}
}
}
statePath = gateDir / "unit-state.json";
state = a.noReuse ? State::EmptyState() : State::LoadState(statePath);
salvageDirs.clear();
@ -551,7 +612,7 @@ namespace Crafter::Review::Run {
round = (Dyn::Truthy(r) ? Int(r) : 1) + (newRound ? 1 : 0);
}
if (!a.dryRun) WriteText(scratch / "round", std::to_string(round));
log(std::format("reads: every code unit of the static code lenses needs {} clean reads, one per round; this is round {} on this hash{}", x, round,
log(std::format("reads: every code unit of the {}code lenses needs {} clean reads, one per round; this is round {} on this hash{}", Kernel() ? "" : "static ", x, round,
newRound || !sameHash ? std::string() : std::string(" (continuing an unfinished run)")));
}
for (const Plan::Lens& lens : Plan::Lenses) {
@ -559,17 +620,18 @@ namespace Crafter::Review::Run {
charters[lf] = Prompt::Charter(lf);
charterSha[lf] = Plan::Sha(charters[lf]);
}
systemText = Prompt::SystemText();
systemText = Prompt::SystemText(a.mode);
systemPaths.clear();
systemByLens.clear();
runLayer = Prompt::RunLayer(Dyn::RelPath(a.artifact, repo.string()), a.base, a.date, trees, hash, lines, units, contextPaths, Basename(reviewed));
runLayer = Kernel() ? Prompt::RunLayerKernel(a.mode, Dyn::RelPath(a.artifact, repo.string()), a.subsystem.value_or(""), a.base, reviewers, a.date, trees, hash, lines, units, refDigest, contextPaths)
: Prompt::RunLayer(Dyn::RelPath(a.artifact, repo.string()), a.base, a.date, trees, hash, lines, units, contextPaths, Basename(reviewed), a.subsystem.value_or(""));
// The application under test: the login fixture, the TOTP helper.
const Json& fcfg = cfg.at("functional");
loginFixture = Config::LoginFixture(cfg, repo);
stacks = LoadStacks();
const Json first = *stacks->First();
appBlock = Prompt::AppBlock(fcfg, loginFixture, Pool::TotpCommand(loginFixture), slotMode);
schemas = {{"block", Schema::SchemaJson("block")}, {"whole", Schema::SchemaJson("whole")}};
schemas = {{"block", Schema::SchemaJson("block", a.mode)}, {"whole", Schema::SchemaJson("whole", a.mode)}};
// Memory: every pass's tree measured, its peak recorded per repository
// and environment; the budget per gate.
calibration = std::make_unique<Memory::Calibration>(scratchRoot / "memory.json", reviewed, DefaultCosts());
@ -652,7 +714,12 @@ namespace Crafter::Review::Run {
}
const Json pc = SeatPool::PoolConfig(processEnv, a.effort);
board = std::make_unique<SeatPool::Board>(SeatPool::BoardPath(), std::string(), std::nullopt, log);
board->Join({{"run", std::format("commits {} @ {}", Basename(Dyn::RStrip(a.artifact, "/")), h8)},
std::vector<std::string> runParts = {a.mode};
if (a.subsystem && !a.subsystem->empty()) runParts.push_back(*a.subsystem);
runParts.push_back(Basename(Dyn::RStrip(a.artifact, "/")));
runParts.emplace_back("@");
runParts.push_back(h8);
board->Join({{"run", Dyn::Join(runParts, " ")},
{"dump", Basename(dumpPath)},
{"scratch", scratch.string()},
{"effort", a.effort},
@ -1061,7 +1128,10 @@ namespace Crafter::Review::Run {
}
const Plan::Environment& env = *Plan::FindEnvironment(Dyn::Str(Dyn::At(environments, lensFile)));
const std::string db = env.db ? Prompt::DbBlock(cfg.at("supabase"), Pool::SqlCommand(), dbRole, env.app, dbChecked, dbBypass) : std::string();
const std::string ll = Prompt::LensLayer(lensFile, charters.at(lensFile), Dispositions::Block(inForce, dispRows, lensFile, hash), env.app ? appBlock : std::string(), summary, db);
const std::string persona = Dyn::StrOr(Dyn::Get(job, "persona"), "");
const auto corpus = corpora.find(persona);
const std::string ll = Prompt::LensLayer(lensFile, charters.at(lensFile), Dispositions::Block(inForce, dispRows, lensFile, hash, a.subsystem.value_or("")), env.app ? appBlock : std::string(), summary, db,
persona, corpus == corpora.end() ? std::string() : corpus->second);
lensLayersBytes[lens] = static_cast<std::int64_t>(Dyn::Length(ll));
const std::string sysText = Prompt::SystemPrompt(systemText, runLayer, ll);
const std::string sha = Plan::Sha(sysText);
@ -1084,7 +1154,7 @@ namespace Crafter::Review::Run {
const Json& login = Dyn::Get(stack, "login");
stackText = Prompt::StackText(stack, Pool::TotpCommand(login.is_null() ? std::nullopt : std::optional<std::string>(Dyn::Str(login))));
}
return {found->second, Prompt::UserPrompt(Prompt::Tail(job, units, lines, summary), job, stackText)};
return {found->second, Prompt::UserPrompt(Prompt::Tail(job, units, lines, summary), job, stackText, Kernel())};
}
std::optional<std::string> Run::CacheKey(Json& job) {
@ -1099,7 +1169,16 @@ namespace Crafter::Review::Run {
// ---- reuse --------------------------------------------------------------------------
const Plan::CommitCode& Run::RangeCode() {
if (!commitCode) commitCode = Plan::CommitCodeOf(Artifact::PatchTexts(a.artifact));
if (!commitCode) {
commitCode = Plan::CommitCodeOf(Artifact::PatchTexts(a.artifact), a.mode == "series");
if (a.mode == "series" && !a.base.empty()) {
// A kernel export carries a base-commit line only when it was
// cut with --base, and then in the cover letter: the
// submission base the run was given is part of every code
// key as well.
commitCode->base = commitCode->base.empty() ? a.base : std::format("{},{}", commitCode->base, a.base);
}
}
return *commitCode;
}
@ -1108,6 +1187,13 @@ namespace Crafter::Review::Run {
// a part, re-normalised against today's dispositions, with its summary
// for the block passes. nullopt: the whole pass runs.
const std::string lensFile = Dyn::Str(job.at("lensFile"));
if (a.mode != "commits") {
// Only the push gates reuse a whole pass; a kernel whole pass
// runs on every hash.
job["reuseKey"] = nullptr;
job["otherKeys"] = Json::array();
return std::nullopt;
}
std::vector<std::string> ids;
for (const Json& d : inForce) ids.push_back(Dyn::Str(d.at("id")));
std::map<std::string, std::optional<std::string>> keys;
@ -1164,6 +1250,14 @@ namespace Crafter::Review::Run {
for (const Json& d : inForce) ids.push_back(Dyn::Str(d.at("id")));
const Json& configured = Dyn::Get(environments, lensFile);
const std::string env = environment && !environment->empty() ? *environment : (configured.is_null() ? std::string() : Dyn::Str(configured));
if (Kernel()) {
// A series' code unit is keyed on the code (the push gates'
// rule, 2026-10-06.6); every text unit on its text and what it
// cites: the [n] URLs, the anchor when it is dated, the R-units
// it names.
if (a.mode == "series" && Dyn::Get(job, "unitKind") == "code") return Plan::CodeReuseKey(job, lensFile, text, RangeCode(), ids, charterSha.at(lensFile), PromptVersion, env);
return Plan::ReuseKey(job, lensFile, text, Plan::TextDeps(text, units, lines, refUrls, anchorText), ids, charterSha.at(lensFile), PromptVersion);
}
return Plan::BlockReuseKey(job, lensFile, text, RangeCode(), ids, charterSha.at(lensFile), PromptVersion, env);
}
@ -1171,6 +1265,7 @@ namespace Crafter::Review::Run {
// The block's keys in every other environment: a defect its pass
// returns here drops the clean verdicts those hold for the same code -
// a finding is not undone by switching the lens's environment.
if (Kernel()) return {};
std::set<std::string> out;
for (const Plan::Environment& e : Plan::Environments) {
const std::optional<std::string> k = ReuseKey(job, std::string(e.name));
@ -1295,6 +1390,7 @@ namespace Crafter::Review::Run {
auto stamp = [&](Json& rec) {
rec["lens"] = job.at("lens");
rec["lensFile"] = job.at("lensFile");
rec["persona"] = Dyn::Get(job, "persona");
rec["kind"] = job.at("kind");
rec["unit"] = Dyn::Get(job, "unit");
rec["unitSha"] = Dyn::Get(job, "unitSha");
@ -1319,12 +1415,12 @@ namespace Crafter::Review::Run {
const std::lock_guard guard(shared);
if (sched) sched->Abort(reason);
};
const std::string now = Artifact::ArtifactHash("commits", a.artifact);
const std::string now = Artifact::ArtifactHash(a.mode, a.artifact);
if (now != hash) {
abortPool(std::format("artifact changed during the run ({} != {})", now.substr(0, 8), h8));
return {{{"label", job.at("label")}, {"status", "error"}, {"reason", "artifact changed during the run - aborted"}}, nullptr};
}
if (const std::optional<std::string> drift = treeDrift()) {
if (const std::optional<std::string> drift = a.mode == "commits" ? treeDrift() : std::nullopt) {
abortPool(std::format("evidence drift during the run ({})", *drift));
return {{{"label", job.at("label")}, {"status", "error"}, {"reason", std::format("{} - aborted", *drift)}}, nullptr};
}
@ -1380,23 +1476,33 @@ namespace Crafter::Review::Run {
// ---- a gate -----------------------------------------------------------------------------
std::vector<Json> Run::RunGate(std::string_view title, std::vector<std::string> lensFiles) {
// lensFiles -> the merged lens results, one per lens.
std::vector<std::pair<std::string, std::string>> specs;
for (const std::string& lf : lensFiles) specs.emplace_back(lf, std::string());
return RunGate(title, std::move(specs));
}
std::vector<Json> Run::RunGate(std::string_view title, std::vector<std::pair<std::string, std::string>> specs) {
// specs -> the merged lens results, one per lens instance.
std::vector<std::string> names;
for (const std::string& lf : lensFiles) names.push_back(Plan::LensName(lf));
for (const auto& [lf, persona] : specs) names.push_back(Plan::LensName(lf, persona));
log(std::format("{}: {}", title, Dyn::Join(names, ", ")));
std::vector<Json> resumed;
std::vector<std::string> live;
for (const std::string& lf : lensFiles) {
if (std::optional<Json> r = resumable(Plan::LensName(lf), lf, title)) {
std::vector<std::pair<std::string, std::string>> live;
for (const auto& [lf, persona] : specs) {
if (std::optional<Json> r = resumable(Plan::LensName(lf, persona), lf, title)) {
resumed.push_back(std::move(*r));
} else {
live.push_back(lf);
live.emplace_back(lf, persona);
}
}
lensFiles = live;
specs = live;
// Nothing runs: no stack, database or application is needed for a
// result already bought.
if (lensFiles.empty()) return resumed;
if (specs.empty()) return resumed;
std::vector<std::string> lensFiles;
for (const auto& [lf, persona] : specs) {
if (!std::ranges::contains(lensFiles, lf)) lensFiles.push_back(lf);
}
instrumentPreflight(lensFiles);
std::map<std::string, std::vector<Json>> parts;
std::map<std::string, std::set<std::string>> missing;
@ -1406,12 +1512,12 @@ namespace Crafter::Review::Run {
const std::vector<std::string> exclude = Strings(cfg.at("functional").at("exclude"));
std::vector<std::string> unitFilter;
if (a.units) unitFilter.assign(a.units->begin(), a.units->end());
for (const std::string& lf : lensFiles) {
const std::string name = Plan::LensName(lf);
for (const auto& [lf, persona] : specs) {
const std::string name = Plan::LensName(lf, persona);
parts[name];
missing[name];
planned[name];
Plan::LensPlan plan = Plan::PlanLens(lf, units, lines, unitFilter, exclude);
Plan::LensPlan plan = Plan::PlanLens(lf, units, lines, unitFilter, exclude, persona);
if (!plan.whole.is_null()) {
plan.whole["gate"] = title;
for (const Json& id : plan.whole.at("expectedIds")) planned[name].insert(Dyn::Str(id));
@ -1457,9 +1563,38 @@ namespace Crafter::Review::Run {
}
{
const std::lock_guard guard(shared);
summaries[name] = Dyn::Slice(Dyn::StrOr(Dyn::Get(part, "summary"), ""), 0, 2500);
// A persona's digest is its summary, else the review it
// would send.
std::string digest = Dyn::StrOr(Dyn::Get(part, "summary"), "");
if (digest.empty()) digest = Dyn::StrOr(Dyn::Get(part, "predictedReview"), "");
summaries[name] = Dyn::Slice(digest, 0, 2500);
}
return PrepareBlocks(name, pendingBlocks[name], parts);
std::vector<Json> blocks = pendingBlocks[name];
if (Dyn::Truthy(Dyn::Get(job, "persona")) && a.personaBlocks != "all") {
// The units the persona's whole pass marked not-mine get no
// block pass from it: each becomes a pruned part with that
// verdict.
std::vector<std::string> notMine;
for (const Json& id : Dyn::Or(Dyn::Get(part, "notMine"), Dyn::EmptyList())) notMine.push_back(Dyn::Str(id));
std::ranges::sort(notMine);
notMine.erase(std::ranges::unique(notMine).begin(), notMine.end());
if (!notMine.empty()) {
const double ratio = static_cast<double>(notMine.size()) / static_cast<double>(std::max<std::size_t>(1, blocks.size()));
log(std::format("{}: {} units marked not-mine by the whole pass ({:.0f}%){}; those block passes are pruned", name, notMine.size(), ratio * 100,
ratio > 0.5 ? " - WARNING above 50%" : ""));
for (const std::string& uid : notMine) {
parts[name].push_back({{"partLabel", std::format("{}:not-mine:{}", name, uid)}, {"partKind", "pruned"}, {"partUnit", uid}, {"verdict", "SEND-READY"}, {"findings", Json::array()},
{"notes", Json::array()}, {"coverage", Json::array({{{"item", uid}, {"verdict", "not-mine"}, {"basis", "whole pass: not this reviewer's tree, file or decision"}}})},
{"coverageMissing", Json::array()}, {"reused", nullptr}});
}
std::vector<Json> kept;
for (const Json& b : blocks) {
if (!std::ranges::contains(notMine, Dyn::Str(b.at("unit")))) kept.push_back(b);
}
blocks = kept;
}
}
return PrepareBlocks(name, blocks, parts);
};
const bool warm = !a.dryRun && (!testMode || a.warm);
std::unique_ptr<Pool::Admission> admission = Admission(lensFiles);
@ -1489,8 +1624,8 @@ namespace Crafter::Review::Run {
}
if (const std::optional<std::string> aborted = scheduler.Aborted()) abort = aborted;
std::vector<Json> out;
for (const std::string& lf : lensFiles) {
const std::string name = Plan::LensName(lf);
for (const auto& [lf, persona] : specs) {
const std::string name = Plan::LensName(lf, persona);
const bool stoppedEarly = std::ranges::any_of(parts[name], [](const Json& p) { return Dyn::Get(p, "partKind") == "skipped"; });
std::vector<Json> real;
for (const Json& p : parts[name]) {
@ -1699,7 +1834,13 @@ namespace Crafter::Review::Run {
for (const std::string& c : contextPaths) {
if (!fs::exists(c)) Die(std::format("--context-paths: {} does not exist", c));
}
reviewers = CommaList(a.reviewers);
if (!reviewers.empty() && std::ranges::contains(off, std::string(Plan::PersonaLens))) {
Die(std::format("--reviewers names {} but the config turns off the persona lens (lenses.{}.run = false): turn it on, or drop --reviewers", Dyn::Join(reviewers, ", "),
Plan::LensName(Plan::PersonaLens)));
}
Gate0();
const std::vector<std::string> modeLenses = Plan::ModeLenses(a.mode);
std::vector<std::string> only;
if (a.only && !a.only->empty()) {
only = CommaList(*a.only);
@ -1708,6 +1849,11 @@ namespace Crafter::Review::Run {
if (!Plan::IsLens(lf)) bad.push_back(lf);
}
if (!bad.empty()) Die(std::format("--only names unknown lens file(s): {}", Dyn::Join(bad, ", ")));
std::vector<std::string> foreign;
for (const std::string& lf : only) {
if (!std::ranges::contains(modeLenses, lf)) foreign.push_back(lf);
}
if (!foreign.empty()) Die(std::format("--only names lens file(s) the {} mode never runs (nothing would run): {}", a.mode, Dyn::Join(foreign, ", ")));
std::vector<std::string> isOff;
for (const std::string& lf : only) {
if (std::ranges::contains(off, lf)) isOff.push_back(lf);
@ -1724,16 +1870,25 @@ namespace Crafter::Review::Run {
if (!Plan::IsLens(lf)) bad.push_back(lf);
}
if (!bad.empty()) Die(std::format("--without names unknown lens file(s): {}", Dyn::Join(bad, ", ")));
std::vector<std::string> foreign;
for (const std::string& lf : without) {
if (!std::ranges::contains(modeLenses, lf)) foreign.push_back(lf);
}
if (!foreign.empty()) Die(std::format("--without names lens file(s) the {} mode never runs: {}", a.mode, Dyn::Join(foreign, ", ")));
log(std::format("without: {} - left out of this run by owner ruling (recorded in the dump's args); the run stays a full run", Dyn::Join(without, ", ")));
}
const std::vector<Plan::Gate> gates = Plan::GatesFor(only, without, off);
const std::vector<Plan::Gate> gates = Plan::GatesFor(a.mode, only, without, off);
if (gates.empty()) {
std::vector<std::string> why;
if (off.size() == Plan::Lenses.size()) {
std::vector<std::string> offHere;
for (const std::string& lf : off) {
if (std::ranges::contains(modeLenses, lf)) offHere.push_back(lf);
}
if (offHere.size() == modeLenses.size()) {
why.emplace_back("the config turns on no lens (every lens is off until lenses.<name>.run = true turns it on)");
} else if (!off.empty()) {
} else if (!offHere.empty()) {
std::vector<std::string> names;
for (const std::string& lf : off) names.push_back(Plan::LensName(lf));
for (const std::string& lf : offHere) names.push_back(Plan::LensName(lf));
why.push_back(std::format("the config turns off {}", Dyn::Join(names, ", ")));
}
if (!without.empty()) why.push_back(std::format("--without drops {}", Dyn::Join(without, ", ")));
@ -1771,6 +1926,32 @@ namespace Crafter::Review::Run {
break;
}
}
personaRan = false;
const std::string personaFile(Plan::PersonaLens);
if (!Dyn::Truthy(stoppedAt) && !reviewers.empty() && (only.empty() || std::ranges::contains(only, personaFile)) && !std::ranges::contains(without, personaFile)) {
// The persona gate: one instance of the persona lens per
// reviewer, after every other gate passed.
std::vector<std::pair<std::string, std::string>> specs;
for (const std::string& r : reviewers) specs.emplace_back(personaFile, r);
const std::vector<Json> res = RunGate(Plan::PersonaGate, specs);
std::ranges::copy(res, std::back_inserter(results));
Checkpoint(Plan::PersonaGate);
std::vector<std::string> notReturned;
for (const Json& r : res) {
if (Dyn::Truthy(r.at("coverageMissing"))) notReturned.push_back(Dyn::Str(r.at("lens")));
}
if (abort) {
stoppedAt = std::format("{} - ABORTED ({})", Plan::PersonaGate, *abort);
incomplete.push_back({{"gate", Plan::PersonaGate}, {"reason", *abort}});
} else if (!notReturned.empty()) {
stoppedAt = std::format("{} - INCOMPLETE ({} not returned)", Plan::PersonaGate, Dyn::Join(notReturned, ", "));
incomplete.push_back({{"gate", Plan::PersonaGate}, {"notReturned", Array(notReturned)}});
} else if (std::ranges::any_of(res, [](const Json& r) { return r.at("verdict") == "FIX FIRST"; })) {
stoppedAt = Plan::PersonaGate;
} else {
personaRan = true;
}
}
const bool anyFailed = std::ranges::any_of(results, [](const Json& r) { return r.at("verdict") == "FIX FIRST"; });
if (x > 1 && !a.dryRun && !testMode) {
// The file after this run's store: a --no-reuse run counts its own
@ -1814,16 +1995,19 @@ namespace Crafter::Review::Run {
owedReads += o.at("reads").get<std::int64_t>();
}
overall = std::format("CONFIRMING - NOT SEND-READY (gates clean; {} code unit(s) owe {} clean read(s) to reach {}: run the same line again)", owedUnits, owedReads, x);
} else {
// Gates 1-3 clean = push-ready (Gate 4 is the owner).
} else if (personaRan || a.mode == "commits") {
// commits: no persona gate - the gates clean = push-ready (the
// owner is the last gate).
overall = "SEND-READY";
} else {
overall = "GATES CLEAN - NOT SEND-READY (no reviewers passed, Gate 4 did not run)";
}
return Finish(overall, stoppedAt, incomplete);
}
std::int32_t Run::Finish(std::string overall, const Json& stoppedAt, Json incomplete) {
if (broker) broker->Close();
if (Artifact::ArtifactHash("commits", a.artifact) != hash) {
if (Artifact::ArtifactHash(a.mode, a.artifact) != hash) {
overall = "INCOMPLETE - NOT SEND-READY (artifact changed during the run)";
incomplete.push_back({{"gate", "end"}, {"reason", "artifact hash changed during the run"}});
}
@ -1924,13 +2108,29 @@ namespace Crafter::Review::Run {
const Json& db = Dyn::Get(s, "dbUrl");
stackList.push_back({{"id", s.at("id")}, {"appUrl", s.at("appUrl")}, {"dbUrl", Dyn::Truthy(db) ? Json(PgWire::Redact(Dyn::Str(db))) : Json()}, {"login", Dyn::Get(s, "login")}});
}
const std::vector<std::string> modeLenses = Plan::ModeLenses(a.mode);
Json lensesOff = Json::array();
for (const std::string& lf : off) lensesOff.push_back(Plan::LensName(lf));
for (const std::string& lf : off) {
if (std::ranges::contains(modeLenses, lf)) lensesOff.push_back(Plan::LensName(lf));
}
Json environmentNames = Json::object();
for (auto it = environments.begin(); it != environments.end(); ++it) environmentNames[Plan::LensName(it.key())] = it.value();
for (auto it = environments.begin(); it != environments.end(); ++it) {
if (std::ranges::contains(modeLenses, it.key())) environmentNames[Plan::LensName(it.key())] = it.value();
}
Json refs;
if (Kernel()) {
std::int64_t fetched = 0;
Json failed = Json::array();
for (auto it = refIndex.begin(); it != refIndex.end(); ++it) {
const Json& status = Dyn::Get(it.value(), "status");
if (status == "cached" || status == "fetched") ++fetched;
if (status == "failed") failed.push_back(it.key());
}
refs = {{"index", std::format("gate-state/refs-{}.json", h8)}, {"fetched", fetched}, {"failed", failed}};
}
Json dump = {
{"driver", DriverLine()},
{"mode", "commits"},
{"mode", a.mode},
{"artifact", Dyn::RelPath(a.artifact, repo.string())},
{"hash", hash},
{"date", a.date},
@ -1950,6 +2150,7 @@ namespace Crafter::Review::Run {
{"results", results},
{"processes", recs},
{"usage", {{"run", State::UsageTotals(recs)}, {"perLens", perLensUsage}, {"prefixHitRate", firsts ? Json(Dyn::Round(static_cast<double>(hits) / static_cast<double>(firsts), 2)) : Json()}}},
{"refs", refs},
{"estimate", runJobs.empty() ? Json() : Plan::Estimate(runJobs, static_cast<std::int64_t>(Dyn::Length(runLayer)), lensLayersBytes)},
{"wallMin", (Platform::Now() - t0) / 60},
{"scratch", scratch.string()},
@ -2166,14 +2367,18 @@ namespace Crafter::Review::Run {
}
sets[key] = {{"lens", rec.at("lensFile")}, {"unit", rec.at("unit")}, {"unitSha", Dyn::Get(rec, "unitSha")}, {"hash", hash}, {"run", Basename(dumpPath)}, {"date", a.date},
{"coverage", cov ? *cov : Json()}, {"findings", citing}, {"notes", Normalize::Dedupe(notes)}};
if (Dyn::Truthy(Dyn::Get(rec, "persona"))) sets[key]["persona"] = rec.at("persona");
++n;
}
// One clean read per round: the count comes from the FILE at write
// time, so a read another driver stored is kept; a second store of the
// same run, or the relaunch of an unfinished run, adds nothing.
const Json read = {{"hash", h8}, {"round", round}, {"run", Basename(dumpPath)}};
// One clean read per round, not for a reply: the count comes from the
// FILE at write time, so a read another driver stored is kept; a
// second store of the same run, or the relaunch of an unfinished run,
// adds nothing. The persona pass is recorded once the persona gate
// passed.
const Json read = a.mode == "reply" ? Json() : Json({{"hash", h8}, {"round", round}, {"run", Basename(dumpPath)}});
const Json persona = personaRan ? Json({{"hash", hash}, {"reviewers", Array(reviewers)}, {"date", a.date}}) : Json();
const std::vector<std::string> drops(dirty.begin(), dirty.end());
Json merged = State::SaveState(statePath, sets, drops, read);
Json merged = State::SaveState(statePath, sets, drops, read, persona);
// A --no-reuse run keeps its empty view for the reuse decisions; its
// reads are counted from the file.
if (!a.noReuse) state = merged;

68
implementations/Crafter.Review-Schema.cpp
View file

@ -32,7 +32,7 @@ namespace Crafter::Review::Schema {
}
}
Json Finding() {
Json Finding(bool kernel) {
Json contests = {
{"type", "object"},
{"additionalProperties", false},
@ -47,8 +47,8 @@ namespace Crafter::Review::Schema {
{"additionalProperties", false},
{"required", {"finding", "evidence", "proposedAction", "items"}},
{"properties", {
{"finding", String("the defect - by your lens's charter question (objective wrongness)")},
{"evidence", String("file:line, a verbatim quote, the steps taken in the application and what was observed (a screenshot or snapshot path) - no vibes")},
{"finding", String(kernel ? "the defect - by your lens's charter question (objective wrongness, or a predicted reviewer comment)" : "the defect - by your lens's charter question (objective wrongness)")},
{"evidence", String(kernel ? "file:line, msgid, or verbatim quote - no vibes" : "file:line, a verbatim quote, the steps taken in the application and what was observed (a screenshot or snapshot path) - no vibes")},
{"proposedAction", String()},
{"items", StringList("the INVENTORY unit ids this finding concerns, copied exactly; a finding that spans units lists them all")},
{"dischargedBy", String("id of a disposition listed under DISPOSITIONS IN FORCE that discharges this finding (you verified its grounds); omit when none applies")},
@ -57,65 +57,75 @@ namespace Crafter::Review::Schema {
};
}
Json Note() {
Json Note(bool kernel) {
Json properties = {
{"kind", {{"type", "string"}, {"enum", kernel ? Strings(KernelNoteKinds) : Strings(NoteKinds)}}},
{"note", String()},
{"evidence", String()},
{"action", String(kernel ? "what the orchestrator does with it (bank / ledger / confirm at send / nothing)" : "what the orchestrator does with it (bank / record in the manifest / check at push time / nothing)")},
{"id", String(kernel ? "disposition-review and refutation-verified only: the disposition id" : "refutation-verified only: the disposition id")},
};
if (kernel) properties["wouldComment"] = {{"type", "boolean"}, {"description", "disposition-review only: would this reviewer comment on the discharged item as the artifact stands?"}};
properties["items"] = StringList("INVENTORY unit ids this note concerns");
return {
{"type", "object"},
{"additionalProperties", false},
{"required", {"kind", "note", "evidence", "action"}},
{"properties", {
{"kind", {{"type", "string"}, {"enum", Strings(NoteKinds)}}},
{"note", String()},
{"evidence", String()},
{"action", String("what the orchestrator does with it (bank / record in the manifest / check at push time / nothing)")},
{"id", String("refutation-verified only: the disposition id")},
{"items", StringList("INVENTORY unit ids this note concerns")},
}},
{"properties", properties},
};
}
Json Coverage() {
Json Coverage(bool kernel) {
return {
{"type", "object"},
{"additionalProperties", false},
{"required", {"item", "verdict", "basis"}},
{"properties", {
{"item", String("an INVENTORY unit id, verbatim")},
{"verdict", {{"type", "string"}, {"enum", Strings(CoverageVerdicts)}}},
{"basis", String("what this verdict rests on: the tree ref, the command run, the steps taken in the application and the evidence file - or the literal string \"no record\"")},
{"verdict", {{"type", "string"}, {"enum", kernel ? Strings(KernelCoverageVerdicts) : Strings(CoverageVerdicts)}}},
{"basis", String(kernel ? "what this verdict rests on: the source fetched / tree ref / units crossed / your own message-id - or the literal string \"no record\""
: "what this verdict rests on: the tree ref, the command run, the steps taken in the application and the evidence file - or the literal string \"no record\"")},
}},
};
}
Json BlockSchema() {
Json BlockSchema(bool kernel) {
return {
{"type", "object"},
{"additionalProperties", false},
{"required", {"verdict", "findings", "notes", "coverage"}},
{"properties", {
{"verdict", Verdict("FIX FIRST iff any finding is undischarged - the pipeline derives the same and fails closed on disagreement")},
{"findings", {{"type", "array"}, {"items", Finding()}}},
{"notes", {{"type", "array"}, {"items", Note()}}},
{"coverage", {{"type", "array"}, {"items", Coverage()}, {"minItems", 1}, {"maxItems", 1}, {"description", "exactly one entry: the unit this pass covers"}}},
{"findings", {{"type", "array"}, {"items", Finding(kernel)}}},
{"notes", {{"type", "array"}, {"items", Note(kernel)}}},
{"coverage", {{"type", "array"}, {"items", Coverage(kernel)}, {"minItems", 1}, {"maxItems", 1}, {"description", "exactly one entry: the unit this pass covers"}}},
}},
};
}
Json WholeSchema() {
Json WholeSchema(bool kernel) {
Json properties = {
{"verdict", Verdict({})},
{"findings", {{"type", "array"}, {"items", Finding(kernel)}}},
{"notes", {{"type", "array"}, {"items", Note(kernel)}}},
{"coverage", {{"type", "array"}, {"items", Coverage(kernel)}, {"description", "one entry per unit you were asked to cover"}}},
{"summary", String(kernel ? "5-10 sentences: the artifact as you judged it and the constraints the block passes must respect; a persona adds its calibration digest (pet issues, phrasing, imperatives found in its own mail)"
: "5-10 sentences: the range as you judged it and the constraints the block passes must respect (the functional lens: how to reach each touched screen, the login used, what the block passes must not touch)")},
};
if (kernel) {
properties["predictedReview"] = String("persona lens only: the review this person would send, in their voice");
properties["notMine"] = StringList("persona lens only: unit ids that are not your tree, file or decision (each must carry a basis in coverage)");
}
return {
{"type", "object"},
{"additionalProperties", false},
{"required", {"verdict", "findings", "notes", "coverage", "summary"}},
{"properties", {
{"verdict", Verdict({})},
{"findings", {{"type", "array"}, {"items", Finding()}}},
{"notes", {{"type", "array"}, {"items", Note()}}},
{"coverage", {{"type", "array"}, {"items", Coverage()}, {"description", "one entry per unit you were asked to cover"}}},
{"summary", String("5-10 sentences: the range as you judged it and the constraints the block passes must respect (the functional lens: how to reach each touched screen, the login used, what the block passes must not touch)")},
}},
{"properties", properties},
};
}
std::string SchemaJson(std::string_view kind) {
return DumpCompactSorted(kind == "block" ? BlockSchema() : WholeSchema());
std::string SchemaJson(std::string_view kind, std::string_view mode) {
const bool kernel = mode == "reply" || mode == "series";
return DumpCompactSorted(kind == "block" ? BlockSchema(kernel) : WholeSchema(kernel));
}
}

20
implementations/Crafter.Review-State.cpp
View file

@ -72,7 +72,7 @@ namespace Crafter::Review::State {
return EmptyState();
}
Json MergeState(const Json& disk, const Json& sets, std::span<const std::string> drops, const Json& read) {
Json MergeState(const Json& disk, const Json& sets, std::span<const std::string> drops, const Json& read, const Json& persona) {
const bool ok = disk.is_object() && Dyn::Get(disk, "entries").is_object();
Json state = {
{"version", 1},
@ -108,6 +108,7 @@ namespace Crafter::Review::State {
}
entries[it.key()] = std::move(entry);
}
if (!persona.is_null()) state["personaPass"] = persona;
return state;
}
@ -121,10 +122,10 @@ namespace Crafter::Review::State {
fs::rename(temp, path);
}
Json SaveState(const fs::path& path, const Json& sets, std::span<const std::string> drops, const Json& read) {
Json SaveState(const fs::path& path, const Json& sets, std::span<const std::string> drops, const Json& read, const Json& persona) {
// Without the lock the write is still made; a failed write throws.
const Platform::FileLock lock(std::format("{}.lock", path.string()));
Json merged = MergeState(LoadState(path), sets, drops, read);
Json merged = MergeState(LoadState(path), sets, drops, read, persona);
Platform::WriteAtomic(path, DumpFile(merged), 0666);
return merged;
}
@ -249,9 +250,16 @@ namespace Crafter::Review::State {
}
const Json& coverage = Dyn::Or(Dyn::Get(dump, "coverage"), Dyn::EmptyDict());
if (Dyn::Truthy(coverage)) {
lines.push_back(std::format("matrix: {} units x {} lenses; holes {}, not reviewed by any lens that runs (messages, diffstats, documentation, the units of a lens the config turns off) {}, units with a defect verdict {}",
Dyn::Str(Dyn::At(coverage, "units")), Len(Dyn::At(coverage, "lenses")), Len(Dyn::At(coverage, "holes")),
Len(Dyn::Or(Dyn::Get(coverage, "notReviewed"), Dyn::EmptyList())), Len(Dyn::At(coverage, "defects"))));
const Json& mode = Dyn::Get(dump, "mode");
if (mode == "reply" || mode == "series") {
lines.push_back(std::format("matrix: {} units x {} lenses; holes {}, persona holes {}, thin {}, modelled {}, units with a defect verdict {}", Dyn::Str(Dyn::At(coverage, "units")),
Len(Dyn::At(coverage, "lenses")), Len(Dyn::At(coverage, "holes")), Len(Dyn::Or(Dyn::Get(coverage, "personaHoles"), Dyn::EmptyList())),
Len(Dyn::Or(Dyn::Get(coverage, "thin"), Dyn::EmptyList())), Len(Dyn::Or(Dyn::Get(coverage, "modelled"), Dyn::EmptyList())), Len(Dyn::At(coverage, "defects"))));
} else {
lines.push_back(std::format("matrix: {} units x {} lenses; holes {}, not reviewed by any lens that runs (messages, diffstats, documentation, the units of a lens the config turns off) {}, units with a defect verdict {}",
Dyn::Str(Dyn::At(coverage, "units")), Len(Dyn::At(coverage, "lenses")), Len(Dyn::At(coverage, "holes")),
Len(Dyn::Or(Dyn::Get(coverage, "notReviewed"), Dyn::EmptyList())), Len(Dyn::At(coverage, "defects"))));
}
}
const Json& usageAll = Dyn::Has(dump, "usage") ? Dyn::At(dump, "usage") : Dyn::EmptyDict();
const Json& usage = Dyn::Or(Dyn::Get(usageAll, "run"), Dyn::EmptyDict());