test(run): full runs end to end on a claude stand-in

The Runs test drives this binary as crafter-review on a scratch repository,
with a /bin/sh claude that answers every pass from its system file and
prompt: a dry run; a run SIGKILLed in its last gate (every pass goes with
it through its supervisor); the next run resuming the first gate from the
checkpoint, salvaging the killed run's outputs and buying only the pass that
never answered; a run with nothing left to buy; a FIX FIRST with
--no-reuse --no-salvage; a stop by the stop file and by SIGTERM, each
leaving no process behind; a stale stop file dropped by the run that claims
its dir; a stop between gates.

Two fixes it led to:
- A stop that lands while a gate's pool is set up is no longer lost: the
  stop is checked under the same lock that publishes the pool.
- A run never takes the number of a dump on record. Past a killed run that
  left no dump, a later run that bought nothing could have its number
  taken again and its dump written over.

Drivers gains the --no-reuse store check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Jorijn van der Graaf 2026-10-31 09:59:12 +01:00
commit 952eb13f5c
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
4 changed files with 455 additions and 11 deletions

View file

@ -436,9 +436,15 @@ namespace Crafter::Review::Run {
}
return false;
};
// Nor the number of a dump on record: past a killed run's gap, a later
// run that bought nothing (no outputs in its dir) would otherwise have
// its number taken again and its dump written over.
auto dumped = [&](std::int64_t n) {
return fs::exists(gateDir / std::format("commits-{}-run-{}.json", h8, n)) || fs::exists(gateDir / std::format("commits-{}-run-{}-test.json", h8, n));
};
while (!a.dryRun) {
const fs::path candidate = scratchRoot / std::format("{}-run-{}", h8, runN);
if (!hasOutputs(candidate) && SeatPool::ClaimScratch(candidate)) break;
if (!dumped(runN) && !hasOutputs(candidate) && SeatPool::ClaimScratch(candidate)) break;
++runN;
}
scratch = scratchRoot / std::format("{}-run-{}{}", h8, runN, a.dryRun ? "-dry" : "");
@ -1440,11 +1446,6 @@ namespace Crafter::Review::Run {
};
const bool warm = !a.dryRun && (!testMode || a.warm);
std::unique_ptr<Pool::Admission> admission = Admission(lensFiles);
std::optional<std::string> before;
{
const std::lock_guard guard(shared);
before = stopped;
}
Pool::Scheduler scheduler({.runner = runner.get(),
.concurrency = a.concurrency,
.log = log,
@ -1455,10 +1456,12 @@ namespace Crafter::Review::Run {
.apiConfig = apiCfg,
.ledger = ledger.get(),
.admission = admission.get(),
.board = board.get(),
.abort = before});
.board = board.get()});
{
// A stop before this gate, or while its pool was set up, stays in
// force: the pool starts aborted, so the next gate never runs.
const std::lock_guard guard(shared);
if (stopped) scheduler.Abort(*stopped);
sched = &scheduler;
}
scheduler.Run(initial, [this](const Json& job, std::int64_t attempt) { return ExecJob(job, attempt); }, onDone);