Jorijn van der Graaf
d5e54ffeb3
The config defines the modes (modes.<name>): the kind of artifact the mode reviews - reply, series or commits, the three shipped partitions; a kind is a mode of its own name - the lenses it runs, in its kind's gate order, and the effort it runs at, else the kind's default (max for a reply or a series, xhigh for a commit range). `run --mode` names any of them, the dump records mode and kind, and everything that decided by the mode's name - the partition and hash, the prompts, the schemas, the disposition and reuse-key rules, what the CLI requires - decides by the kind. Every lens stays off until a mode lists it; lenses.<name>.run is retired with a warning that says so, and a lens of another kind, a name that is no lens, a lens listed twice and an unknown effort are config problems. The defaults run no lens; the template ships the three kinds as modes with every lens of their kind at the kind's effort, to trim (the owner's ruling of 2026-11-01), and init says so as it writes it - a kept config is the repository's own, and only one that lists no lens gets a step. The readers of the retired key follow: the CI smoke config, the shipped skill, the procedure (its description of the config as well as its mechanics), the manifest template, the run's messages and the comments that named the three modes. The run lock, the run numbers and the dump names are the kind's, so one full run per hash holds across the modes of a kind and their rounds count on together; the dump says which mode ran. The database broker starts for a lens the mode runs, as every other reader of the lenses off reads them. A named mode runs through the CLI in the kernel-modes test.
321 lines
16 KiB
C++
321 lines
16 KiB
C++
// SPDX-License-Identifier: AGPL-3.0-only
|
|
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
|
|
|
|
export module Crafter.Review:Run;
|
|
import std;
|
|
import :Json;
|
|
import :Plan;
|
|
import :Pool;
|
|
import :SeatPool;
|
|
import :Memory;
|
|
import :PgWire;
|
|
import :State;
|
|
import :Platform;
|
|
import :Refs;
|
|
|
|
// Run: the driver for a commit range about to be pushed
|
|
// (process/prepush.md), and for the kernel modes - a mailing-list reply or
|
|
// a patch series about to be sent. One claude process per inventory unit
|
|
// per lens, whole passes where a lens has one (each gating its block
|
|
// passes), fail-closed coverage per unit, the matrix, per-unit reuse across
|
|
// hashes, a seat pool.
|
|
//
|
|
// Gates (commits): 1 repo-better-solution (approach) -> 2 repo-functional
|
|
// (the range exercised in the running application) -> 3 repo-bug-prover
|
|
// and repo-security -> 4 the owner. SEND-READY = push-ready. The kernel
|
|
// modes run their own gates (Plan::Gates), then the persona gate for every
|
|
// reviewer given; gates clean without a persona pass is GATES CLEAN, not
|
|
// SEND-READY. Exit codes: 0 SEND-READY, 1 FIX FIRST, 2 INCOMPLETE / PARTIAL
|
|
// / GATES CLEAN / CONFIRMING (gates clean, code units still owe clean reads
|
|
// - run the same line again), 3 preflight.
|
|
//
|
|
// A Run is built empty and filled field by field, as the tests fill one, so
|
|
// its attributes are public; Main builds and runs the real one.
|
|
export namespace Crafter::Review::Run {
|
|
using Log = std::function<void(const std::string&)>;
|
|
|
|
// `crafter-review run`'s options: the dump records them under `args`, in
|
|
// this order.
|
|
struct Arguments {
|
|
std::optional<std::string> config;
|
|
// The mode's name (the config's modes.<name>) and its artifact
|
|
// kind (reply, series, commits): the kind decides the partition, the
|
|
// prompts, the schemas and the rules; the mode the lenses and effort.
|
|
std::string mode = "commits";
|
|
std::string kind = "commits";
|
|
// The journal subsystem: required in the kernel modes (its ledger,
|
|
// captures and threads are the lenses' evidence); optional in
|
|
// commits mode.
|
|
std::optional<std::string> subsystem;
|
|
std::string artifact;
|
|
std::optional<std::string> hash;
|
|
// The reviewers a persona reads as, comma-separated (the kernel
|
|
// modes).
|
|
std::string reviewers;
|
|
std::optional<std::string> dispositions;
|
|
std::string base;
|
|
std::string date;
|
|
std::optional<std::string> trees;
|
|
std::string contextPaths;
|
|
std::optional<std::string> only;
|
|
std::optional<std::string> without;
|
|
// --units, split on commas.
|
|
std::optional<std::set<std::string>> units;
|
|
bool warm = false;
|
|
// A cap on the passes in flight on this machine, every driver's
|
|
// counted; none by default (2026-10-31, Jorijn: "there is no need to
|
|
// do a global cap anymore, the cap should be 128 per available
|
|
// account"): each account's own cap is the limit.
|
|
std::optional<std::int64_t> concurrency;
|
|
std::string effort;
|
|
std::int64_t reads = Plan::Reads;
|
|
bool noReuse = false;
|
|
bool noSalvage = false;
|
|
// pruned: a persona's block passes skip the units its whole pass
|
|
// marked not-mine; all: every unit.
|
|
std::string personaBlocks = "pruned";
|
|
bool dryRun = false;
|
|
std::optional<std::string> claudeBin;
|
|
std::string models;
|
|
std::optional<std::string> scratch;
|
|
std::optional<std::string> seats;
|
|
std::optional<std::string> envFile;
|
|
bool noProbe = false;
|
|
std::optional<double> apiBudget;
|
|
std::string harness = "claude";
|
|
std::optional<std::string> variant;
|
|
std::optional<std::string> opencodeBin;
|
|
};
|
|
// vars(a) without `cmd`, the units sorted.
|
|
Json ArgsJson(const Arguments& a);
|
|
|
|
// die(): "preflight: <message>" on stderr, exit 3.
|
|
class Preflight : public std::runtime_error {
|
|
public:
|
|
using std::runtime_error::runtime_error;
|
|
};
|
|
|
|
// (ok, detail): does the application under test answer at <url><path>?
|
|
// Below 500 is ok.
|
|
std::pair<bool, std::string> AppReady(std::string_view url, std::string_view path = "/", double timeout = 10);
|
|
|
|
// The machine's memory as SlotsPlan reads it (the tests give their own).
|
|
struct Machine {
|
|
std::function<std::optional<std::int64_t>()> available;
|
|
std::function<std::int64_t()> reserve;
|
|
};
|
|
// `slots`: how many stacks to bring up - as many as free memory holds,
|
|
// and no more than the range's browser passes can use at once - and the
|
|
// commands for the ones not up yet. (want, commands).
|
|
std::pair<std::int64_t, std::vector<std::string>> SlotsPlan(const Json& cfg, const std::filesystem::path& repo, const std::optional<std::filesystem::path>& artifact = std::nullopt,
|
|
const std::optional<std::filesystem::path>& scratchRoot = std::nullopt, const Log& log = {}, const Machine& machine = {});
|
|
|
|
// ({version: path} of the migrations at the range HEAD, the range's added
|
|
// or changed migrations, {path: fingerprint at HEAD} of those).
|
|
using MigrationsHead = std::tuple<std::map<std::string, std::string>, std::vector<std::string>, std::map<std::string, std::string>>;
|
|
using QueryResult = std::pair<std::vector<std::string>, std::vector<std::vector<std::optional<std::string>>>>;
|
|
|
|
class Run {
|
|
public:
|
|
Run();
|
|
Run(Arguments a, Json cfg);
|
|
~Run();
|
|
Run(const Run&) = delete;
|
|
Run& operator=(const Run&) = delete;
|
|
|
|
// The whole run: the gates, the dump, the summary; the exit code.
|
|
std::int32_t Execute();
|
|
// A signal or `crafter-review stop`: every running pass killed, the
|
|
// pool aborted, and the stop stays in force for the gates after.
|
|
void Stop(const std::string& why, std::int32_t signal = 0);
|
|
// The file `crafter-review stop` writes, once the scratch dir is
|
|
// known.
|
|
std::optional<std::filesystem::path> StopFile() const;
|
|
// The handler a signal reaches: false before gate 0 set it up (the
|
|
// signal then ends the process).
|
|
bool Ready() const;
|
|
|
|
// ---- gate 0 and its preflights ---------------------------------------
|
|
void Gate0();
|
|
std::map<std::string, std::string> AppLenses(std::span<const std::string> lensFiles) const;
|
|
std::map<std::string, std::string> DbLenses(std::span<const std::string> lensFiles) const;
|
|
void InstrumentPreflight(std::span<const std::string> lensFiles);
|
|
static std::optional<std::string> MigrationVersion(std::string_view file);
|
|
MigrationsHead MigrationsAtHead();
|
|
// (problem, note): the migrations at the range HEAD against the ones
|
|
// the database applied.
|
|
std::pair<std::optional<std::string>, std::optional<std::string>> MigrationsDrift(const std::string& url);
|
|
std::unique_ptr<Pool::Stacks> LoadStacks();
|
|
std::string StacksLine() const;
|
|
std::vector<Json> DbStacks(std::span<const std::string> lensFiles) const;
|
|
void StackPreflight(std::span<const std::string> lensFiles);
|
|
std::optional<std::string> LoginMissing(const Json& stack) const;
|
|
std::optional<std::string> TreeDrift();
|
|
|
|
// ---- prompts ---------------------------------------------------------------
|
|
// (appended system prompt file, stdin user message) of a pass.
|
|
std::pair<std::filesystem::path, std::string> PromptFor(const Json& job);
|
|
std::optional<std::string> CacheKey(Json& job);
|
|
bool WarmProbe(const Json& job) const;
|
|
|
|
// ---- reuse -----------------------------------------------------------------
|
|
const Plan::CommitCode& RangeCode();
|
|
std::optional<Json> ReuseWhole(const std::string& name, Json& job);
|
|
std::optional<std::string> ReuseKey(const Json& job, const std::optional<std::string>& environment = std::nullopt);
|
|
std::vector<std::string> OtherKeys(const Json& job, const std::optional<std::string>& key);
|
|
std::optional<std::pair<Json, Json>> Salvage(const Json& job);
|
|
Json DirEnvironments(const std::filesystem::path& dir);
|
|
static std::int64_t DirRound(const std::filesystem::path& dir);
|
|
std::int64_t ReadsNeeded(const Json& job) const;
|
|
|
|
// ---- one process, a gate -------------------------------------------------
|
|
std::pair<Json, Json> ExecJob(const Json& job, std::int64_t attempt);
|
|
Json PartFrom(const Json& job, const Json& rec, Json raw);
|
|
static Json DryPart(const Json& job);
|
|
std::vector<Json> RunGate(std::string_view title, std::vector<std::string> lensFiles);
|
|
// (lens file, persona) per lens instance: a persona's instance reads
|
|
// as that reviewer.
|
|
std::vector<Json> RunGate(std::string_view title, std::vector<std::pair<std::string, std::string>> specs);
|
|
// The kernel modes (reply, series).
|
|
bool Kernel() const;
|
|
std::unique_ptr<Pool::Admission> Admission(std::span<const std::string> lensFiles);
|
|
std::optional<Json> Resumable(const std::string& name, const std::string& lensFile, std::string_view title);
|
|
void Checkpoint(std::string_view title);
|
|
std::vector<Json> PrepareBlocks(const std::string& name, std::vector<Json> blocks, std::map<std::string, std::vector<Json>>& parts);
|
|
|
|
// ---- the end ---------------------------------------------------------------
|
|
std::int32_t Finish(std::string overall, const Json& stoppedAt, Json incomplete);
|
|
Json BuildDump(std::string_view overall, const Json& stoppedAt, const Json& incomplete);
|
|
Json DryReport();
|
|
bool Storable(const Json& rec) const;
|
|
Json StoreState();
|
|
|
|
void Message(const std::string& message);
|
|
|
|
// ---- the run's state -----------------------------------------------------
|
|
Arguments a;
|
|
Json cfg = Json::object();
|
|
std::vector<std::string> off;
|
|
// {lens file: environment}, in the config's lens order: the dump, the
|
|
// resume file and the scratch dir's marker write it in that order.
|
|
Json environments = Json::object();
|
|
// The working directory: the config, the env file and the slots are
|
|
// its, the lens processes run in it, and every path logged is relative
|
|
// to it.
|
|
std::filesystem::path repo;
|
|
// The repository under review, from the artifact's freeze record (the
|
|
// working directory when it has none): the prompt names it, and the
|
|
// calibration is kept under it.
|
|
std::filesystem::path reviewed;
|
|
std::vector<std::string> logs;
|
|
double t0 = 0;
|
|
std::optional<std::filesystem::path> logFile;
|
|
std::vector<Json> records;
|
|
std::vector<Json> results;
|
|
std::map<std::string, std::int64_t> lensLayersBytes;
|
|
std::vector<Json> jobsRun;
|
|
std::map<std::string, std::string> summaries;
|
|
// The kernel modes: the reviewers with a persona, their calibration
|
|
// maps, the references the artifact cites and their digest, the
|
|
// anchor unit's text, and whether the persona gate passed.
|
|
std::vector<std::string> reviewers;
|
|
std::map<std::string, std::string> corpora;
|
|
Refs::Table refUrls;
|
|
Json refIndex = Json::object();
|
|
std::string refDigest;
|
|
std::string anchorText;
|
|
bool personaRan = false;
|
|
std::optional<std::string> abort;
|
|
Pool::Scheduler* sched = nullptr;
|
|
std::optional<Plan::CommitCode> commitCode;
|
|
std::int64_t x = 1;
|
|
std::int64_t round = 1;
|
|
bool newRound = false;
|
|
Json owed = Json::object();
|
|
std::map<std::string, Json> wholeParts;
|
|
std::optional<std::string> dbChecked;
|
|
std::optional<bool> dbBypass;
|
|
std::optional<std::string> dbRole;
|
|
std::unique_ptr<PgWire::Broker> broker;
|
|
std::unique_ptr<Pool::Stacks> stacks;
|
|
bool slotMode = false;
|
|
std::vector<std::string> slotProblems;
|
|
std::unique_ptr<SeatPool::Board> board;
|
|
std::unique_ptr<State::RunLock> runLock;
|
|
std::unique_ptr<Platform::LifetimeLock> lifetime;
|
|
std::vector<std::string> trees;
|
|
std::vector<std::string> contextPaths;
|
|
std::string hash;
|
|
std::string h8;
|
|
std::string head;
|
|
std::filesystem::path gateDir;
|
|
Json units = Json::array();
|
|
std::vector<std::string> lines;
|
|
bool testMode = false;
|
|
std::filesystem::path scratch;
|
|
std::filesystem::path dumpPath;
|
|
std::vector<std::string> notReviewed;
|
|
std::vector<Json> dispEntries;
|
|
std::vector<Json> dispRows;
|
|
std::vector<Json> inForce;
|
|
std::filesystem::path statePath;
|
|
Json state = State::EmptyState();
|
|
std::vector<std::filesystem::path> salvageDirs;
|
|
std::filesystem::path resumePath;
|
|
Json resume;
|
|
std::map<std::string, std::string> charters;
|
|
std::map<std::string, std::string> charterSha;
|
|
std::string systemText;
|
|
// {sha: path}, in the order first written.
|
|
std::vector<std::pair<std::string, std::filesystem::path>> systemPaths;
|
|
// {lens: the shas of its system files}, lenses in the order first met.
|
|
std::vector<std::pair<std::string, std::set<std::string>>> systemByLens;
|
|
std::string runLayer;
|
|
std::optional<std::string> loginFixture;
|
|
std::string appBlock;
|
|
std::map<std::string, std::string> schemas;
|
|
std::unique_ptr<Memory::Calibration> calibration;
|
|
std::unique_ptr<Memory::Sampler> sampler;
|
|
std::unique_ptr<Pool::Runner> runner;
|
|
Json apiCfg = Json::object();
|
|
std::unique_ptr<SeatPool::Ledger> ledger;
|
|
std::vector<Json> seats;
|
|
std::map<std::filesystem::path, Json> envMarkers;
|
|
std::set<std::pair<std::string, std::string>> envSkips;
|
|
std::map<std::pair<std::string, std::string>, MigrationsHead> migCache;
|
|
// The log every line goes to (the tests put their own here).
|
|
Log log;
|
|
|
|
// ---- the seams a test replaces ---------------------------------------------
|
|
// Each starts as the real one; the methods call through these.
|
|
std::function<std::pair<bool, std::string>(std::string_view url, std::string_view path)> appReady;
|
|
std::function<std::pair<bool, std::string>(std::string_view url)> probe;
|
|
std::function<QueryResult(std::string_view url, std::string_view sql)> query;
|
|
std::function<std::optional<bool>(std::string_view url)> bypassesRls;
|
|
std::function<MigrationsHead()> migrationsAtHead;
|
|
std::function<std::pair<std::optional<std::string>, std::optional<std::string>>(const std::string& url)> migrationsDrift;
|
|
std::function<std::optional<std::string>()> treeDrift;
|
|
std::function<std::pair<std::filesystem::path, std::string>(const Json& job)> promptFor;
|
|
std::function<std::pair<Json, Json>(const Json& job, const std::string& prompt, const std::filesystem::path& systemPath, std::string_view schema, std::int64_t maxTurns, std::int64_t timeoutMin,
|
|
bool persist)>
|
|
runPass;
|
|
std::function<std::optional<Json>(const std::string& name, const std::string& lensFile, std::string_view title)> resumable;
|
|
std::function<void(std::span<const std::string> lensFiles)> instrumentPreflight;
|
|
|
|
private:
|
|
// What exec_job's threads share with the pool's: the system files,
|
|
// the lens layers, the jobs run.
|
|
mutable std::recursive_mutex shared;
|
|
mutable std::mutex logLock;
|
|
std::optional<std::string> stopped;
|
|
std::atomic<bool> ready = false;
|
|
std::optional<std::filesystem::path> stopFile;
|
|
};
|
|
|
|
// The `run` command: argv parsed by the CLI; the exit code.
|
|
std::int32_t Main(Arguments a, const Json& cfg);
|
|
// What SIGINT, SIGTERM and SIGHUP do (Process::InstallSignalThread):
|
|
// stop the run once gate 0 is through; before that, and for every other
|
|
// command, the process ends, its descendants with it.
|
|
void OnSignal(std::int32_t signal);
|
|
}
|