Crafter.Build/interfaces/Crafter.Build-External.cppm

126 lines
6.1 KiB
Text
Raw Normal View History

2026-07-23 01:24:42 +02:00
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
2026-04-27 07:04:42 +02:00
export module Crafter.Build:External;
import std;
namespace fs = std::filesystem;
export namespace Crafter {
2026-04-30 02:20:19 +02:00
struct Configuration;
2026-04-27 07:04:42 +02:00
struct GitSource {
std::string url;
std::string branch;
std::string commit;
};
enum class ExternalBuilder {
None,
CMake,
};
struct ExternalDependency {
std::string name;
GitSource source;
ExternalBuilder builder = ExternalBuilder::None;
std::vector<std::string> options;
std::vector<fs::path> includeDirs;
2026-05-06 03:59:19 +02:00
// Extra library search paths (each becomes a -L flag), interpreted
// relative to the CMake build dir. Currently only honoured for
// ExternalBuilder::CMake. The CMake builder always adds its
// top-level build dir as a -L; libDirs is for projects (msquic,
// others) whose CMakeLists set LIBRARY_OUTPUT_DIRECTORY to a subdir
// of the build tree.
std::vector<fs::path> libDirs;
2026-04-27 07:04:42 +02:00
std::vector<std::string> libs;
};
struct ExternalBuildResult {
std::string error;
std::vector<std::string> compileFlags;
std::vector<std::string> linkFlags;
fs::file_time_type latestArtifact = fs::file_time_type::min();
};
feat(lint): no-char-pointer reads the AST, retiring the interop denylist The rule was `\bchar\s*\*` over the text minus a substring denylist — argv, getenv, setenv, dlerror, c_str, .data(, reinterpret_cast, extern ". Every entry was a patch for one interop site, the list could only grow as libraries arrived, and each entry disabled the rule for the whole LINE it appeared on. It now walks declarations and asks the question the denylist was approximating: whose header dictates this spelling? A declaration with C language linkage, or one whose initialiser binds to an entity declared outside the project root, is somebody else's API and keeps its spelling. getenv, c_str and friends are exempt because of where they are declared, not because they are named here, so a new external library needs no new entry. Two bugs found while testing this, both of which had made the rule silently pass over the entire repository: clang_getCursorLanguage cannot be used to detect extern "C". Its default answer is CXLanguage_C for a plain function, variable or parameter even in a C++ translation unit, so isExternC was true almost everywhere and exempted everything. Replaced by tracking CXCursor_LinkageSpec depth during the walk, reading the extent text to tell extern "C" from extern "C++". Attributing any foreign reference in a subtree to the enclosing declaration was too broad: a function that merely touched libc++ somewhere in its body would exempt its own signature. Narrowed to initialiser contexts — a variable, field or parameter — which is where a binding to a foreign API actually occurs. Also: functions now carry their RESULT type rather than the whole function type, since the parameters arrive as their own declarations and would otherwise be reported twice. main's parameters are exempt structurally, its signature being fixed by the language rather than chosen here. Two sites keep an explicit lint-disable, both Crafter::Run taking main's argv verbatim. That is two visible, reasoned suppressions in place of a denylist that silently disabled the rule for every line mentioning one of eight tokens. ExternalCloneDir and ExternalIncludeFlags are now exposed from :External, so a source that includes an external dependency's headers can be parsed without running a build to discover where they are. BuildExternal derives its own working directory through the same function, so the two cannot drift. Crafter.Build-Shader.cpp needed this to parse at all. A file with no compile command — project.cpp, which LoadProject builds with its own flags — is not a translation unit of the build graph, so AST rules skip it the way a rule self-filters by extension. That is distinct from a file that should have parsed and did not, which stays an error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:46:32 +02:00
// Where this dependency's clone lives in the cache. A pure function of the
// declaration and target — BuildExternal derives its own working directory
// through this, so a caller that only needs to know where the headers ended
// up cannot drift from where they actually are.
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
fs::path ExternalCloneDir(const ExternalDependency& dep, std::string_view target);
feat(lint): no-char-pointer reads the AST, retiring the interop denylist The rule was `\bchar\s*\*` over the text minus a substring denylist — argv, getenv, setenv, dlerror, c_str, .data(, reinterpret_cast, extern ". Every entry was a patch for one interop site, the list could only grow as libraries arrived, and each entry disabled the rule for the whole LINE it appeared on. It now walks declarations and asks the question the denylist was approximating: whose header dictates this spelling? A declaration with C language linkage, or one whose initialiser binds to an entity declared outside the project root, is somebody else's API and keeps its spelling. getenv, c_str and friends are exempt because of where they are declared, not because they are named here, so a new external library needs no new entry. Two bugs found while testing this, both of which had made the rule silently pass over the entire repository: clang_getCursorLanguage cannot be used to detect extern "C". Its default answer is CXLanguage_C for a plain function, variable or parameter even in a C++ translation unit, so isExternC was true almost everywhere and exempted everything. Replaced by tracking CXCursor_LinkageSpec depth during the walk, reading the extent text to tell extern "C" from extern "C++". Attributing any foreign reference in a subtree to the enclosing declaration was too broad: a function that merely touched libc++ somewhere in its body would exempt its own signature. Narrowed to initialiser contexts — a variable, field or parameter — which is where a binding to a foreign API actually occurs. Also: functions now carry their RESULT type rather than the whole function type, since the parameters arrive as their own declarations and would otherwise be reported twice. main's parameters are exempt structurally, its signature being fixed by the language rather than chosen here. Two sites keep an explicit lint-disable, both Crafter::Run taking main's argv verbatim. That is two visible, reasoned suppressions in place of a denylist that silently disabled the rule for every line mentioning one of eight tokens. ExternalCloneDir and ExternalIncludeFlags are now exposed from :External, so a source that includes an external dependency's headers can be parsed without running a build to discover where they are. BuildExternal derives its own working directory through the same function, so the two cannot drift. Crafter.Build-Shader.cpp needed this to parse at all. A file with no compile command — project.cpp, which LoadProject builds with its own flags — is not a translation unit of the build graph, so AST rules skip it the way a rule self-filters by extension. That is distinct from a file that should have parsed and did not, which stays an error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 23:46:32 +02:00
// The -I flags this dependency contributes, from its declared includeDirs.
// Also pure, which is what lets the linter's AST layer parse a source that
// includes an external's headers without running a build to find out where
// they are. The paths only resolve once something has fetched the clone.
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
std::vector<std::string> ExternalIncludeFlags(const ExternalDependency& dep, std::string_view target);
ExternalBuildResult BuildExternal(const ExternalDependency& dep, std::string_view target, std::atomic<bool>& cancelled);
2026-04-30 02:20:19 +02:00
// Specification for a sibling crafter-build project to fetch and depend on.
// GitSource picks the revision: leave branch + commit empty for the
// remote's default branch HEAD (sloppy but convenient), set branch to
// track a branch tip, or set commit to pin to an immutable SHA. args are
// forwarded verbatim to the dep's CrafterBuildProject — typically you
// pass through the parent's args so target/march/debug propagate.
struct GitProjectSpec {
GitSource source;
fs::path projectFile = "project.cpp";
std::vector<std::string> args;
};
// Clones the spec's git URL into the per-project external cache (keyed by
// url+branch+commit+args+projectFile), recursively LoadProject's the
// remote's project.cpp, and returns a stable Configuration* you can drop
// into cfg.dependencies. The Configuration is owned by an internal cache
// for the lifetime of the build; identical specs share one Configuration.
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
Configuration* GitProject(const GitProjectSpec& spec);
2026-04-30 02:20:19 +02:00
// Specification for a local sibling crafter-build project (e.g. an
// example folder depending on its parent project, or a workspace where
// multiple projects live side-by-side without going through git).
struct LocalProjectSpec {
fs::path projectFile; // relative to cwd or absolute
std::vector<std::string> args; // forwarded to dep's CrafterBuildProject
};
// Same as GitProject but for a local on-disk project — no fetch, no
// cache copy. Resolves projectFile to its canonical absolute path and
// recursively LoadProject's it. Returns a stable Configuration* owned
// by the same internal cache as GitProject.
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
Configuration* LocalProject(const LocalProjectSpec& spec);
2026-05-19 00:50:06 +02:00
// Clone (or update) the given git source into the shared external
// cache and return the path to the cloned working tree. Intended for
// example projects that need large asset bundles which shouldn't
// live in the parent repo — push the returned path (or specific
// files inside it) into `cfg.assets`. Reuses the same cache root as
// ExternalDependency / GitProject. Caching key = url + branch + commit;
// identical calls reuse the existing clone. Pin `source.commit` to a
// SHA for reproducible, offline-after-first-fetch builds.
refactor: drop Windows host support crafter-build is moving to a single Flatpak distribution, and there is no Windows host in that picture: no build.cmd, no MSVC or mingw host build, no DLL plus launcher. Delete every host-Windows path instead of leaving it to rot unverified: the cmd.exe command runner and quoting, the LOCALAPPDATA cache, the MSVC and mingw-host LoadProject variants with their LIBCXX_DIR and CRAFTER_MINGW_DIR plumbing, the winsock port probe, the LoadLibrary libclang loader, the console-width and isatty shims, and the preprocessor guards around the Linux bodies. The x86_64-w64-mingw32 cross-compile target stays. It is target logic (output names, -femulated-tls, the libstdc++ link extras, copying a dependency DLL next to a mingw exe, wine as the runner) and did not depend on any of the removed code. The msvc target goes with the host that was the only thing able to use it. Three target-path warts fixed on the way: ExpectedOutput named a mingw static library <name>.lib while the archive step wrote lib<name>.a, so those libraries relinked on every run; the import lib is no longer copied next to a mingw exe, since the dependency's own bin dir is already on the link line; and llvm-ar now writes every archive, not only the LTO ones, so Debug and Release archives are made the same way. CRAFTER_API and Crafter.Build-Api.h existed only for the DLL boundary. Nothing downstream used them, so they are removed rather than kept as an empty macro nobody can verify. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-02 21:56:38 +02:00
fs::path GitFetch(const GitSource& source);
2026-04-27 07:04:42 +02:00
}
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md An agent working on a crafter-build project reads its AGENTS.md, and learns nothing there about the libraries the project builds against unless someone copied that in by hand. This repository shows the cost: the code style section of its AGENTS.md restates what Crafter.LintRules enforces, and goes stale whenever the rules change. A library now ships those notes itself, as AGENTS.consumer.md beside its project.cpp; nothing in its project.cpp changes. When a consumer has an AGENTS.md beside its own project.cpp, `crafter-build` and `crafter-build test` keep a section in it, between two marker comments, holding its dependencies' notes. The section is appended the first time and rewritten in place after that. The file is written only when the section changes, and an AGENTS.md is never created. lint, format, run and clean leave it alone. The update runs after loading, which has just fetched every dependency, and before compiling, so a build that fails still has the notes. cfg.dependencyNotes picks the dependencies. Transitive, the default, follows `dependencies` all the way down from the project's own configurations, its tests and its `#pragma crafter dependency` tooling; Direct stops at the first loaded project on each path; None removes the section. A dependency's own pragma tooling never counts: it is about how that dependency is built, not how it is used, and being no `dependencies` edge it is never reached. Nor are a dependency's tests: nothing builds them for the consumer, and their configurations can point at a local their project file has since moved out of. The walk has to know which Configurations belong to another project, and where that project lives, and cfg.path cannot say: only the Configuration GitProject and LocalProject return is rebased, so a dependency's internal configurations keep "./", which resolves to the consumer. The loader records it instead, LoadedProjectOf beside the project cache, and keeps what the root's pragmas resolved to past the frame LoadProject pops, PragmaDependencies. Neither is exported. The markers count only as whole lines outside fenced code blocks, so documentation can quote them. Unbalanced markers and a document ending inside a code block leave the file alone with a warning: guessing which part of someone's file is generated would overwrite their text, and missing the section would add another one on every build. A note that would break the section when it is read back, through a marker line of its own or a code block it never closes, is left out with a warning. The file is replaced through a temporary renamed onto the symlink's target, so a linked AGENTS.md stays a link and a write cut short leaves the old file rather than part of the new one. The code is a new partition, Crafter.Build:Agents, listed in build.sh, CrafterBuildModules, project.cpp and the umbrella. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
namespace Crafter {
// Where GitProject or LocalProject loaded a Configuration from. The
// loader keeps this beside the cache that owns the Configuration, since
// the Configuration itself cannot say: only the one they return has its
// path rebased, and the configurations a dependency's project file makes
// for itself keep a path such as "./", which resolves against the
// consumer's directory.
struct LoadedProject {
fs::path projectFile; // absolute
fix(agents): what the review of dependency notes found A review of the feature turned up the following, fixed here. Writing. AGENTS.md was replaced through a temporary renamed over it, which swaps in another file: a read-only AGENTS.md was rewritten anyway, a hard link such as a CLAUDE.md beside it kept the old text, ACLs and ownership were dropped, and without an fsync a power cut could still leave it empty. The temporary was named after the PID, which every Flatpak instance shares as PID 2. The file is now written in place, from the first byte that differs, then truncated and synced, under an flock on the file itself. A write that fails partway is undone where the disk allows; with the section last, where it is appended, only the section can be damaged. A read-only file is opened for writing only when something would change, so it is refused then and silent otherwise. Reading. libc++'s filebuf reports a failed read as the end of the file, so an I/O error came back as a short file and the section was written over what was left. Reads go through read(2). Code fences. ```make``` was taken for a fence, and a fence in a list item was missed while its indented closer opened one, so markers a user quoted in a code block could be taken for the section, or the real one hidden and appended again. Backtick info strings, block quotes and list items are now followed, and a block ends with its container. Warnings with nothing to write. With no notes, nothing short of a well-formed section is reported, so an AGENTS.md that ends in an open code block, or a project set to None, no longer warns on every build. An unclosed fence after a section no longer stops it being rewritten in place. Labels. The `from` line carried the git URL, credentials and all, or a path relative to the consumer that changed with the machine, the cache location and --local. It is now the repository's or the directory's name. Notes are deduplicated by repository rather than by clone, so one library pinned two ways is described once, as the nearer spec has it. The walk. It no longer looks at the project's tests: a plain build never did, and a test's dependency can point at the Configuration the project file returned by value. It goes ring by ring over projects rather than configurations, so what the project names itself comes first even with the library/executable split. The project's own AGENTS.consumer.md is excluded by path, also when a dependency's project file sits beside the project's. Removing a section dropped a blank line even from between two paragraphs, joining them; only a section that ends the file takes it now. A CRLF file gets a CRLF section. A note was checked on its raw bytes and again, under other whitespace rules, on the trimmed ones, so it could pass the first check and fail the merge, which then dropped every note. It is normalised once and checked on exactly what is merged. `test --list` only lists, and no longer writes AGENTS.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 05:02:28 +01:00
// What a consumer's AGENTS.md calls it: the repository's name for a
// git project, with the path below the clone's root for one that is
// not at it, and the directory's name for a local one. The same on
// every machine and for a --local checkout and its clone alike,
// which a path into the cache or a sibling checkout is not, and free
// of what a URL can carry, credentials included.
std::string name;
// Which library this is, whatever the revision: the repository and
// the project file in it for git, the project file for a local one.
// Two specs that pin one library differently are one library.
std::string identity;
feat(agents): keep dependencies' AGENTS.consumer.md in AGENTS.md An agent working on a crafter-build project reads its AGENTS.md, and learns nothing there about the libraries the project builds against unless someone copied that in by hand. This repository shows the cost: the code style section of its AGENTS.md restates what Crafter.LintRules enforces, and goes stale whenever the rules change. A library now ships those notes itself, as AGENTS.consumer.md beside its project.cpp; nothing in its project.cpp changes. When a consumer has an AGENTS.md beside its own project.cpp, `crafter-build` and `crafter-build test` keep a section in it, between two marker comments, holding its dependencies' notes. The section is appended the first time and rewritten in place after that. The file is written only when the section changes, and an AGENTS.md is never created. lint, format, run and clean leave it alone. The update runs after loading, which has just fetched every dependency, and before compiling, so a build that fails still has the notes. cfg.dependencyNotes picks the dependencies. Transitive, the default, follows `dependencies` all the way down from the project's own configurations, its tests and its `#pragma crafter dependency` tooling; Direct stops at the first loaded project on each path; None removes the section. A dependency's own pragma tooling never counts: it is about how that dependency is built, not how it is used, and being no `dependencies` edge it is never reached. Nor are a dependency's tests: nothing builds them for the consumer, and their configurations can point at a local their project file has since moved out of. The walk has to know which Configurations belong to another project, and where that project lives, and cfg.path cannot say: only the Configuration GitProject and LocalProject return is rebased, so a dependency's internal configurations keep "./", which resolves to the consumer. The loader records it instead, LoadedProjectOf beside the project cache, and keeps what the root's pragmas resolved to past the frame LoadProject pops, PragmaDependencies. Neither is exported. The markers count only as whole lines outside fenced code blocks, so documentation can quote them. Unbalanced markers and a document ending inside a code block leave the file alone with a warning: guessing which part of someone's file is generated would overwrite their text, and missing the section would add another one on every build. A note that would break the section when it is read back, through a marker line of its own or a code block it never closes, is left out with a warning. The file is replaced through a temporary renamed onto the symlink's target, so a linked AGENTS.md stays a link and a write cut short leaves the old file rather than part of the new one. The code is a new partition, Crafter.Build:Agents, listed in build.sh, CrafterBuildModules, project.cpp and the umbrella. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 03:18:06 +01:00
};
// The LoadedProject behind a Configuration that GitProject or
// LocalProject returned; nullopt for any other: the project crafter-build
// was pointed at, and every configuration a project file makes itself.
std::optional<LoadedProject> LoadedProjectOf(const Configuration& cfg);
}