fix: rebuild and relink on what changed, submodules at their revisions, and mingw found where it is
Some checks failed
Flatpak / flatpak (push) Failing after 1m12s
Some checks failed
Flatpak / flatpak (push) Failing after 1m12s
- Shaders record the files glslang opened (a depfile) and their settings
(source path, stage, entry point, defines, include directories) in the
build directory, and rebuild when an included file or a setting changes,
not only the shader's own source.
- An executable relinks when its link command changed, not only when an
object is newer: a link flag in project.cpp, or a dependency's library
that moved (an ExternalDependency built into a new cache directory). The
command is recorded beside the objects; its flags come from ordered sets,
so the order the dependency threads finish in can't change it.
- An external's newest artifact counts its libDirs and versioned .so names.
- Submodules are checked out at the revisions the fetched commit records,
from the URLs its .gitmodules gives, after every clone, switch, checkout
and pull, so a pinned commit builds against its own submodules, and one
moved by hand is put back; with no .gitmodules, git isn't asked.
- Each external clone's fetch and build hold a file lock: parallel builds
pulled into one clone at once ("cannot lock ref").
- mingw:
- -stdlib=libstdc++ always;
- the std module's source is found through libstdc++.modules.json in the
mingw toolchain's own library directories (the generic lib/ holds the
host's Linux one), asking the compiler once a run;
- a toolchain that isn't there is a build error saying where it looked,
not std::terminate;
- CRAFTER_MINGW_DIR names the prefix, the directory holding
x86_64-w64-mingw32/ and lib/gcc/;
- inside the Flatpak the host's /usr mingw is used.
- RunScript's exit-status check reads as the one range it is.
- Tests: ConcurrentExternalFetch, ExternalArtifactTime, GitSubmodules,
LinkFlagsChange, MingwMissing, ShaderIncludeChange; ShaderCompile and
ToolchainResolve extended.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
959c65dbb2
commit
653dd0bb9b
26 changed files with 1075 additions and 109 deletions
|
|
@ -14,12 +14,15 @@ export namespace Crafter {
|
|||
struct BuildResult {
|
||||
std::string result;
|
||||
bool repack;
|
||||
std::unordered_set<std::string> libs;
|
||||
// Sets in order: the commands they go into are compared with the ones
|
||||
// recorded last time, so they come out the same however the
|
||||
// dependency threads that fill them happen to finish.
|
||||
std::set<std::string> libs;
|
||||
// Compile flags (typically -I include paths) the dep wants its
|
||||
// consumers to see — sourced from its external dependencies' include
|
||||
// dirs so headers a dep exposes in its public modules are reachable
|
||||
// when a consumer #includes them directly. Propagates transitively.
|
||||
std::unordered_set<std::string> publicCompileFlags;
|
||||
std::set<std::string> publicCompileFlags;
|
||||
};
|
||||
|
||||
struct Define {
|
||||
|
|
|
|||
|
|
@ -14,6 +14,21 @@ namespace Crafter {
|
|||
bool timedOut = false;
|
||||
std::int32_t signal = 0;
|
||||
};
|
||||
// An exclusive lock on the file `path` (created if missing), held from
|
||||
// construction to destruction. An flock, so it keeps out the other
|
||||
// threads of this process (each opens the file anew) and other
|
||||
// crafter-build processes alike. The OS drops it when the descriptor
|
||||
// closes, process death included, so an interrupted build never leaves a
|
||||
// stale lock; the descriptor is close-on-exec, so the tools a holder
|
||||
// spawns don't keep it past its release.
|
||||
class FileLock {
|
||||
std::int32_t fd = -1;
|
||||
public:
|
||||
explicit FileLock(const fs::path& path);
|
||||
~FileLock();
|
||||
FileLock(const FileLock&) = delete;
|
||||
FileLock& operator=(const FileLock&) = delete;
|
||||
};
|
||||
std::string BuildStdPcm(const Configuration& config, fs::path stdPcm);
|
||||
fs::path GetCacheDir();
|
||||
CommandResult RunCommandChecked(std::string_view command);
|
||||
|
|
|
|||
|
|
@ -38,16 +38,23 @@ namespace Crafter {
|
|||
// gives "<name>.spv", "sets" gives "<name>.sets.spv". One file-name
|
||||
// component: no path separators. Every entry of a configuration must
|
||||
// end up with its own OutputName(); Build() fails if two collide, so
|
||||
// each extra compile of a source needs its own variant. Check()
|
||||
// compares only the source's mtime: changing defines without touching
|
||||
// the source leaves the old output in place.
|
||||
// each extra compile of a source needs its own variant.
|
||||
std::string variant;
|
||||
Shader(fs::path&& path, std::string&& entrypoint, ShaderType type);
|
||||
// File name of the compiled SPIR-V, relative to the output directory.
|
||||
// The single source of truth for it: Check, Compile and the artifact
|
||||
// forwarding in Build() all go through here.
|
||||
fs::path OutputName() const;
|
||||
bool Check(const fs::path& outputDir) const;
|
||||
std::string Compile(const fs::path& outputDir, std::span<const fs::path> includeDirs) const;
|
||||
// Whether the output in `outputDir` is up to date: newer than the
|
||||
// source and than every file the source #included, and compiled with
|
||||
// this entry's stage, entry point and defines, searching
|
||||
// `includeDirs`. Compile records the last two in `buildDir`; without
|
||||
// its record an output is stale.
|
||||
bool Check(const fs::path& outputDir, const fs::path& buildDir, std::span<const fs::path> includeDirs) const;
|
||||
// Compiles into `outputDir`, searching `includeDirs` for #include, and
|
||||
// records what Check needs in `buildDir`: a depfile naming the source
|
||||
// and every file it included, and the settings it was compiled with.
|
||||
// Returns an error message, empty on success.
|
||||
std::string Compile(const fs::path& outputDir, const fs::path& buildDir, std::span<const fs::path> includeDirs) const;
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -37,12 +37,13 @@ export namespace Crafter {
|
|||
// Empty when the crafter prefix is unknown; need not exist.
|
||||
fs::path stdModulesDir;
|
||||
fs::path wasiSysroot;
|
||||
// Holds include/c++/<ver>/bits/std.cc for the mingw target. Passed to
|
||||
// clang as --sysroot only when it was set explicitly: clang's own
|
||||
// mingw discovery handles the distro layout, and a --sysroot pointing
|
||||
// at the triple directory would lose crtbegin.o and libgcc.
|
||||
// The mingw-w64 install prefix, the directory that holds
|
||||
// x86_64-w64-mingw32/ and lib/gcc/x86_64-w64-mingw32/ (/usr for a
|
||||
// distro package), passed to clang as --sysroot for the mingw target.
|
||||
// Empty means clang's own discovery, which follows the mingw gcc on
|
||||
// PATH. Set by CRAFTER_MINGW_DIR, else the host's /usr when running
|
||||
// sandboxed and it has mingw-w64 (the sandbox's own /usr never does).
|
||||
fs::path mingwSysroot;
|
||||
bool mingwSysrootExplicit = false;
|
||||
// `clang++ -print-target-triple` and the first line of
|
||||
// `clang++ --version`; both empty when clang++ could not be run.
|
||||
std::string hostTriple;
|
||||
|
|
@ -72,6 +73,9 @@ export namespace Crafter {
|
|||
std::map<std::string, std::string, std::less<>> env;
|
||||
// The <prefix> of <prefix>/share/crafter-build; empty when unknown.
|
||||
fs::path crafterPrefix;
|
||||
// Where the host's root shows inside a sandbox (/run/host in a
|
||||
// Flatpak); empty outside one.
|
||||
fs::path hostRoot;
|
||||
};
|
||||
// Pure path resolution: an env override wins, then a toolchain installed
|
||||
// beside crafter-build (<crafterPrefix>/bin/clang++), then bare names on
|
||||
|
|
|
|||
Loading…
Reference in a new issue