Catcrafts/Crafter.Build
Watch
2
0
Fork
You've already forked Crafter.Build
 0
Code Issues 5 Pull requests Projects Releases 7 Packages Wiki Activity Actions

Compare commits

base: Catcrafts:6e2d835aa631aa9136dcfa8f00017ff26004a538
Branches Tags
Catcrafts:master
Catcrafts:claude/issue-43
Catcrafts:claude/issue-41
Catcrafts:claude/forward-project-args
Catcrafts:6.0.0
Catcrafts:5.0.0
Catcrafts:4.0.0
Catcrafts:3.0.0
Catcrafts:2.2.0
Catcrafts:2.1.0
Catcrafts:2.0.0
..
compare: Catcrafts:0cbded4a3d249692d4536210dd6d0ae3ace65ece
Branches Tags
Catcrafts:master
Catcrafts:claude/issue-43
Catcrafts:claude/issue-41
Catcrafts:claude/forward-project-args
Catcrafts:6.0.0
Catcrafts:5.0.0
Catcrafts:4.0.0
Catcrafts:3.0.0
Catcrafts:2.2.0
Catcrafts:2.1.0
Catcrafts:2.0.0
10 changed files with 125 additions and 313 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AGENTS.md
View file

@ -86,7 +86,7 @@ Conventions the existing tests follow:
- Collect failures with a `Check(cond, msg)` helper that prints `FAIL: ...` to stderr and counts, then return nonzero at the end. Do not stop at the first failure unless later steps are meaningless without it.
- Drive the engine in-process: `Build(cfg, depResults, depMutex)`, `RunTests`, the lint driver. Do not shell out to `bin/crafter-build` and grep its stdout.
- Fixtures under `tests/<Name>/fixture/` are inputs. A test that edits or touches files first stages a copy under `fs::temp_directory_path() / "crafter-build-<name>"` and removes it on the next run. Variant sources are checked in as `*.in` files and copied over the original inside the staged copy.
- Use a fresh `depResults` map per build pass: the map memoises each configuration's result, and each external dependency's fetch and build, for one pass, so reusing it skips rebuilds and pulls you meant to observe. `Build()` itself is safe to call repeatedly on the same `Configuration`.
- Use a fresh `depResults` map per build pass: the map memoises each configuration's result for one pass, so reusing it skips rebuilds you meant to observe. `Build()` itself is safe to call repeatedly on the same `Configuration`.
- Set `cfg.path` before calling `GetInterfacesAndImplementations`, and spell `cFiles` as absolute paths: both resolve against the working directory, not the configuration.
A bug fix ships with a test that fails before the fix and passes after it, in its own `test:` commit when it documents the behaviour on its own.

2
README.md
View file

@ -60,7 +60,7 @@ glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "glslang", "OSDependent" };
```
Clones live at `~/.cache/crafter.build/external/<name>-<hash>/`, keyed by `(url, branch, commit, options, target)` so different projects sharing the same dep with the same spec reuse the clone and cmake build, while projects with diverging specs each get their own cache entry. Without a `commit`, the clone is `git pull`ed once per build: every configuration of the build that declares the dependency, such as each test of a project whose tests copy its `externalDependencies`, shares that one fetch and cmake build. The clone is resilient to partial-clone recovery and CMake reconfigure is triggered automatically on `options` changes. Submodules are checked out at the revisions the fetched commit records, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules and not the default branch's.
Clones live at `~/.cache/crafter.build/external/<name>-<hash>/`, keyed by `(url, branch, commit, options)` so different projects sharing the same dep with the same spec reuse the clone and cmake build, while projects with diverging specs each get their own cache entry. The clone is resilient to partial-clone recovery and CMake reconfigure is triggered automatically on `options` changes. Submodules are checked out at the revisions the fetched commit records, after every clone, switch, checkout and pull, so a pinned commit builds against its own submodules and not the default branch's.
**Header-only git deps** — same as cmake deps but `builder = ExternalBuilder::None`. Just clones and propagates `-I` flags.

83
implementations/Crafter.Build-Clang.cpp
View file

@ -423,28 +423,6 @@ CompileCommand Crafter::GetCompileCommand(const Configuration& config) {
return out;
}
namespace {
// The outcome of `work` for `key` in this build pass. The first caller
// runs it; every other caller, whether at the same time or later, waits
// for that run and gets its result or its exception. The keys are the
// directories the work writes, a configuration's PcmDir and an external
// dependency's ExternalBuildDir, so the two kinds never collide.
std::shared_future<BuildResult> OncePerPass(std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex, const fs::path& key, const std::function<BuildResult()>& work) {
std::promise<BuildResult> promise;
std::shared_future<BuildResult> outcome = promise.get_future().share();
{
std::lock_guard lock(depMutex);
if (auto [it, inserted] = depResults.try_emplace(key, outcome); !inserted) return it->second;
}
try {
promise.set_value(work());
} catch (...) {
promise.set_exception(std::current_exception());
}
return outcome;
}
}
BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex) {
// Before anything is spawned: one message naming the missing compiler
// beats a clang++-not-found per translation unit.
@ -700,30 +678,14 @@ BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, s
externalThreads.reserve(config.externalDependencies.size());
for (std::size_t i = 0; i < config.externalDependencies.size(); ++i) {
externalThreads.emplace_back([&, i]() {
const ExternalDependency& dep = config.externalDependencies[i];
Progress::Task task(std::format("Building external dep {}", dep.name));
Progress::Task task(std::format("Building external dep {}", config.externalDependencies[i].name));
if (buildCancelled.load(std::memory_order_relaxed)) return;
std::string error;
try {
// Every configuration of the pass that declares this
// dependency shares one fetch and build of it. A project's
// tests often each copy its externalDependencies, and RunTests
// builds them side by side: each of them pulling the clone
// cost a network round trip per test, and before the clone had
// a lock, the simultaneous pulls broke its FETCH_HEAD and
// failed every test (issue #34).
const std::shared_future<BuildResult> built = OncePerPass(depResults, depMutex, ExternalBuildDir(dep, config.target), [&] {
return BuildResult{BuildExternal(dep, config.target), false, {}};
});
error = built.get().result;
if (error.empty()) externalResults[i] = ExternalOutputs(dep, config.target);
} catch (const std::exception& e) {
error = std::format("external dep {} threw: {}", dep.name, e.what());
}
if (error.empty()) return;
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(error);
externalResults[i] = BuildExternal(config.externalDependencies[i], config.target, buildCancelled);
if (!externalResults[i].error.empty()) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = externalResults[i].error;
}
}
});
}
@ -784,8 +746,35 @@ BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, s
try {
if (buildCancelled.load(std::memory_order_relaxed)) return;
const std::shared_future<BuildResult> built = OncePerPass(depResults, depMutex, dep->PcmDir(), [&] { return Build(*dep, depResults, depMutex); });
const BuildResult& result = built.get();
std::shared_ptr<std::promise<BuildResult>> promise;
std::shared_future<BuildResult> resultFuture;
bool isBuilder = false;
depMutex.lock();
fs::path cacheKey = dep->PcmDir();
auto it = depResults.find(cacheKey);
if (it == depResults.end()) {
isBuilder = true;
promise = std::make_shared<std::promise<BuildResult>>();
resultFuture = promise->get_future().share();
depResults.emplace(cacheKey, resultFuture);
} else {
resultFuture = it->second;
}
depMutex.unlock();
if (isBuilder) {
BuildResult built;
try {
built = Build(*dep, depResults, depMutex);
} catch (...) {
promise->set_exception(std::current_exception());
throw;
}
promise->set_value(std::move(built));
}
const BuildResult& result = resultFuture.get();
fileMutex.lock();
for (const std::string& lib : result.libs) libSet.insert(lib);
for (const std::string& f : result.publicCompileFlags) publicFlagSet.insert(f);

127
implementations/Crafter.Build-External.cpp
View file

@ -231,16 +231,15 @@ std::vector<std::string> Crafter::ExternalIncludeFlags(const ExternalDependency&
return flags;
}
fs::path Crafter::ExternalBuildDir(const ExternalDependency& dep, std::string_view target) {
fs::path cloneDir = ExternalCloneDir(dep, target);
if (cloneDir.empty() || dep.builder != ExternalBuilder::CMake) return cloneDir;
return cloneDir / "build";
}
ExternalBuildResult Crafter::BuildExternal(const ExternalDependency& dep, std::string_view target, std::atomic<bool>& cancelled) {
ExternalBuildResult result;
if (cancelled.load(std::memory_order_relaxed)) return result;
std::string Crafter::BuildExternal(const ExternalDependency& dep, std::string_view target) {
std::string name = dep.name.empty() ? DeriveName(dep.source) : dep.name;
if (name.empty()) {
return std::format("Could not derive name for external dependency from URL '{}'", dep.source.url);
result.error = std::format("Could not derive name for external dependency from URL '{}'", dep.source.url);
return result;
}
fs::path externalRoot = GetCacheDir() / "external";
@ -248,73 +247,77 @@ std::string Crafter::BuildExternal(const ExternalDependency& dep, std::string_vi
std::error_code ec;
fs::create_directories(externalRoot, ec);
if (ec) {
return std::format("Failed to create {}: {}", externalRoot.string(), ec.message());
result.error = std::format("Failed to create {}: {}", externalRoot.string(), ec.message());
return result;
}
}
fs::path cloneDir = ExternalCloneDir(dep, target);
// Build() makes one call for all the configurations of a pass, but two
// crafter-builds, or two passes, can still reach the same clone at once,
// and two pulls in one clone race on its .git/FETCH_HEAD.
// Builds running side by side (tests built in parallel, two
// crafter-builds) share the clone: one fetches and builds it at a time.
FileLock lock(fs::path(std::format("{}.lock", cloneDir.string())));
if (std::string err = FetchGit(dep.source, cloneDir); !err.empty()) {
return std::format("git fetch for '{}': {}", name, err);
std::string fetchErr = FetchGit(dep.source, cloneDir);
if (!fetchErr.empty()) {
result.error = std::format("git fetch for '{}': {}", name, fetchErr);
return result;
}
if (dep.builder != ExternalBuilder::CMake) return "";
const fs::path cmakeBuildDir = ExternalBuildDir(dep, target);
if (std::string err = ConfigureCMake(cloneDir, cmakeBuildDir, dep.options, target); !err.empty()) {
return std::format("cmake configure for '{}': {}", name, err);
}
if (std::string err = BuildCMake(cmakeBuildDir); !err.empty()) {
return std::format("cmake build for '{}': {}", name, err);
}
return "";
}
if (cancelled.load(std::memory_order_relaxed)) return result;
fs::path cmakeBuildDir;
if (dep.builder == ExternalBuilder::CMake) {
cmakeBuildDir = cloneDir / "build";
if (std::string err = ConfigureCMake(cloneDir, cmakeBuildDir, dep.options, target); !err.empty()) {
result.error = std::format("cmake configure for '{}': {}", name, err);
return result;
}
if (cancelled.load(std::memory_order_relaxed)) return result;
if (std::string err = BuildCMake(cmakeBuildDir); !err.empty()) {
result.error = std::format("cmake build for '{}': {}", name, err);
return result;
}
}
ExternalBuildResult Crafter::ExternalOutputs(const ExternalDependency& dep, std::string_view target) {
ExternalBuildResult result;
result.compileFlags = ExternalIncludeFlags(dep, target);
const fs::path cmakeBuildDir = ExternalBuildDir(dep, target);
if (dep.builder != ExternalBuilder::CMake || cmakeBuildDir.empty()) return result;
if (dep.builder == ExternalBuilder::CMake) {
// Each search path gets both a -L (link-time) and a -Wl,-rpath
// (runtime-loader). The rpath stays embedded in the produced
// binary so it picks up shared deps from the cache without any
// LD_LIBRARY_PATH gymnastics. Static deps (.a) ignore the rpath
// harmlessly. The PE/COFF target (mingw) resolves DLLs via PATH or
// adjacency rather than rpath, so lld warns if we pass it.
const bool isPe = target == "x86_64-w64-mingw32";
std::string buildDirAbs = fs::absolute(cmakeBuildDir).string();
result.linkFlags.push_back(std::format("-L{}", buildDirAbs));
if (!isPe) result.linkFlags.push_back(std::format("-Wl,-rpath,{}", buildDirAbs));
for (const fs::path& libDir : dep.libDirs) {
fs::path full = libDir.is_absolute() ? libDir : cmakeBuildDir / libDir;
std::string fullAbs = fs::absolute(full).string();
result.linkFlags.push_back(std::format("-L{}", fullAbs));
if (!isPe) result.linkFlags.push_back(std::format("-Wl,-rpath,{}", fullAbs));
}
for (const std::string& lib : dep.libs) {
result.linkFlags.push_back(std::format("-l{}", lib));
}
// Each search path gets both a -L (link-time) and a -Wl,-rpath
// (runtime-loader). The rpath stays embedded in the produced
// binary so it picks up shared deps from the cache without any
// LD_LIBRARY_PATH gymnastics. Static deps (.a) ignore the rpath
// harmlessly. The PE/COFF target (mingw) resolves DLLs via PATH or
// adjacency rather than rpath, so lld warns if we pass it.
const bool isPe = target == "x86_64-w64-mingw32";
std::string buildDirAbs = fs::absolute(cmakeBuildDir).string();
result.linkFlags.push_back(std::format("-L{}", buildDirAbs));
if (!isPe) result.linkFlags.push_back(std::format("-Wl,-rpath,{}", buildDirAbs));
for (const fs::path& libDir : dep.libDirs) {
fs::path full = libDir.is_absolute() ? libDir : cmakeBuildDir / libDir;
std::string fullAbs = fs::absolute(full).string();
result.linkFlags.push_back(std::format("-L{}", fullAbs));
if (!isPe) result.linkFlags.push_back(std::format("-Wl,-rpath,{}", fullAbs));
}
for (const std::string& lib : dep.libs) {
result.linkFlags.push_back(std::format("-l{}", lib));
}
// The newest library in every directory the link searches: the build
// directory and the libDirs, where projects like msquic put theirs,
// versioned names (libmsquic.so.2) included.
auto isLibrary = [](const std::string& name) {
return name.ends_with(".a") || name.ends_with(".so") || name.contains(".so.") || name.ends_with(".dll");
};
std::vector<fs::path> searched = { cmakeBuildDir };
for (const fs::path& libDir : dep.libDirs) searched.push_back(libDir.is_absolute() ? libDir : cmakeBuildDir / libDir);
for (const fs::path& dir : searched) {
std::error_code listEc;
for (fs::directory_iterator it(dir, listEc), end; !listEc && it != end; it.increment(listEc)) {
std::error_code ec;
if (!it->is_regular_file(ec) || !isLibrary(it->path().filename().string())) continue;
fs::file_time_type t = it->last_write_time(ec);
if (!ec && t > result.latestArtifact) result.latestArtifact = t;
// The newest library in every directory the link searches: the build
// directory and the libDirs, where projects like msquic put theirs,
// versioned names (libmsquic.so.2) included.
auto isLibrary = [](const std::string& name) {
return name.ends_with(".a") || name.ends_with(".so") || name.contains(".so.") || name.ends_with(".dll");
};
std::vector<fs::path> searched = { cmakeBuildDir };
for (const fs::path& libDir : dep.libDirs) searched.push_back(libDir.is_absolute() ? libDir : cmakeBuildDir / libDir);
for (const fs::path& dir : searched) {
std::error_code listEc;
for (fs::directory_iterator it(dir, listEc), end; !listEc && it != end; it.increment(listEc)) {
std::error_code ec;
if (!it->is_regular_file(ec) || !isLibrary(it->path().filename().string())) continue;
fs::file_time_type t = it->last_write_time(ec);
if (!ec && t > result.latestArtifact) result.latestArtifact = t;
}
}
}

7
interfaces/Crafter.Build-Clang.cppm
View file

@ -656,13 +656,6 @@ export namespace Crafter {
};
CompileCommand GetCompileCommand(const Configuration& config);
// Builds `config` and the configurations it depends on. depResults is the
// record of one build pass, guarded by depMutex and keyed by the directory
// each piece of work writes: a dependency's PcmDir, an external
// dependency's ExternalBuildDir. Configurations built with the same map,
// as RunTests builds its tests, build a shared dependency and fetch a
// shared external once between them. A fresh map starts a new pass, which
// checks everything again and pulls unpinned externals again.
BuildResult Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex);
// Takes main's argv verbatim so the CLI entry point is a one-liner;

21
interfaces/Crafter.Build-External.cppm
View file

@ -36,6 +36,7 @@ export namespace Crafter {
};
struct ExternalBuildResult {
std::string error;
std::vector<std::string> compileFlags;
std::vector<std::string> linkFlags;
fs::file_time_type latestArtifact = fs::file_time_type::min();
@ -46,30 +47,12 @@ export namespace Crafter {
// through this, so a caller that only needs to know where the headers ended
// up cannot drift from where they actually are.
fs::path ExternalCloneDir(const ExternalDependency& dep, std::string_view target);
// Where BuildExternal leaves this dependency: the CMake build directory,
// or the clone itself when it is only fetched. Pure, like ExternalCloneDir.
// Build() keys a pass's record of the work on it rather than on the clone,
// so a declaration that shares the clone but builds it with CMake is not
// handed the outcome of a fetch alone.
fs::path ExternalBuildDir(const ExternalDependency& dep, std::string_view target);
// The -I flags this dependency contributes, from its declared includeDirs.
// Also pure, which is what lets the linter's AST layer parse a source that
// includes an external's headers without running a build to find out where
// they are. The paths only resolve once something has fetched the clone.
std::vector<std::string> ExternalIncludeFlags(const ExternalDependency& dep, std::string_view target);
// Fetches the clone and, for ExternalBuilder::CMake, configures and builds
// it. Empty on success, otherwise what failed. A lock on the clone is held
// throughout, so another crafter-build, or another build pass in this one,
// works on it before or after this call and never during it. Not
// cancellable once started: Build() hands the outcome to every
// configuration of the pass that declares the dependency, and a run cut
// short would leave them a clone nobody fetched.
std::string BuildExternal(const ExternalDependency& dep, std::string_view target);
// What a configuration that declares `dep` compiles and links with once
// BuildExternal has run: the -I flags and, for CMake, the -L, -rpath and -l
// flags and the newest library. Per declaration, because two that share a
// clone can still differ in includeDirs, libDirs and libs.
ExternalBuildResult ExternalOutputs(const ExternalDependency& dep, std::string_view target);
ExternalBuildResult BuildExternal(const ExternalDependency& dep, std::string_view target, std::atomic<bool>& cancelled);
// Specification for a sibling crafter-build project to fetch and depend on.
// GitSource picks the revision: leave branch + commit empty for the

1
project.cpp
View file

@ -262,7 +262,6 @@ extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> a
cfg.AddTest("LinkFlagsChange").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ExternalArtifactTime").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("ConcurrentExternalFetch").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("SharedExternalFetch").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("StandardArgs").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("TestRunnerSpec").Dependencies({ CrafterBuildLib.get() });
cfg.AddTest("VariantId").Dependencies({ CrafterBuildLib.get() });

48
tests/ConcurrentExternalFetch/main.cpp
View file

@ -7,19 +7,17 @@ namespace fs = std::filesystem;
using namespace Crafter;
// Builds that run at the same time share their dependencies' clones in the
// cache, and two crafter-builds can run at once. Fetching a clone that
// already exists ran `git pull` (and the CMake build) in it with nothing to
// keep a second fetch out, and the second git fails on the first one's
// index.lock. A clone's fetch and build now hold a lock of their own. Within
// one run, the configurations that declare the same dependency share one
// fetch of it instead (SharedExternalFetch).
// cache: the tests `crafter-build test --jobs=N` builds side by side each
// declare the same ExternalDependency, and two crafter-builds can run at
// once. Fetching a clone that already exists ran `git pull` (and the CMake
// build) in it with nothing to keep a second fetch out, and the second git
// fails on the first one's index.lock. A clone's fetch and build now hold a
// lock of their own.
//
// Eight threads fetch the same tracked branch of a local repository at
// once, a few rounds over, the branch moving between rounds so every pull
// has work to do; every fetch has to succeed. Each thread fetches through
// GitFetch and through BuildExternal, which every build of an
// ExternalDependency calls, each into a clone of its own. Everything is
// local; XDG_CACHE_HOME keeps the clones out of the real cache.
// has work to do; every fetch has to succeed. Everything is local;
// XDG_CACHE_HOME keeps the clone out of the real cache.
extern "C" int setenv(const char* name, const char* value, int overwrite);
@ -44,13 +42,6 @@ namespace {
Git(repo, "add round.txt");
Git(repo, std::format("commit -q -m round-{}", round));
}
std::uint32_t Round(const fs::path& clone) {
std::ifstream in(clone / "round.txt");
std::uint32_t round = 0;
in >> round;
return round;
}
}
int main() {
@ -69,12 +60,6 @@ int main() {
Commit(repo, 0);
const GitSource source { .url = std::format("file://{}", repo.string()), .branch = "main" };
const fs::path clone = GitFetch(source);
// A clone of its own: BuildExternal keys the cache differently.
ExternalDependency dep;
dep.name = "shared";
dep.source = source;
const std::string cloned = BuildExternal(dep, HostTarget());
Check(cloned.empty(), std::format("BuildExternal clones the repository: {}", cloned));
constexpr std::uint32_t threads = 8;
constexpr std::uint32_t rounds = 4;
@ -85,23 +70,20 @@ int main() {
std::vector<std::jthread> fetchers;
for (std::uint32_t i = 0; i < threads; ++i) {
fetchers.emplace_back([&] {
std::string error;
try {
(void)GitFetch(source);
error = BuildExternal(dep, HostTarget());
} catch (const std::exception& e) {
error = e.what();
std::lock_guard lock(errorsMutex);
errors.push_back(e.what());
}
if (error.empty()) return;
std::lock_guard lock(errorsMutex);
errors.push_back(std::move(error));
});
}
}
Check(errors.empty(), std::format("{} threads fetching at once, {} rounds: every fetch succeeds ({} failed{}{})", threads, rounds, errors.size(), errors.empty() ? "" : ", the first: ", errors.empty() ? "" : errors.front()));
Check(Round(clone) == rounds, std::format("and GitFetch's clone ends at the last round ({})", Round(clone)));
const fs::path externalClone = ExternalCloneDir(dep, HostTarget());
Check(Round(externalClone) == rounds, std::format("and so does BuildExternal's ({})", Round(externalClone)));
Check(errors.empty(), std::format("{} fetches at once, {} rounds: every one succeeds ({} failed{}{})", threads, rounds, errors.size(), errors.empty() ? "" : ", the first: ", errors.empty() ? "" : errors.front()));
std::ifstream round(clone / "round.txt");
std::uint32_t last = 0;
round >> last;
Check(last == rounds, std::format("and the clone ends at the last round ({})", last));
std::error_code ec;
fs::remove_all(scratch, ec);

10
tests/ExternalArtifactTime/main.cpp
View file

@ -7,8 +7,8 @@ namespace fs = std::filesystem;
using namespace Crafter;
// A CMake dependency's newest library decides when its consumers are stale
// (the latestArtifact of ExternalOutputs). Only files directly in the CMake
// build directory named *.a or *.so were looked at, so msquic's
// (BuildExternal's latestArtifact). Only files directly in the CMake build
// directory named *.a or *.so were looked at, so msquic's
// bin/Release/libmsquic.so.2, in one of the dependency's libDirs and with a
// version after the .so, never counted: a rebuilt msquic left everything
// built against the old one in place.
@ -69,9 +69,9 @@ int main() {
dep.builder = ExternalBuilder::CMake;
dep.libDirs = { "bin/Release" };
dep.libs = { "versioned" };
const std::string error = BuildExternal(dep, HostTarget());
Check(error.empty(), std::format("the dependency builds: {}", error));
const ExternalBuildResult result = ExternalOutputs(dep, HostTarget());
std::atomic<bool> cancelled = false;
const ExternalBuildResult result = BuildExternal(dep, HostTarget(), cancelled);
Check(result.error.empty(), std::format("the dependency builds: {}", result.error));
const fs::path library = ExternalCloneDir(dep, HostTarget()) / "build" / "bin" / "Release" / "libversioned.so.2.0.0";
Check(fs::exists(library), std::format("the library is where its CMake project puts it: {}", library.string()));

137
tests/SharedExternalFetch/main.cpp
View file

@ -1,137 +0,0 @@
// SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
// A project whose tests each copy its externalDependencies gives every test
// configuration the same unpinned dependency, and `crafter-build test`
// builds those configurations side by side. Each of them pulled the shared
// clone: at first all at once, which broke the clone's FETCH_HEAD and failed
// every test (issue #34), and then, under the clone's lock, one after
// another, a network round trip per test. The configurations of one build
// pass now share a single fetch and build of the dependency, and the next
// pass fetches it again.
//
// Eight tests that include a header from the same tracked branch of a local
// repository run through RunTests at once, and again after the branch has
// moved. GIT_TRACE logs every git process, and every fetch of the
// repository, clone or pull, runs `git upload-pack` in it once. Everything
// is local; XDG_CACHE_HOME keeps the clone out of the real cache.
extern "C" int setenv(const char* name, const char* value, int overwrite);
namespace {
std::int32_t failures = 0;
void Check(bool cond, std::string_view msg) {
if (!cond) {
std::println(std::cerr, "FAIL: {}", msg);
++failures;
}
}
void Git(const fs::path& dir, std::string_view args) {
const std::string command = std::format("git -C {} {}", ShellQuote(dir.string()), args);
auto result = RunCommandWithTimeout(command, std::chrono::seconds(60));
if (result.exitCode != 0) throw std::runtime_error(std::format("{} failed: {}", command, result.output));
}
void Write(const fs::path& file, std::string_view text) {
std::ofstream(file, std::ios::binary | std::ios::trunc) << text;
}
void Commit(const fs::path& repo, std::uint32_t round) {
Write(repo / "round.h", std::format("#define ROUND {}\n", round));
Git(repo, "add round.h");
Git(repo, std::format("commit -q -m round-{}", round));
}
std::uint32_t Fetches(const fs::path& trace) {
std::ifstream in(trace);
std::uint32_t count = 0;
for (std::string line; std::getline(in, line);) {
if (line.contains("trace: built-in: git upload-pack")) ++count;
}
return count;
}
// The first test that did not pass, for the message.
std::string FirstFailure(const TestSummary& summary) {
for (const TestResult& result : summary.results) {
if (result.outcome != TestOutcome::Pass) return std::format(", the first: {}: {}", result.name, result.output);
}
return "";
}
}
int main() {
const fs::path scratch = fs::temp_directory_path() / "crafter-build-shared-external-fetch";
fs::remove_all(scratch);
fs::create_directories(scratch);
setenv("XDG_CACHE_HOME", (scratch / "cache").c_str(), 1);
setenv("GIT_CONFIG_NOSYSTEM", "1", 1);
setenv("GIT_CONFIG_GLOBAL", "/dev/null", 1);
for (const char* name : { "GIT_AUTHOR_NAME", "GIT_COMMITTER_NAME" }) setenv(name, "crafter-build test", 1);
for (const char* name : { "GIT_AUTHOR_EMAIL", "GIT_COMMITTER_EMAIL" }) setenv(name, "test@example.invalid", 1);
const fs::path trace = scratch / "git-trace.log";
setenv("GIT_TRACE", trace.c_str(), 1);
const fs::path repo = scratch / "shared";
fs::create_directories(repo);
Git(repo, "init -q -b main");
Commit(repo, 1);
ExternalDependency shared;
shared.name = "shared";
shared.source = { .url = std::format("file://{}", repo.string()), .branch = "main" };
shared.includeDirs = { "" };
const fs::path project = scratch / "project";
fs::create_directories(project);
Write(project / "main.cpp", "#include \"round.h\"\n\nint main() { return ROUND > 0 ? 0 : 1; }\n");
constexpr std::int32_t tests = 8;
Configuration cfg;
cfg.path = project;
cfg.name = "project";
cfg.target = HostTarget();
for (std::int32_t i = 0; i < tests; ++i) {
Test& test = cfg.tests.emplace_back();
test.config.path = project;
test.config.name = std::format("t{}", i);
test.config.outputName = test.config.name;
test.config.target = HostTarget();
test.config.type = ConfigurationType::Executable;
test.config.externalDependencies = { shared };
std::array<fs::path, 0> interfaces = {};
std::array<fs::path, 1> implementations = { "main" };
test.config.GetInterfacesAndImplementations(interfaces, implementations);
}
const RunTestsOptions options { .jobs = tests };
TestSummary summary = RunTests(cfg, options);
Check(summary.passed == tests, std::format("{} tests that share the dependency build and pass at once ({} passed{})", tests, summary.passed, FirstFailure(summary)));
const std::uint32_t first = Fetches(trace);
Check(first == 1, std::format("and fetch it once between them, not once each ({} fetches)", first));
Commit(repo, 2);
summary = RunTests(cfg, options);
Check(summary.passed == tests, std::format("after the branch moved, they pass again ({} passed{})", summary.passed, FirstFailure(summary)));
const std::uint32_t second = Fetches(trace) - first;
Check(second == 1, std::format("and the next run fetches it once more ({} fetches)", second));
std::ifstream header(ExternalCloneDir(shared, HostTarget()) / "round.h");
std::string round;
std::getline(header, round);
Check(round == "#define ROUND 2", std::format("which brings the clone to the new commit ({})", round));
std::error_code ec;
fs::remove_all(scratch, ec);
if (failures > 0) {
std::println(std::cerr, "{} assertions failed", failures);
return 1;
}
return 0;
}