V2: WASI, -r flag, CI pipeline, examples & tests cleanup #1

Merged
jorijnvdgraaf merged 10 commits from V2 into master 2026-04-29 02:35:36 +02:00
24 changed files with 1467 additions and 314 deletions
Showing only changes of commit f13671b2be - Show all commits

v2 nearly done

Jorijn van der Graaf 2026-04-27 07:04:42 +02:00

2
.gitignore vendored
View file

@ -1,2 +1,4 @@
build/ build/
bin/ bin/
share/crafter-build/
.claude

25
PKGBUILD Normal file
View file

@ -0,0 +1,25 @@
pkgname=crafter-build
pkgver=0.1.0
pkgrel=1
pkgdesc='C++26 modules build system'
arch=('x86_64')
url='https://forgejo.catcrafts.net/Catcrafts/Crafter.Build'
license=('LGPL-3.0-only')
depends=('clang' 'libc++' 'lld')
makedepends=('cmake' 'git')
source=()
sha256sums=()
options=('!strip' '!debug' '!lto')
build() {
cd "$startdir"
rm -rf bin build share/crafter-build
CRAFTER_BUILD_MARCH=x86-64-v3 CRAFTER_BUILD_MTUNE=generic ./build.sh
}
package() {
cd "$startdir"
install -Dm755 bin/crafter-build "$pkgdir/usr/bin/crafter-build"
install -dm755 "$pkgdir/usr/share/crafter-build"
install -m644 share/crafter-build/*.cppm "$pkgdir/usr/share/crafter-build/"
}

126
README.md Normal file
View file

@ -0,0 +1,126 @@
# Crafter Build
A C++26 modules build system. Project descriptions are written in C++ — no JSON, no Lua, no embedded DSL. You write a `project.cpp` that constructs a `Configuration` and returns it; Crafter Build compiles, loads, and executes it.
## Status
- **Linux**: working end-to-end, packageable as a distro package. Verified on Arch Linux via [PKGBUILD](PKGBUILD) in a fresh container.
- **Windows**: bootstrap builds and runs; cross-EXE/DLL symbol-resolution work in progress.
## Quick start (Linux)
Bootstrap requires `clang`, `cmake`, `git`, `lld`, and `libc++`.
```bash
./build.sh # produces bin/crafter-build
CRAFTER_BUILD_HOME=$PWD/build ./bin/crafter-build # rebuild via project.cpp
```
For distro-packaged installs, `crafter-build` finds its modules at `<prefix>/share/crafter-build/` automatically — no env var required.
To build the system as a distro package on Arch:
```bash
makepkg -si # uses CRAFTER_BUILD_MARCH=x86-64-v3 by default for portability
```
## Writing a project.cpp
Crafter Build loads a `project.cpp` from the current directory. The file exports one function that returns a populated `Configuration`:
```cpp
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
Configuration cfg;
cfg.path = "./";
cfg.name = "myapp";
cfg.outputName = "myapp";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
std::array<fs::path, 1> ifaces = { "interfaces/Hello" };
std::array<fs::path, 2> impls = { "implementations/Hello", "implementations/main" };
cfg.GetInterfacesAndImplementations(ifaces, impls);
return cfg;
}
```
Run `crafter-build` in that directory; outputs land at `bin/myapp-<target>-<march>/myapp` and intermediates at `build/myapp-<target>-<march>/`.
## Dependencies
Three kinds, all fetched/built incrementally:
**Cross-project Crafter sub-projects** — point `cfg.dependencies` at another `Configuration*`. Cross-project module imports (`import OtherProject;`) are tracked per-import: only the modules that actually import a changed dep rebuild.
**External git+cmake deps** — for things like glslang. Declare in `cfg.externalDependencies`:
```cpp
ExternalDependency& glslang = cfg.externalDependencies.emplace_back();
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "glslang", "OSDependent" };
```
Clones live at `~/.cache/crafter.build/external/<name>-<hash>/`, keyed by `(url, branch, commit, options)` so different projects sharing the same dep with the same spec reuse the clone and cmake build, while projects with diverging specs each get their own cache entry. The clone is resilient to partial-clone recovery and CMake reconfigure is triggered automatically on `options` changes.
**Header-only git deps** — same as cmake deps but `builder = ExternalBuilder::None`. Just clones and propagates `-I` flags.
## Install layout
```
<prefix>/bin/crafter-build # the executable
<prefix>/share/crafter-build/*.cppm # module sources (rebuilt on user machine on first run)
~/.cache/crafter.build/<target>-<march>/ # std.pcm + Crafter.Build*.pcm, built locally
```
Sources ship instead of PCMs because libc++ ABI varies between machines — the user's machine builds its own PCMs against its own libc++ on first run (~6-8s one-time cost), making the install resilient to libc++ version differences across distros.
## Build incrementality
Per-import precise tracking for both within-project and cross-project module dependencies:
- Touch `lib/Hello.cppm` → only consumers of `Hello` rebuild.
- Touch `lib/Other.cppm` → only consumers of `Other` rebuild.
- External CMake dep produces fresh `.a` files → whole project rebuilds (deliberately coarse — cmake-dep changes are rare).
Diamond deps (`A → {B, C}; B → X; C → X`) build `X` exactly once via a `std::shared_future<BuildResult>` cache.
## Architecture
- **Modules**: `Crafter.Build:Shader` / `:Platform` / `:Interface` / `:Implementation` / `:External` / `:Clang` partitions, re-exported by the `Crafter.Build` umbrella.
- **Build process**: parallel — interface PCMs, implementation `.o` files, external dep clones, dep-config recursive builds, and shader compilations all spawn threads, sync at well-defined join points.
- **`LoadProject`** compiles `project.cpp` to `<project>/build/project.so` and `dlopen`s it. The host exe is linked with `-Wl,--export-dynamic` so the project's undefined symbols resolve against the running binary.
## Compatibility / portability
- Linux: x86_64. Tested on Arch in a clean container.
- Windows: bootstrap works (build.cmd produces working exe); the `LoadProject` path needs the in-progress DLL+launcher refactor to support cross-EXE/DLL symbol resolution.
The `CRAFTER_BUILD_MARCH` and `CRAFTER_BUILD_MTUNE` env vars override the default `-march=native` / `-mtune=native` for distro CI builds:
```bash
CRAFTER_BUILD_MARCH=x86-64-v3 CRAFTER_BUILD_MTUNE=generic ./build.sh
```
## Roadmap
- [ ] Windows DLL+launcher refactor (in progress)
- [ ] Configuration inheritance (`extends:`-style merging from base configs)
- [ ] Test runner (dlopen-based test loading)
- [ ] Target-triple auto-detection (`clang -print-target-triple`)
- [ ] Prefer system glslang when available, fall back to git+cmake
- [ ] Order-preserving deduplication of `BuildResult.libs` (current `unordered_set` doesn't preserve link order — only matters for cyclic static libs)
## License
LGPL-3.0-only. See [LICENSE](LICENSE).

View file

@ -1,46 +1,84 @@
@echo OFF @echo OFF
mkdir build mkdir build 2>nul
mkdir bin mkdir bin 2>nul
mkdir bin\executable-windows-msvc mkdir share\crafter-build 2>nul
git clone https://github.com/KhronosGroup/glslang.git .\build\glslang copy /Y interfaces\Crafter.Build.cppm share\crafter-build\
copy /Y interfaces\Crafter.Build-Shader.cppm share\crafter-build\
copy /Y interfaces\Crafter.Build-Platform.cppm share\crafter-build\
copy /Y interfaces\Crafter.Build-Interface.cppm share\crafter-build\
copy /Y interfaces\Crafter.Build-Implementation.cppm share\crafter-build\
copy /Y interfaces\Crafter.Build-External.cppm share\crafter-build\
copy /Y interfaces\Crafter.Build-Clang.cppm share\crafter-build\
if not exist .\build\glslang\NUL git clone https://github.com/KhronosGroup/glslang.git .\build\glslang
set useLibcCommon=-nostdinc++ -nostdlib++ set useLibcCommon=-nostdinc++ -nostdlib++
set useLibcLinker=-L %LIBCXX_DIR%\lib -lc++ %useLibcCommon% set useLibcLinker=-L %LIBCXX_DIR%\lib -lc++ %useLibcCommon%
set useLibcSource=-isystem %LIBCXX_DIR%\include\c++\v1 %useLibcCommon% set useLibcSource=-isystem %LIBCXX_DIR%\include\c++\v1 %useLibcCommon%
set buildDir=%CD%\build set buildDir=%CD%\build
cd .\build\glslang if "%CRAFTER_BUILD_MARCH%"=="" set CRAFTER_BUILD_MARCH=native
if "%CRAFTER_BUILD_MTUNE%"=="" set CRAFTER_BUILD_MTUNE=native
cmake -G Ninja -B build -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="%useLibcSource%" -DCMAKE_EXE_LINKER_FLAGS="%useLibcLinker%" -DCMAKE_SHARED_LINKER_FLAGS="%useLibcLinker%" -DCMAKE_BUILD_TYPE=Release -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY="%buildDir%" -DCMAKE_LIBRARY_OUTPUT_DIRECTORY="%buildDir%" -DENABLE_OPT=OFF cd .\build\glslang
cmake -G Ninja -B build ^
-DCMAKE_C_COMPILER=clang ^
-DCMAKE_CXX_COMPILER=clang++ ^
-DCMAKE_CXX_FLAGS="%useLibcSource%" ^
-DCMAKE_EXE_LINKER_FLAGS="%useLibcLinker%" ^
-DCMAKE_SHARED_LINKER_FLAGS="%useLibcLinker%" ^
-DCMAKE_BUILD_TYPE=Release ^
-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY="%buildDir%" ^
-DCMAKE_LIBRARY_OUTPUT_DIRECTORY="%buildDir%" ^
-DENABLE_OPT=OFF
cmake --build build --config Release cmake --build build --config Release
cd ..\..\ cd ..\..\
set common_options=-I.\build\glslang -std=c++26 -O3 -march=native -mtune=native -fprebuilt-module-path=.\build -D CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc -D CRAFTER_BUILD_CONFIGURATION_TYPE_EXECUTABLE %useLibcSource% -c set common_options=-I.\build\glslang -std=c++26 -O3 -march=%CRAFTER_BUILD_MARCH% -mtune=%CRAFTER_BUILD_MTUNE% -fprebuilt-module-path=.\build -D CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc -D CRAFTER_BUILD_CONFIGURATION_TYPE_EXECUTABLE -D _CRT_SECURE_NO_WARNINGS %useLibcSource% -c
clang++ %useLibcSource% -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile %LIBCXX_DIR%\modules\c++\v1\std.cppm -o .\build\std.pcm clang++ %useLibcSource% -std=c++26 -march=%CRAFTER_BUILD_MARCH% -mtune=%CRAFTER_BUILD_MTUNE% -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile %LIBCXX_DIR%\modules\c++\v1\std.cppm -o .\build\std.pcm
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-CompileStatus.cppm -o .\build\Crafter.Build-CompileStatus.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Command.cppm -o .\build\Crafter.Build-Command.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Shader.cppm -o .\build\Crafter.Build-Shader.o clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Shader.cppm -o .\build\Crafter.Build-Shader.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Module.cppm -o .\build\Crafter.Build-Module.o clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Platform.cppm -o .\build\Crafter.Build-Platform.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Interface.cppm -o .\build\Crafter.Build-Interface.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Implementation.cppm -o .\build\Crafter.Build-Implementation.o clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Implementation.cppm -o .\build\Crafter.Build-Implementation.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Configuration.cppm -o .\build\Crafter.Build-Configuration.o clang++ %common_options% -fmodule-output interfaces\Crafter.Build-External.cppm -o .\build\Crafter.Build-External.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Test.cppm -o .\build\Crafter.Build-Test.o clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Clang.cppm -o .\build\Crafter.Build-Clang.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build-Project.cppm -o .\build\Crafter.Build-Project.o
clang++ %common_options% -fmodule-output interfaces\Crafter.Build.cppm -o .\build\Crafter.Build.o clang++ %common_options% -fmodule-output interfaces\Crafter.Build.cppm -o .\build\Crafter.Build.o
clang++ %common_options% .\implementations\Crafter.Build-Command.cpp -o .\build\Crafter.Build-Command_impl.o
clang++ %common_options% .\implementations\Crafter.Build-Test.cpp -o .\build\Crafter.Build-Test_impl.o
clang++ %common_options% .\implementations\Crafter.Build-Implementation.cpp -o .\build\Crafter.Build-Implementation_impl.o
clang++ %common_options% .\implementations\Crafter.Build-Shader.cpp -o .\build\Crafter.Build-Shader_impl.o clang++ %common_options% .\implementations\Crafter.Build-Shader.cpp -o .\build\Crafter.Build-Shader_impl.o
clang++ %common_options% .\implementations\Crafter.Build-Module.cpp -o .\build\Crafter.Build-Module_impl.o clang++ %common_options% .\implementations\Crafter.Build-Platform.cpp -o .\build\Crafter.Build-Platform_impl.o
clang++ %common_options% .\implementations\Crafter.Build-Configuration.cpp -o .\build\Crafter.Build-Configuration_impl.o clang++ %common_options% .\implementations\Crafter.Build-Interface.cpp -o .\build\Crafter.Build-Interface_impl.o
clang++ %common_options% .\implementations\Crafter.Build-Project.cpp -o .\build\Crafter.Build-Project_impl.o clang++ %common_options% .\implementations\Crafter.Build-Implementation.cpp -o .\build\Crafter.Build-Implementation_impl.o
clang++ %common_options% .\implementations\Crafter.Build-External.cpp -o .\build\Crafter.Build-External_impl.o
clang++ %common_options% .\implementations\Crafter.Build-Clang.cpp -o .\build\Crafter.Build-Clang_impl.o
clang++ %common_options% .\implementations\main.cpp -o .\build\main.o clang++ %common_options% .\implementations\main.cpp -o .\build\main.o
clang++ %useLibcLinker% -std=c++26 -O3 -march=native -mtune=native -L.\build -fuse-ld=lld -lSPIRV -GenericCodeGen -lglslang -lOSDependent -lMachineIndependent -lglslang-default-resource-limits .\build\Crafter.Build-Command.o .\build\Crafter.Build-CompileStatus.o .\build\Crafter.Build-Shader.o .\build\Crafter.Build-Module.o .\build\Crafter.Build-Configuration.o .\build\Crafter.Build-Project.o .\build\Crafter.Build.o .\build\Crafter.Build-Command_impl.o .\build\Crafter.Build-Shader_impl.o .\build\Crafter.Build-Module_impl.o .\build\Crafter.Build-Configuration_impl.o .\build\Crafter.Build-Project_impl.o .\build\Crafter.Build-Implementation.o .\build\Crafter.Build-Implementation_impl.o .\build\Crafter.Build-Test_impl.o .\build\Crafter.Build-Test.o .\build\main.o -o .\bin\executable-windows-msvc\crafter-build.exe REM Step 1: link all impl .o files into crafter-build.dll, generating crafter-build.lib import lib
clang++ %useLibcLinker% -shared -std=c++26 -O3 -march=%CRAFTER_BUILD_MARCH% -mtune=%CRAFTER_BUILD_MTUNE% -L.\build -fuse-ld=lld ^
-Wl,-export-all-symbols ^
-Wl,/IMPLIB:.\bin\crafter-build.lib ^
-lSPIRV -lGenericCodeGen -lglslang -lOSDependent -lMachineIndependent -lglslang-default-resource-limits ^
.\build\Crafter.Build-Shader.o ^
.\build\Crafter.Build-Platform.o ^
.\build\Crafter.Build-Interface.o ^
.\build\Crafter.Build-Implementation.o ^
.\build\Crafter.Build-External.o ^
.\build\Crafter.Build-Clang.o ^
.\build\Crafter.Build.o ^
.\build\Crafter.Build-Shader_impl.o ^
.\build\Crafter.Build-Platform_impl.o ^
.\build\Crafter.Build-Interface_impl.o ^
.\build\Crafter.Build-Implementation_impl.o ^
.\build\Crafter.Build-External_impl.o ^
.\build\Crafter.Build-Clang_impl.o ^
-o .\bin\crafter-build.dll
copy "%LIBCXX_DIR%\lib\c++.dll" ".\bin\executable-windows-msvc\c++.dll" REM Step 2: link the launcher exe against crafter-build.lib
xcopy "binlib\*" "bin\" /E /I /Y /Q clang++ %useLibcLinker% -std=c++26 -O3 -march=%CRAFTER_BUILD_MARCH% -mtune=%CRAFTER_BUILD_MTUNE% -L.\bin -fuse-ld=lld ^
rmdir /S /Q "build" .\build\main.o ^
.\bin\crafter-build.lib ^
-o .\bin\crafter-build.exe
copy /Y "%LIBCXX_DIR%\lib\c++.dll" ".\bin\c++.dll"

View file

@ -1,6 +1,14 @@
mkdir build mkdir -p build
mkdir bin mkdir -p bin
mkdir bin/executable-linux-gnu mkdir -p share/crafter-build
cp interfaces/Crafter.Build.cppm share/crafter-build/
cp interfaces/Crafter.Build-Shader.cppm share/crafter-build/
cp interfaces/Crafter.Build-Platform.cppm share/crafter-build/
cp interfaces/Crafter.Build-Interface.cppm share/crafter-build/
cp interfaces/Crafter.Build-Implementation.cppm share/crafter-build/
cp interfaces/Crafter.Build-External.cppm share/crafter-build/
cp interfaces/Crafter.Build-Clang.cppm share/crafter-build/
git clone https://github.com/KhronosGroup/glslang.git ./build/glslang git clone https://github.com/KhronosGroup/glslang.git ./build/glslang
@ -21,30 +29,46 @@ cmake -B build \
cmake --build build --config Release cmake --build build --config Release
cd ../../ cd ../../
common_options="-stdlib=libc++ -I./build/glslang -std=c++26 -O3 -march=native -mtune=native -fprebuilt-module-path=./build -D CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu -D CRAFTER_BUILD_CONFIGURATION_TYPE_EXECUTABLE -c" MARCH="${CRAFTER_BUILD_MARCH:-native}"
MTUNE="${CRAFTER_BUILD_MTUNE:-native}"
clang++ -std=c++26 -stdlib=libc++ -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile /usr/share/libc++/v1/std.cppm -o ./build/std.pcm common_options="--target=x86_64-pc-linux-gnu -stdlib=libc++ -I./build/glslang -std=c++26 -O3 -march=$MARCH -mtune=$MTUNE -fprebuilt-module-path=./build -D CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu -D CRAFTER_BUILD_CONFIGURATION_TYPE_EXECUTABLE -c"
clang++ --target=x86_64-pc-linux-gnu -std=c++26 -stdlib=libc++ -O3 -march=$MARCH -mtune=$MTUNE -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile /usr/share/libc++/v1/std.cppm -o ./build/std.pcm
clang++ $common_options -fmodule-output interfaces/Crafter.Build-CompileStatus.cppm -o ./build/Crafter.Build-CompileStatus.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Command.cppm -o ./build/Crafter.Build-Command.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Shader.cppm -o ./build/Crafter.Build-Shader.o clang++ $common_options -fmodule-output interfaces/Crafter.Build-Shader.cppm -o ./build/Crafter.Build-Shader.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Module.cppm -o ./build/Crafter.Build-Module.o clang++ $common_options -fmodule-output interfaces/Crafter.Build-Platform.cppm -o ./build/Crafter.Build-Platform.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Interface.cppm -o ./build/Crafter.Build-Interface.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Implementation.cppm -o ./build/Crafter.Build-Implementation.o clang++ $common_options -fmodule-output interfaces/Crafter.Build-Implementation.cppm -o ./build/Crafter.Build-Implementation.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Configuration.cppm -o ./build/Crafter.Build-Configuration.o clang++ $common_options -fmodule-output interfaces/Crafter.Build-External.cppm -o ./build/Crafter.Build-External.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Test.cppm -o ./build/Crafter.Build-Test.o clang++ $common_options -fmodule-output interfaces/Crafter.Build-Clang.cppm -o ./build/Crafter.Build-Clang.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build-Project.cppm -o ./build/Crafter.Build-Project.o
clang++ $common_options -fmodule-output interfaces/Crafter.Build.cppm -o ./build/Crafter.Build.o clang++ $common_options -fmodule-output interfaces/Crafter.Build.cppm -o ./build/Crafter.Build.o
clang++ $common_options ./implementations/Crafter.Build-Command.cpp -o ./build/Crafter.Build-Command_impl.o
clang++ $common_options ./implementations/Crafter.Build-Test.cpp -o ./build/Crafter.Build-Test_impl.o
clang++ $common_options ./implementations/Crafter.Build-Implementation.cpp -o ./build/Crafter.Build-Implementation_impl.o
clang++ $common_options ./implementations/Crafter.Build-Shader.cpp -o ./build/Crafter.Build-Shader_impl.o clang++ $common_options ./implementations/Crafter.Build-Shader.cpp -o ./build/Crafter.Build-Shader_impl.o
clang++ $common_options ./implementations/Crafter.Build-Module.cpp -o ./build/Crafter.Build-Module_impl.o clang++ $common_options ./implementations/Crafter.Build-Platform.cpp -o ./build/Crafter.Build-Platform_impl.o
clang++ $common_options ./implementations/Crafter.Build-Configuration.cpp -o ./build/Crafter.Build-Configuration_impl.o clang++ $common_options ./implementations/Crafter.Build-Interface.cpp -o ./build/Crafter.Build-Interface_impl.o
clang++ $common_options ./implementations/Crafter.Build-Project.cpp -o ./build/Crafter.Build-Project_impl.o clang++ $common_options ./implementations/Crafter.Build-Implementation.cpp -o ./build/Crafter.Build-Implementation_impl.o
clang++ $common_options ./implementations/Crafter.Build-External.cpp -o ./build/Crafter.Build-External_impl.o
clang++ $common_options ./implementations/Crafter.Build-Clang.cpp -o ./build/Crafter.Build-Clang_impl.o
clang++ $common_options ./implementations/main.cpp -o ./build/main.o clang++ $common_options ./implementations/main.cpp -o ./build/main.o
clang++ -std=c++26 -stdlib=libc++ -O3 -march=native -mtune=native -L./build -fuse-ld=lld -lSPIRV -GenericCodeGen -lglslang -lOSDependent -lMachineIndependent -lglslang-default-resource-limits ./build/Crafter.Build-Command.o ./build/Crafter.Build-CompileStatus.o ./build/Crafter.Build-Shader.o ./build/Crafter.Build-Module.o ./build/Crafter.Build-Configuration.o ./build/Crafter.Build-Project.o ./build/Crafter.Build.o ./build/Crafter.Build-Command_impl.o ./build/Crafter.Build-Shader_impl.o ./build/Crafter.Build-Module_impl.o ./build/Crafter.Build-Configuration_impl.o ./build/Crafter.Build-Project_impl.o ./build/Crafter.Build-Implementation.o ./build/Crafter.Build-Implementation_impl.o ./build/Crafter.Build-Test_impl.o ./build/Crafter.Build-Test.o ./build/main.o -o ./bin/executable-linux-gnu/crafter-build clang++ -std=c++26 -stdlib=libc++ -O3 -march=$MARCH -mtune=$MTUNE -fuse-ld=lld \
-Wl,--export-dynamic \
cp -r binlib/* bin/ -L./build \
rm -rf build ./build/Crafter.Build-Shader.o \
./build/Crafter.Build-Platform.o \
./build/Crafter.Build-Interface.o \
./build/Crafter.Build-Implementation.o \
./build/Crafter.Build-External.o \
./build/Crafter.Build-Clang.o \
./build/Crafter.Build.o \
./build/Crafter.Build-Shader_impl.o \
./build/Crafter.Build-Platform_impl.o \
./build/Crafter.Build-Interface_impl.o \
./build/Crafter.Build-Implementation_impl.o \
./build/Crafter.Build-External_impl.o \
./build/Crafter.Build-Clang_impl.o \
./build/main.o \
-lSPIRV -lGenericCodeGen -lglslang -lOSDependent -lMachineIndependent -lglslang-default-resource-limits \
-ldl \
-o ./bin/crafter-build

View file

@ -1,21 +0,0 @@
# WASI Example
## Description
This example demonstrates how to build a WASI project
## Expected Result
Hello World! appears in the browser console.
## How to Run
```bash
crafter-build build executable
run.sh
```
and go to `http://localhost:8080/`
if caddy is not installed you can use your favorite static file server instead

View file

@ -1,5 +0,0 @@
import std;
int main() {
std::println("Hello World!");
}

View file

@ -1,10 +0,0 @@
{
"name": "main",
"configurations": [
{
"name": "executable",
"implementations": ["main"],
"target": "wasm32-wasi"
}
]
}

View file

@ -1 +0,0 @@
caddy file-server --listen :8080 --root bin/executable

View file

@ -20,12 +20,173 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
export module Crafter.Build:Clang_impl; export module Crafter.Build:Clang_impl;
import std; import std;
import :Clang; import :Clang;
import :Platform;
namespace fs = std::filesystem; namespace fs = std::filesystem;
using namespace Crafter;
BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& depSet, std::mutex& depMutex) {
fs::path pwd = std::filesystem::path cwd = std::filesystem::current_path(); void Configuration::GetInterfacesAndImplementations(std::span<fs::path> interfaces, std::span<fs::path> implementations) {
fs::path buildDir = pwd/"build"; auto resolveImport = [this](const std::string& importName,
fs::path outputDir = pwd/"bin"; std::vector<Module*>& localDeps,
std::vector<std::pair<Module*, fs::path>>& externalDeps) -> bool {
for(const std::unique_ptr<Module>& interface : this->interfaces) {
if(interface->name == importName) {
localDeps.push_back(interface.get());
return true;
}
}
for(Configuration* depCfg : this->dependencies) {
for(const std::unique_ptr<Module>& depInterface : depCfg->interfaces) {
if(depInterface->name == importName) {
fs::path depPcmPath = (depCfg->PcmDir() / depInterface->path.filename()).string() + ".pcm";
externalDeps.emplace_back(depInterface.get(), std::move(depPcmPath));
return true;
}
}
}
return false;
};
std::vector<std::tuple<fs::path, std::string, ModulePartition*, Module*>> tempModulePaths = std::vector<std::tuple<fs::path, std::string, ModulePartition*, Module*>>(interfaces.size());
for(std::uint16_t i = 0; i < interfaces.size(); i++){
fs::path file = path / interfaces[i];
file += ".cppm";
std::ifstream t(file);
std::stringstream buffer;
buffer << t.rdbuf();
std::string fileContent = buffer.str();
fileContent = std::regex_replace(fileContent, std::regex(R"(//[^\n]*)"), "");
fileContent = std::regex_replace(fileContent, std::regex(R"(/\*.*?\*/)"), "");
tempModulePaths[i] = {file, fileContent, nullptr, nullptr};
}
std::erase_if(tempModulePaths, [this](std::tuple<fs::path, std::string, ModulePartition*, Module*>& file) {
std::smatch match;
if (std::regex_search(std::get<1>(file), match, std::regex(R"(export module ([a-zA-Z0-9_\.\-]+);)"))) {
std::get<0>(file).replace_extension("");
this->interfaces.push_back(std::make_unique<Module>(std::move(match[1].str()), std::move(std::get<0>(file))));
return true;
} else {
return false;
}
});
for(std::uint16_t i = 0; i < tempModulePaths.size(); i++) {
std::smatch match;
if (std::regex_search(std::get<1>(tempModulePaths[i]), match, std::regex(R"(export module ([a-zA-Z_0-9\.\-]+):([a-zA-Z_0-9\.\-]+);)"))) {
for(const std::unique_ptr<Module>& modulee : this->interfaces) {
if(modulee->name == match[1]) {
std::string name = match[2].str();
fs::path pthCpy = std::get<0>(tempModulePaths[i]);
pthCpy.replace_extension("");
std::unique_ptr<ModulePartition> partition = std::make_unique<ModulePartition>(std::move(name), std::move(pthCpy));
std::get<2>(tempModulePaths[i]) = partition.get();
modulee->partitions.push_back(std::move(partition));
std::get<3>(tempModulePaths[i]) = modulee.get();
goto next;
}
}
throw std::runtime_error(std::format("Module {} not found, referenced in {}", match[1].str(), std::get<0>(tempModulePaths[i]).string()));
} else {
throw std::runtime_error(std::format("No module declaration found in {}", std::get<0>(tempModulePaths[i]).string()));
}
next:;
}
for(std::tuple<fs::path, std::string, ModulePartition*, Module*>& file : tempModulePaths) {
ModulePartition* partition = std::get<2>(file);
Module* parentModule = std::get<3>(file);
const std::string& fileContent = std::get<1>(file);
std::regex partitionPattern(R"(import :([a-zA-Z_\-0-9\.]+);)");
std::sregex_iterator currentMatch(fileContent.begin(), fileContent.end(), partitionPattern);
std::sregex_iterator lastMatch;
while (currentMatch != lastMatch) {
std::smatch match = *currentMatch;
for(std::unique_ptr<ModulePartition>& sibling : parentModule->partitions) {
if(sibling->name == match[1]) {
partition->partitionDependencies.push_back(sibling.get());
goto next2;
}
}
throw std::runtime_error(std::format("imported partition {}:{} not found, referenced in {}", parentModule->name, match[1].str(), std::get<0>(file).string()));
next2: ++currentMatch;
}
std::regex modulePattern(R"(import ([a-zA-Z_0-9\.\-]+);)");
std::sregex_iterator modCurrent(fileContent.begin(), fileContent.end(), modulePattern);
while (modCurrent != lastMatch) {
std::smatch match = *modCurrent;
resolveImport(match[1].str(), partition->moduleDependencies, partition->externalModuleDependencies);
++modCurrent;
}
}
for(const fs::path& tempFile : implementations) {
fs::path file = path / tempFile;
file += ".cpp";
std::ifstream t(file);
std::stringstream buffer;
buffer << t.rdbuf();
std::string fileContent = buffer.str();
fileContent = std::regex_replace(fileContent, std::regex(R"(//[^\n]*)"), "");
fileContent = std::regex_replace(fileContent, std::regex(R"(/\*.*?\*/)"), "");
std::smatch match;
fs::path fileCopy = file;
fileCopy.replace_extension("");
Implementation& implementation = this->implementations.emplace_back(std::move(fileCopy));
if (std::regex_search(fileContent, match, std::regex(R"(module ([a-zA-Z0-9_\.\-]+)(:[a-zA-Z0-9_\.\-]+)?\s*;)"))) {
bool isPartitionImpl = match[2].length() > 0;
for(const std::unique_ptr<Module>& interface : this->interfaces) {
if(interface->name == match[1]) {
if (!isPartitionImpl) {
implementation.moduleDependencies.push_back(interface.get());
}
std::regex partitionPattern(R"(import :([a-zA-Z_\-0-9\.]+);)");
std::sregex_iterator currentMatch(fileContent.begin(), fileContent.end(), partitionPattern);
std::sregex_iterator lastMatch;
while (currentMatch != lastMatch) {
std::smatch match2 = *currentMatch;
for(const std::unique_ptr<ModulePartition>& partition : interface->partitions) {
if(partition->name == match2[1]) {
implementation.partitionDependencies.push_back(partition.get());
goto next3;
}
}
throw std::runtime_error(std::format("imported partition {}:{} not found, referenced in {}", match[1].str(), match2[1].str(), file.string()));
next3: ++currentMatch;
}
std::regex modulePattern(R"(import ([a-zA-Z_0-9\.\-]+);)");
std::sregex_iterator modCurrent(fileContent.begin(), fileContent.end(), modulePattern);
while (modCurrent != lastMatch) {
std::smatch match2 = *modCurrent;
if (match2[1] != match[1]) {
resolveImport(match2[1].str(), implementation.moduleDependencies, implementation.externalModuleDependencies);
}
++modCurrent;
}
goto next4;
}
}
throw std::runtime_error(std::format("Module {} not found not found, referenced in {}", match[1].str(), file.string()));
next4:;
} else {
std::regex pattern(R"(import ([a-zA-Z_\-0-9\.]+);)");
std::sregex_iterator currentMatch(fileContent.begin(), fileContent.end(), pattern);
std::sregex_iterator lastMatch;
while (currentMatch != lastMatch) {
std::smatch match2 = *currentMatch;
resolveImport(match2[1].str(), implementation.moduleDependencies, implementation.externalModuleDependencies);
++currentMatch;
}
}
}
}
BuildResult Crafter::Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex) {
fs::path buildDir = config.path/"build"/std::format("{}-{}-{}", config.name, config.target, config.march);
fs::path outputDir = config.path/"bin"/std::format("{}-{}-{}", config.name, config.target, config.march);
if (!fs::exists(buildDir)) { if (!fs::exists(buildDir)) {
fs::create_directories(buildDir); fs::create_directories(buildDir);
@ -35,6 +196,8 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
fs::create_directories(outputDir); fs::create_directories(outputDir);
} }
BuildResult buildResult;
std::vector<std::thread> threads; std::vector<std::thread> threads;
threads.reserve(config.shaders.size() + 1 + config.interfaces.size() + config.implementations.size()); threads.reserve(config.shaders.size() + 1 + config.interfaces.size() + config.implementations.size());
@ -106,7 +269,23 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
} }
}); });
fs::path stdPcmDir = GetCacheDir()/config.target/"-"/config.march; std::vector<ExternalBuildResult> externalResults(config.externalDependencies.size());
std::vector<std::thread> externalThreads;
externalThreads.reserve(config.externalDependencies.size());
for (std::size_t i = 0; i < config.externalDependencies.size(); ++i) {
externalThreads.emplace_back([&, i]() {
if (buildCancelled.load(std::memory_order_relaxed)) return;
externalResults[i] = BuildExternal(config.externalDependencies[i], buildCancelled);
if (!externalResults[i].error.empty()) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = externalResults[i].error;
}
}
});
}
fs::path stdPcmDir = GetCacheDir()/(config.target+"-"+config.march);
if (!fs::exists(stdPcmDir)) { if (!fs::exists(stdPcmDir)) {
fs::create_directories(stdPcmDir); fs::create_directories(stdPcmDir);
@ -114,7 +293,10 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
std::string stdPcmResult = BuildStdPcm(config, stdPcmDir/"std.pcm"); std::string stdPcmResult = BuildStdPcm(config, stdPcmDir/"std.pcm");
if(!stdPcmResult.empty()) { if(!stdPcmResult.empty()) {
return {result, false, {}}; buildCancelled.store(true);
for(std::thread& thread : threads) thread.join();
for(std::thread& thread : externalThreads) thread.join();
return {stdPcmResult, false, {}};
} }
fs::path pcmDir; fs::path pcmDir;
@ -125,55 +307,90 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
pcmDir = buildDir; pcmDir = buildDir;
} }
std::string command = std::format("{} --target={} -march={} -mtune={} -std=c++26 -D CRAFTER_BUILD_CONFIGURATION_TARGET=\\\"{}\\\" -D CRAFTER_BUILD_CONFIGURATION_TARGET_{} -fprebuilt-module-path={} -fprebuilt-module-path={}", GetBaseCommand(), config.target, config.march, config.mtune, editedTarget, editedTarget, stdPcm.parent_path().string(), stdPcmDir); fs::copy_file(stdPcmDir/"std.pcm", pcmDir/"std.pcm", fs::copy_options::update_existing);
if(config.type == CRAFTER_CONFIGURATION_TYPE_SHARED_LIBRARY) { std::string editedTarget = config.target;
std::replace(editedTarget.begin(), editedTarget.end(), '-', '_');
std::string command = std::format("{} --target={} -march={} -mtune={} -std=c++26 -D CRAFTER_BUILD_CONFIGURATION_TARGET=\\\"{}\\\" -D CRAFTER_BUILD_CONFIGURATION_TARGET_{} -fprebuilt-module-path={} -fprebuilt-module-path={}", GetBaseCommand(config), config.target, config.march, config.mtune, editedTarget, editedTarget, stdPcmDir.string(), pcmDir.string());
if(config.type == ConfigurationType::LibraryDynamic) {
#ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu #ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu
command += " -fPIC -D CRAFTER_BUILD_CONFIGURATION_TYPE_SHARED_LIBRARY"; command += " -fPIC -D CRAFTER_BUILD_CONFIGURATION_TYPE_SHARED_LIBRARY";
#endif #endif
#if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32) #if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32)
command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_SHARED_LIBRARY"; command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_SHARED_LIBRARY";
#endif #endif
} else if(config.type == CRAFTER_CONFIGURATION_TYPE_LIBRARY) { } else if(config.type == ConfigurationType::Executable) {
command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_LIBRARY";
} else {
command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_EXECUTABLE"; command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_EXECUTABLE";
} else {
command += " -D CRAFTER_BUILD_CONFIGURATION_TYPE_LIBRARY";
} }
std::string files; std::string files;
std::unordered_set<std::string> libSet; std::unordered_set<std::string> libSet;
std::mutex fileMutex; std::mutex fileMutex;
std::vector<std::thread> depThreads; std::vector<std::thread> depThreads;
depThreads.reserve(config.dependencies); depThreads.reserve(config.dependencies.size());
std::atomic<bool> repack(false); std::atomic<bool> repack(false);
for(const Dependency& dep : config.dependencies) { for(Configuration* dep : config.dependencies) {
for (const auto& entry : fs::recursive_directory_iterator(dep.path)) { for (const auto& entry : fs::recursive_directory_iterator(dep->path)) {
if (entry.is_directory() && entry.path().filename() == "include") { if (entry.is_directory() && entry.path().filename() == "include") {
command += " -I" + entry.path().string(); command += " -I" + entry.path().string();
} }
} }
command += std::format(" -I{} -fprebuilt-module-path={}", dep.path.string(), (dep.path/"bin"/(config.target+config.march)).string()); command += std::format(" -I{} -fprebuilt-module-path={}", dep->path.string(), dep->PcmDir().string());
depThreads.emplace_back([&, dep](){
try {
if (buildCancelled.load(std::memory_order_relaxed)) return;
std::shared_ptr<std::promise<BuildResult>> promise;
std::shared_future<BuildResult> resultFuture;
bool isBuilder = false;
depThreads.emplace_back([&](){
if (!buildCancelled.load(std::memory_order_relaxed)) {
depMutex.lock(); depMutex.lock();
if(!depSet.contains(dep.path)) { auto it = depResults.find(dep->path);
depSet.insert(dep.path); if (it == depResults.end()) {
depMutex.unlock(); isBuilder = true;
BuildResult result = Build(dep.path, depSet, depMutex); promise = std::make_shared<std::promise<BuildResult>>();
fileMutex.lock(); resultFuture = promise->get_future().share();
libSet.merge(result.libs); depResults.emplace(dep->path, resultFuture);
fileMutex.unlock();
if (buildCancelled.compare_exchange_strong(false, true)) {
buildError = std::move(result.result);
}
if(result.repack) {
repack = true;
}
} else { } else {
depMutex.unlock(); resultFuture = it->second;
}
depMutex.unlock();
if (isBuilder) {
BuildResult built;
try {
built = Build(*dep, depResults, depMutex);
} catch (...) {
promise->set_exception(std::current_exception());
throw;
}
promise->set_value(std::move(built));
}
const BuildResult& result = resultFuture.get();
fileMutex.lock();
for (const std::string& lib : result.libs) libSet.insert(lib);
fileMutex.unlock();
if (!result.result.empty()) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = result.result;
}
}
if (result.repack) {
repack = true;
}
} catch (const std::exception& e) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::format("dep build for {} threw: {}", dep->path.string(), e.what());
} }
} }
}); });
@ -187,6 +404,10 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
} }
} }
for(const std::string& flag : config.compileFlags) {
command += " " + flag;
}
std::string cmakeBuildType; std::string cmakeBuildType;
if(config.debug) { if(config.debug) {
@ -197,12 +418,12 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
command += " -O3"; command += " -O3";
} }
for (const fs::path& cFile : config.c_files) { for (const fs::path& cFile : config.cFiles) {
files += std::format(" {}_source.o ", (buildDir / cFile.filename()).string()); files += std::format(" {}_source.o ", (buildDir / cFile.filename()).string());
const std::string objPath = (buildDir / cFile.filename()).string() + "_source.o"; const std::string objPath = (buildDir / cFile.filename()).string() + "_source.o";
const std::string srcPath = cFile.string() + ".c"; const std::string srcPath = cFile.string() + ".c";
if (!fs::exists(objPath) || fs::last_write_time(srcPath) > fs::last_write_time(objPath)) { if (!fs::exists(objPath) || fs::last_write_time(srcPath) > fs::last_write_time(objPath)) {
threads.emplace_back([&cFile, &buildDir, &buildError, &buildCancelled]() { threads.emplace_back([&cFile, &buildDir, &buildError, &buildCancelled, &config]() {
if (buildCancelled.load(std::memory_order_relaxed)) return; if (buildCancelled.load(std::memory_order_relaxed)) return;
std::string result = RunCommand(std::format("clang {}.c --target={} -march={} -mtune={} -O3 -c -o {}_source.o", cFile.string(), config.target, config.march, config.mtune, (buildDir / cFile.filename()).string())); std::string result = RunCommand(std::format("clang {}.c --target={} -march={} -mtune={} -O3 -c -o {}_source.o", cFile.string(), config.target, config.march, config.mtune, (buildDir / cFile.filename()).string()));
@ -238,92 +459,70 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
for(std::thread& thread : depThreads) { for(std::thread& thread : depThreads) {
thread.join(); thread.join();
} }
for(std::thread& thread : externalThreads) {
thread.join();
}
if(buildCancelled.load()) { if(buildCancelled.load()) {
for(std::thread& thread : threads) thread.join();
return {buildError, false, {}}; return {buildError, false, {}};
} }
std::vector<std::unique_ptr<Module>> interfaces; if(repack.load()) {
std::vector<std::tuple<fs::path, std::string, ModulePartition*, Module*>> tempModulePaths = std::vector<std::tuple<fs::path, std::string, ModulePartition*, Module*>>(config.interfaces.size()); buildResult.repack = true;
for(std::uint16_t i = 0; i < config.interfaces.size(); i++){ }
const std::filesystem::path file = workingDir / (config.interfaces[i]+".cppm"); buildResult.libs = std::move(libSet);
std::ifstream t(file); for(const std::string& flag : config.linkFlags) {
std::stringstream buffer; buildResult.libs.insert(flag);
buffer << t.rdbuf(); }
std::string fileContent = buffer.str(); fs::file_time_type externalFloor = fs::file_time_type::min();
fileContent = std::regex_replace(fileContent, std::regex(R"(//[^\n]*)"), ""); for(const ExternalBuildResult& ext : externalResults) {
fileContent = std::regex_replace(fileContent, std::regex(R"(/\*.*?\*/)"), ""); for(const std::string& flag : ext.compileFlags) {
tempModulePaths[i] = {file, fileContent, nullptr, nullptr}; command += " " + flag;
}
for(const std::string& flag : ext.linkFlags) {
buildResult.libs.insert(flag);
}
if (ext.latestArtifact > externalFloor) externalFloor = ext.latestArtifact;
} }
std::erase_if(tempModulePaths, [this](std::tuple<fs::path, std::string, ModulePartition*, Module*>& file) { for(std::unique_ptr<Module>& interface : config.interfaces) {
std::smatch match; if(interface->Check(pcmDir, externalFloor)) {
if (std::regex_search(std::get<1>(file), match, std::regex(R"(export module ([a-zA-Z0-9_\.\-]+);)"))) { Module* mod = interface.get();
std::get<0>(file).replace_extension(""); threads.emplace_back([mod, &command, &pcmDir, &buildDir, &buildCancelled, &buildError]() {
interfaces.push_back(std::make_unique<Module>(std::move(match[1].str()), std::move(std::get<0>(file)))); try {
return true; mod->Compile(command, pcmDir, buildDir, buildCancelled, buildError);
} else { } catch (const std::exception& e) {
return false; bool expected = false;
} if (buildCancelled.compare_exchange_strong(expected, true)) {
}); buildError = std::format("Module::Compile threw: {}", e.what());
}
for(std::uint16_t i = 0; i < tempModulePaths.size(); i++) {
std::smatch match;
if (std::regex_search(std::get<1>(tempModulePaths[i]), match, std::regex(R"(export module ([a-zA-Z_0-9\.\-]+):([a-zA-Z_0-9\.\-]+);)"))) {
for(const std::unique_ptr<Module>& modulee : interfaces) {
if(modulee->name == match[1]) {
std::string name = match[2].str();
fs::path pthCpy = std::get<0>(tempModulePaths[i]);
pthCpy.replace_extension("");
std::unique_ptr<ModulePartition> partition = std::make_unique<ModulePartition>(std::move(name), std::move(pthCpy));
std::get<2>(tempModulePaths[i]) = partition.get();
modulee->partitions.push_back(std::move(partition));
std::get<3>(tempModulePaths[i]) = modulee.get();
goto next;
} }
} });
throw std::runtime_error(std::format("Module {} not found, referenced in {}", match[1].str(), std::get<0>(tempModulePaths[i]).string()));
} else {
throw std::runtime_error(std::format("No module declaration found in {}", std::get<0>(tempModulePaths[i]).string()));
}
next:;
}
for(std::tuple<fs::path, std::string, ModulePartition*, Module*>& file : tempModulePaths) {
std::regex pattern(R"(import :([a-zA-Z_\-0-9\.]+);)");
std::sregex_iterator currentMatch(std::get<1>(file).begin(), std::get<1>(file).end(), pattern);
std::sregex_iterator lastMatch;
while (currentMatch != lastMatch) {
std::smatch match = *currentMatch;
for(std::unique_ptr<ModulePartition>& partition2 : std::get<3>(file)->partitions) {
if(partition2->name == match[1]) {
std::get<2>(file)->partitionDependencies.push_back(partition2.get());
goto next2;
}
}
throw std::runtime_error(std::format("imported partition {}:{} not found, referenced in {}", std::get<3>(file)->name, match[1].str(), std::get<0>(file).string()));
next2: ++currentMatch;
}
}
for(Module& interface : interfaces) {
if(config.interfaces[i]->Check(pcmDir)) {
threads.emplace_back(&Module::Compile, interface, command, pcmDir, buildDir, buildCancelled, buildError);
buildResult.repack = true; buildResult.repack = true;
} }
files += std::format(" {}/{}.o", buildDir.string(), interface.path.filename().string()); files += std::format(" {}/{}.o", buildDir.string(), interface->path.filename().string());
for(std::unique_ptr<ModulePartition>& part : interface.partitions) { for(std::unique_ptr<ModulePartition>& part : interface->partitions) {
files += std::format(" {}/{}.o", buildDir.string(), part->path.filename().string()); files += std::format(" {}/{}.o", buildDir.string(), part->path.filename().string());
} }
} }
for(Implementation& interface : implementations) { for(Implementation& implementation : config.implementations) {
if(config.implementations[i].Check(buildDir, pcmDir)) { if(implementation.Check(buildDir, pcmDir, externalFloor)) {
buildResult.repack = true; buildResult.repack = true;
threads.emplace_back(&Implementation::Compile, &config.implementations[i], command, buildDir, buildCancelled, buildError); Implementation* impl = &implementation;
threads.emplace_back([impl, &command, &buildDir, &buildCancelled, &buildError]() {
try {
impl->Compile(command, buildDir, buildCancelled, buildError);
} catch (const std::exception& e) {
bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::format("Implementation::Compile threw: {}", e.what());
}
}
});
} }
files += std::format(" {}/{}_impl.o", buildDir.string(), config.implementations[i].path.filename().string()); files += std::format(" {}/{}_impl.o", buildDir.string(), implementation.path.filename().string());
} }
for(std::thread& thread : threads) { for(std::thread& thread : threads) {
@ -334,29 +533,95 @@ BuildResult Build(const Configuration& config, std::unordered_set<fs::path>& dep
return {buildError, false, {}}; return {buildError, false, {}};
} }
std::string linkExtras;
for(const std::string& flag : buildResult.libs) {
linkExtras += " " + flag;
}
if(buildResult.repack) { if(buildResult.repack) {
if(config.type == CRAFTER_CONFIGURATION_TYPE_EXECUTABLE) { if(config.type == ConfigurationType::Executable) {
#ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu #ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu
buildResult.result = RunCommand(std::format("{}{} -o {} -fuse-ld=lld ", command, files, (binDir/outputName).string())); if(config.target == "x86_64-w64-mingw32") {
try {
// Iterate over the source directory
for (const auto& entry : fs::directory_iterator("/usr/x86_64-w64-mingw32/bin/")) {
// Check if the file is a regular file and ends with ".dll"
if (fs::is_regular_file(entry) && entry.path().extension() == ".dll") {
// Construct the destination file path
fs::path dest_file = outputDir / entry.path().filename();
// Check if the destination file exists and if it is older than the source file
if (!fs::exists(dest_file) || fs::last_write_time(entry.path()) > fs::last_write_time(dest_file)) {
// Copy the file if it doesn't exist or is older
fs::copy(entry.path(), dest_file, fs::copy_options::overwrite_existing);
}
}
}
} catch (const fs::filesystem_error& e) {
return {e.what(), false, {}};
}
}
buildResult.result = RunCommand(std::format("{}{} -o {} -fuse-ld=lld{}", command, files, (outputDir/config.outputName).string(), linkExtras));
#endif #endif
#if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32) #if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32)
system(std::format("copy \"%LIBCXX_DIR%\\lib\\c++.dll\" \"{}\\c++.dll\"", binDir.string()).c_str()); std::system(std::format("copy \"%LIBCXX_DIR%\\lib\\c++.dll\" \"{}\\c++.dll\"", outputDir.string()).c_str());
buildResult.result = RunCommand(std::format("{}{} -o {}.exe -fuse-ld=lld -L %LIBCXX_DIR%\\lib -lc++ -nostdinc++ -nostdlib++", command, files, (binDir/outputName).string())); buildResult.result = RunCommand(std::format("{}{} -o {}.exe -fuse-ld=lld -L %LIBCXX_DIR%\\lib -lc++ -nostdinc++ -nostdlib++{}", command, files, (outputDir/config.outputName).string(), linkExtras));
#endif #endif
} else if(config.type == CRAFTER_CONFIGURATION_TYPE_LIBRARY) { } else if(config.type == ConfigurationType::LibraryStatic) {
#ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu #ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu
buildResult.result = RunCommand(std::format("ar r {}.a {}", (binDir/fs::path(std::string("lib")+outputName)).string(), files)); buildResult.result = RunCommand(std::format("ar rcs {}.a {}", (outputDir/fs::path(std::string("lib")+config.outputName)).string(), files));
#endif #endif
#if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32) #if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32)
buildResult.result = RunCommand(std::format("llvm-lib.exe {} /OUT:{}.lib", files, (binDir/fs::path(outputName)).string())); buildResult.result = RunCommand(std::format("llvm-lib.exe {} /OUT:{}.lib", files, (outputDir/fs::path(config.outputName)).string()));
#endif #endif
} else { } else {
buildResult.result = RunCommand(std::format("{}{} -shared -o {}.so -Wl,-rpath,'$ORIGIN' -fuse-ld=lld", command, files, (binDir/(std::string("lib")+outputName)).string())); buildResult.result = RunCommand(std::format("{}{} -shared -o {}.so -Wl,-rpath,'$ORIGIN' -fuse-ld=lld{}", command, files, (outputDir/(std::string("lib")+config.outputName)).string(), linkExtras));
} }
} }
if (config.type == ConfigurationType::LibraryStatic || config.type == ConfigurationType::LibraryDynamic) {
buildResult.libs.insert(std::format("-L{}", outputDir.string()));
buildResult.libs.insert(std::format("-l{}", config.outputName));
}
return buildResult; return buildResult;
} }
int Crafter::Run(int argc, char** argv) {
try {
fs::path projectFile = "./project.cpp";
std::vector<std::string_view> projectArgs;
projectArgs.reserve(argc);
for (int i = 1; i < argc; ++i) {
std::string_view arg = argv[i];
if (arg.starts_with("--project=")) {
projectFile = arg.substr(std::string_view("--project=").size());
} else {
projectArgs.push_back(arg);
}
}
if (!fs::exists(projectFile)) {
std::println(std::cerr, "No project file at {}", projectFile.string());
return 1;
}
Configuration config = LoadProject(projectFile, projectArgs);
std::unordered_map<fs::path, std::shared_future<BuildResult>> depResults;
std::mutex depMutex;
BuildResult result = Build(config, depResults, depMutex);
if (!result.result.empty()) {
std::println(std::cerr, "{}", result.result);
return 1;
}
return 0;
} catch (const std::exception& e) {
std::println(std::cerr, "{}", e.what());
return 1;
}
}

View file

@ -0,0 +1,251 @@
/*
Crafter® Build
Copyright (C) 2026 Catcrafts®
Catcrafts.net
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License version 3.0 as published by the Free Software Foundation;
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
module Crafter.Build:External_impl;
import std;
import :External;
import :Platform;
namespace fs = std::filesystem;
using namespace Crafter;
namespace {
std::string DeriveName(const GitSource& source) {
std::string url = source.url;
while (!url.empty() && (url.back() == '/' || url.back() == '\\')) url.pop_back();
if (url.ends_with(".git")) url.resize(url.size() - 4);
auto slash = url.find_last_of("/\\");
return slash == std::string::npos ? url : url.substr(slash + 1);
}
std::string ShellQuote(std::string_view s) {
std::string out = "'";
for (char c : s) {
if (c == '\'') out += "'\\''";
else out += c;
}
out += "'";
return out;
}
std::string JoinOptions(std::span<const std::string> options) {
std::string out;
for (const std::string& opt : options) {
if (!out.empty()) out += ' ';
out += ShellQuote(opt);
}
return out;
}
std::string FetchGit(const GitSource& source, const fs::path& cloneDir) {
auto runGit = [](std::string_view cmd) -> std::optional<std::string> {
CommandResult r = RunCommandChecked(cmd);
if (r.exitCode != 0) return std::format("`{}` failed (exit {}): {}", cmd, r.exitCode, r.output);
return std::nullopt;
};
if (fs::exists(cloneDir)) {
CommandResult remote = RunCommandChecked(std::format("git -C {} remote get-url origin", ShellQuote(cloneDir.string())));
std::string currentUrl = remote.output;
while (!currentUrl.empty() && (currentUrl.back() == '\n' || currentUrl.back() == '\r')) currentUrl.pop_back();
if (remote.exitCode != 0 || currentUrl != source.url) {
fs::remove_all(cloneDir);
} else if (!source.commit.empty()) {
CommandResult head = RunCommandChecked(std::format("git -C {} rev-parse HEAD", ShellQuote(cloneDir.string())));
std::string currentHead = head.output;
while (!currentHead.empty() && (currentHead.back() == '\n' || currentHead.back() == '\r')) currentHead.pop_back();
if (head.exitCode == 0 && currentHead != source.commit) {
if (auto err = runGit(std::format("git -C {} fetch origin", ShellQuote(cloneDir.string())))) return *err;
if (auto err = runGit(std::format("git -C {} checkout {}", ShellQuote(cloneDir.string()), ShellQuote(source.commit)))) return *err;
}
return "";
} else {
return "";
}
}
fs::path tmpDir = cloneDir;
tmpDir += ".tmp";
if (fs::exists(tmpDir)) fs::remove_all(tmpDir);
if (auto err = runGit(std::format("git clone --recursive {} {}", ShellQuote(source.url), ShellQuote(tmpDir.string())))) {
if (fs::exists(tmpDir)) fs::remove_all(tmpDir);
return *err;
}
if (!source.branch.empty()) {
if (auto err = runGit(std::format("git -C {} switch {}", ShellQuote(tmpDir.string()), ShellQuote(source.branch)))) {
fs::remove_all(tmpDir);
return *err;
}
}
if (!source.commit.empty()) {
if (auto err = runGit(std::format("git -C {} checkout {}", ShellQuote(tmpDir.string()), ShellQuote(source.commit)))) {
fs::remove_all(tmpDir);
return *err;
}
}
std::error_code ec;
fs::rename(tmpDir, cloneDir, ec);
if (ec) {
fs::remove_all(tmpDir);
return std::format("rename {} -> {} failed: {}", tmpDir.string(), cloneDir.string(), ec.message());
}
return "";
}
std::string BuildInjectedCMakeFlags(const fs::path& cmakeBuildDir) {
std::string out;
out += " -DCMAKE_C_COMPILER=clang";
out += " -DCMAKE_CXX_COMPILER=clang++";
#ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu
out += " -DCMAKE_CXX_FLAGS=-stdlib=libc++";
out += " -DCMAKE_EXE_LINKER_FLAGS=-stdlib=libc++";
out += " -DCMAKE_SHARED_LINKER_FLAGS=-stdlib=libc++";
#endif
fs::path absBuildDir = fs::absolute(cmakeBuildDir);
out += std::format(" -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY={}", ShellQuote(absBuildDir.string()));
out += std::format(" -DCMAKE_LIBRARY_OUTPUT_DIRECTORY={}", ShellQuote(absBuildDir.string()));
return out;
}
std::string ConfigureCMake(const fs::path& cloneDir, const fs::path& cmakeBuildDir, std::span<const std::string> options) {
fs::path optionsFile = cmakeBuildDir / ".crafter-options";
std::string optionsKey = JoinOptions(options);
bool needsConfigure = !fs::exists(cmakeBuildDir / "CMakeCache.txt");
if (!needsConfigure) {
std::ifstream existing(optionsFile);
if (!existing) {
needsConfigure = true;
} else {
std::stringstream buf;
buf << existing.rdbuf();
if (buf.str() != optionsKey) needsConfigure = true;
}
}
if (!needsConfigure) return "";
if (!fs::exists(cmakeBuildDir)) fs::create_directories(cmakeBuildDir);
std::string cmd = std::format("cmake -S {} -B {}{}",
ShellQuote(fs::absolute(cloneDir).string()),
ShellQuote(fs::absolute(cmakeBuildDir).string()),
BuildInjectedCMakeFlags(cmakeBuildDir));
if (!options.empty()) {
cmd += " ";
cmd += optionsKey;
}
CommandResult r = RunCommandChecked(cmd);
if (r.exitCode != 0) {
return std::format("cmake configure failed (exit {}): {}", r.exitCode, r.output);
}
std::ofstream(optionsFile) << optionsKey;
return "";
}
std::string BuildCMake(const fs::path& cmakeBuildDir) {
std::string cmd = std::format("cmake --build {}", ShellQuote(fs::absolute(cmakeBuildDir).string()));
CommandResult r = RunCommandChecked(cmd);
if (r.exitCode != 0) {
return std::format("cmake --build failed (exit {}): {}", r.exitCode, r.output);
}
return "";
}
} // namespace
ExternalBuildResult Crafter::BuildExternal(
const ExternalDependency& dep,
std::atomic<bool>& cancelled) {
ExternalBuildResult result;
if (cancelled.load(std::memory_order_relaxed)) return result;
std::string name = dep.name.empty() ? DeriveName(dep.source) : dep.name;
if (name.empty()) {
result.error = std::format("Could not derive name for external dependency from URL '{}'", dep.source.url);
return result;
}
fs::path externalRoot = GetCacheDir() / "external";
if (!fs::exists(externalRoot)) {
std::error_code ec;
fs::create_directories(externalRoot, ec);
if (ec) {
result.error = std::format("Failed to create {}: {}", externalRoot.string(), ec.message());
return result;
}
}
std::string keyMaterial = std::format("{}|{}|{}|{}",
dep.source.url, dep.source.branch, dep.source.commit, JoinOptions(dep.options));
std::size_t key = std::hash<std::string>{}(keyMaterial);
fs::path cloneDir = externalRoot / std::format("{}-{:016x}", name, key);
std::string fetchErr = FetchGit(dep.source, cloneDir);
if (!fetchErr.empty()) {
result.error = std::format("git fetch for '{}': {}", name, fetchErr);
return result;
}
if (cancelled.load(std::memory_order_relaxed)) return result;
fs::path cmakeBuildDir;
if (dep.builder == ExternalBuilder::CMake) {
cmakeBuildDir = cloneDir / "build";
if (std::string err = ConfigureCMake(cloneDir, cmakeBuildDir, dep.options); !err.empty()) {
result.error = std::format("cmake configure for '{}': {}", name, err);
return result;
}
if (cancelled.load(std::memory_order_relaxed)) return result;
if (std::string err = BuildCMake(cmakeBuildDir); !err.empty()) {
result.error = std::format("cmake build for '{}': {}", name, err);
return result;
}
}
for (const fs::path& include : dep.includeDirs) {
fs::path full = include.empty() ? cloneDir : cloneDir / include;
result.compileFlags.push_back(std::format("-I{}", fs::absolute(full).string()));
}
if (dep.builder == ExternalBuilder::CMake) {
result.linkFlags.push_back(std::format("-L{}", fs::absolute(cmakeBuildDir).string()));
for (const std::string& lib : dep.libs) {
result.linkFlags.push_back(std::format("-l{}", lib));
}
if (fs::exists(cmakeBuildDir)) {
for (const auto& entry : fs::directory_iterator(cmakeBuildDir)) {
if (!entry.is_regular_file()) continue;
const fs::path& p = entry.path();
if (p.extension() != ".a" && p.extension() != ".so") continue;
std::error_code ec;
fs::file_time_type t = entry.last_write_time(ec);
if (!ec && t > result.latestArtifact) result.latestArtifact = t;
}
}
}
return result;
}

View file

@ -20,49 +20,55 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
module Crafter.Build:Implementation_impl; module Crafter.Build:Implementation_impl;
import std; import std;
import :Implementation; import :Implementation;
import :Module; import :Interface;
import :Command; import :Platform;
namespace fs = std::filesystem; namespace fs = std::filesystem;
namespace Crafter { namespace Crafter {
Implementation::Implementation(fs::path&& path) : path(std::move(path)) { Implementation::Implementation(fs::path&& path) : path(std::move(path)) {
} }
bool Implementation::Check(const fs::path& buildDir, const fs::path& pcmDir) const { bool Implementation::Check(const fs::path& buildDir, const fs::path& pcmDir, fs::file_time_type sourceFloor) const {
if(fs::exists((buildDir/path.filename()).string()+"_impl.o") && fs::last_write_time(path.string()+".cpp") < fs::last_write_time((buildDir/path.filename()).string()+"_impl.o")) { std::string objPath = (buildDir/path.filename()).string()+"_impl.o";
for(ModulePartition* dependency : partitionDependencies) { std::string cppPath = path.string()+".cpp";
if(dependency->Check(pcmDir)) { if(!fs::exists(objPath) || std::max(fs::last_write_time(cppPath), sourceFloor) >= fs::last_write_time(objPath)) {
return true;
}
}
for(Module* dependency : moduleDependencies) {
if(dependency->Check(pcmDir)) {
return true;
}
}
return false;
} else {
return true; return true;
} }
fs::file_time_type objTime = fs::last_write_time(objPath);
for(ModulePartition* dependency : partitionDependencies) {
if(dependency->Check(pcmDir, sourceFloor)) return true;
}
for(Module* dependency : moduleDependencies) {
if(dependency->Check(pcmDir, sourceFloor)) return true;
}
for(const auto& [externalMod, externalPcmPath] : externalModuleDependencies) {
std::error_code ec;
fs::file_time_type pcmTime = fs::last_write_time(externalPcmPath, ec);
if (!ec && pcmTime >= objTime) return true;
}
return false;
} }
void Implementation::Compile(const std::string_view clang, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError) const { void Implementation::Compile(const std::string_view clang, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError) const {
for(ModulePartition* dependency : partitionDependencies) { for(ModulePartition* dependency : partitionDependencies) {
if(!dependency->compiled.load()) { if(!dependency->compiled.load()) {
dependency->compiled.wait(true); dependency->compiled.wait(false);
} }
} }
for(Module* dependency : moduleDependencies) { for(Module* dependency : moduleDependencies) {
if(!dependency->compiled.load()) { if(!dependency->compiled.load()) {
dependency->compiled.wait(true); dependency->compiled.wait(false);
} }
} }
if (!buildCancelled.load(std::memory_order_relaxed)) { if (buildCancelled.load(std::memory_order_relaxed)) {
return; return;
} }
std::string result = RunCommand(std::format("{} {}.cpp -c -o {}_impl.o", clang, path.string(), (buildDir/path.filename()).string())); std::string result = RunCommand(std::format("{} {}.cpp -c -o {}_impl.o", clang, path.string(), (buildDir/path.filename()).string()));
bool expected = false;
if(!result.empty() && buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result);
}
} }
} }

View file

@ -20,30 +20,41 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
module Crafter.Build:Interface_impl; module Crafter.Build:Interface_impl;
import std; import std;
import :Interface; import :Interface;
import :Command; import :Platform;
namespace fs = std::filesystem; namespace fs = std::filesystem;
namespace Crafter { namespace Crafter {
ModulePartition::ModulePartition(std::string&& name, fs::path&& path) : name(std::move(name)), path(std::move(path)), compiled(false), checked(false) {} ModulePartition::ModulePartition(std::string&& name, fs::path&& path) : name(std::move(name)), path(std::move(path)), compiled(false), checked(false) {}
bool ModulePartition::Check(const fs::path& pcmDir) { bool ModulePartition::Check(const fs::path& pcmDir, fs::file_time_type sourceFloor) {
if(!checked) { if(!checked) {
checked = true; checked = true;
if(fs::exists((pcmDir/path.filename()).generic_string()+".pcm") && fs::last_write_time(path.generic_string()+".cppm") < fs::last_write_time((pcmDir/path.filename()).generic_string()+".pcm")) { std::string pcmPath = (pcmDir/path.filename()).generic_string()+".pcm";
std::string cppmPath = path.generic_string()+".cppm";
if(fs::exists(pcmPath) && std::max(fs::last_write_time(cppmPath), sourceFloor) < fs::last_write_time(pcmPath)) {
fs::file_time_type pcmTime = fs::last_write_time(pcmPath);
for(ModulePartition* dependency : partitionDependencies) { for(ModulePartition* dependency : partitionDependencies) {
if(dependency->Check(pcmDir)) { if(dependency->Check(pcmDir, sourceFloor)) {
needsRecompiling = true; needsRecompiling = true;
return true; return true;
} }
} }
for(Module* dependency : moduleDependencies) { for(Module* dependency : moduleDependencies) {
if(dependency->Check(pcmDir)) { if(dependency->Check(pcmDir, sourceFloor)) {
needsRecompiling = true;
return true;
}
}
for(const auto& [externalMod, externalPcmPath] : externalModuleDependencies) {
std::error_code ec;
fs::file_time_type t = fs::last_write_time(externalPcmPath, ec);
if (!ec && t >= pcmTime) {
needsRecompiling = true; needsRecompiling = true;
return true; return true;
} }
} }
needsRecompiling = false; needsRecompiling = false;
compiled.store(CRAFTER_COMPILE_STATUS_COMPLETED); compiled.store(true);
return false; return false;
} else { } else {
needsRecompiling = true; needsRecompiling = true;
@ -54,10 +65,15 @@ namespace Crafter {
} }
} }
void ModulePartition::Compile(const std::string_view clang, const fs::path& pcmDir, std::atomic<bool>& buildCancelled, std::string& buildError) { void ModulePartition::Compile(const std::string_view clang, const fs::path& pcmDir, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError) {
for(ModulePartition* dependency : partitionDependencies) { for(ModulePartition* dependency : partitionDependencies) {
if(!dependency->compiled.load()) { if(!dependency->compiled.load()) {
dependency->compiled.wait(true); dependency->compiled.wait(false);
}
}
for(Module* dependency : moduleDependencies) {
if(!dependency->compiled.load()) {
dependency->compiled.wait(false);
} }
} }
@ -70,7 +86,8 @@ namespace Crafter {
std::string result = RunCommand(std::format("{} {}.cppm --precompile -o {}.pcm", clang, path.string(), (pcmDir/path.filename()).string())); std::string result = RunCommand(std::format("{} {}.cppm --precompile -o {}.pcm", clang, path.string(), (pcmDir/path.filename()).string()));
if (!result.empty()) { if (!result.empty()) {
if (buildCancelled.compare_exchange_strong(false, true)) { bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result); buildError = std::move(result);
compiled.store(true); compiled.store(true);
compiled.notify_all(); compiled.notify_all();
@ -81,10 +98,11 @@ namespace Crafter {
compiled.store(true); compiled.store(true);
compiled.notify_all(); compiled.notify_all();
result = RunClang(std::format("{} -Wno-unused-command-line-argument {}.pcm -c -o {}.o", clang, (pcmDir/path.filename()).string(), (buildDir/path.filename()).string())); result = RunCommand(std::format("{} -Wno-unused-command-line-argument {}.pcm -c -o {}.o", clang, (pcmDir/path.filename()).string(), (buildDir/path.filename()).string()));
if (!result.empty()) { if (!result.empty()) {
if (buildCancelled.compare_exchange_strong(false, true)) { bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result); buildError = std::move(result);
} }
} }
@ -92,13 +110,15 @@ namespace Crafter {
Module::Module(std::string&& name, fs::path&& path) : name(std::move(name)), path(std::move(path)), compiled(false), checked(false) {} Module::Module(std::string&& name, fs::path&& path) : name(std::move(name)), path(std::move(path)), compiled(false), checked(false) {}
bool Module::Check(const fs::path& pcmDir) { bool Module::Check(const fs::path& pcmDir, fs::file_time_type sourceFloor) {
if(!checked) { if(!checked) {
checked = true; checked = true;
if(fs::exists((pcmDir/path.filename()).generic_string()+".pcm") && fs::last_write_time(path.generic_string()+".cppm") < fs::last_write_time((pcmDir/path.filename()).generic_string()+".pcm")) { std::string pcmPath = (pcmDir/path.filename()).generic_string()+".pcm";
std::string cppmPath = path.generic_string()+".cppm";
if(fs::exists(pcmPath) && std::max(fs::last_write_time(cppmPath), sourceFloor) < fs::last_write_time(pcmPath)) {
bool depCheck = false; bool depCheck = false;
for(std::unique_ptr<ModulePartition>& partition : partitions) { for(std::unique_ptr<ModulePartition>& partition : partitions) {
if(partition->Check(pcmDir)) { if(partition->Check(pcmDir, sourceFloor)) {
depCheck = true; depCheck = true;
} }
} }
@ -112,7 +132,7 @@ namespace Crafter {
} }
} else { } else {
for(std::unique_ptr<ModulePartition>& partition : partitions) { for(std::unique_ptr<ModulePartition>& partition : partitions) {
partition->Check(pcmDir); partition->Check(pcmDir, sourceFloor);
} }
needsRecompiling = true; needsRecompiling = true;
return true; return true;
@ -124,10 +144,10 @@ namespace Crafter {
void Module::Compile(const std::string_view clang, const fs::path& pcmDir, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError) { void Module::Compile(const std::string_view clang, const fs::path& pcmDir, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError) {
std::vector<std::thread> threads; std::vector<std::thread> threads;
threads.reserve(interface.partitions.size()); threads.reserve(partitions.size());
for(std::unique_ptr<ModulePartition>& part : interface.partitions) { for(std::unique_ptr<ModulePartition>& part : partitions) {
if(part.needsRecompiling) { if(part->needsRecompiling) {
threads.emplace_back(&ModulePartition::Compile, part.get(), clang, pcmDir, buildDir, buildCancelled, buildError); threads.emplace_back(&ModulePartition::Compile, part.get(), clang, pcmDir, buildDir, std::ref(buildCancelled), std::ref(buildError));
} }
} }
@ -144,7 +164,8 @@ namespace Crafter {
std::string result = RunCommand(std::format("{} {}.cppm --precompile -o {}.pcm", clang, path.string(), (pcmDir/path.filename()).string())); std::string result = RunCommand(std::format("{} {}.cppm --precompile -o {}.pcm", clang, path.string(), (pcmDir/path.filename()).string()));
if (!result.empty()) { if (!result.empty()) {
if (buildCancelled.compare_exchange_strong(false, true)) { bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result); buildError = std::move(result);
compiled.store(true); compiled.store(true);
compiled.notify_all(); compiled.notify_all();
@ -158,7 +179,8 @@ namespace Crafter {
result = RunCommand(std::format("{} -Wno-unused-command-line-argument {}.pcm -c -o {}.o", clang, (pcmDir/path.filename()).string(), (buildDir/path.filename()).string())); result = RunCommand(std::format("{} -Wno-unused-command-line-argument {}.pcm -c -o {}.o", clang, (pcmDir/path.filename()).string(), (buildDir/path.filename()).string()));
if (!result.empty()) { if (!result.empty()) {
if (buildCancelled.compare_exchange_strong(false, true)) { bool expected = false;
if (buildCancelled.compare_exchange_strong(expected, true)) {
buildError = std::move(result); buildError = std::move(result);
} }
} }

View file

@ -16,8 +16,19 @@ You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/ */
module;
export module Crafter.Build:Platform_impl; #if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32)
#include <windows.h>
#endif
#include <stdio.h>
#include <stdlib.h>
#include <limits.h>
#ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu
#include <unistd.h>
#include <dlfcn.h>
#include <sys/wait.h>
#endif
module Crafter.Build:Platform_impl;
import std; import std;
import :Platform; import :Platform;
import :Clang; import :Clang;
@ -26,7 +37,7 @@ using namespace Crafter;
#if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32) #if defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_windows_msvc) || defined(CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_w64_mingw32)
std::string RunCommand(const std::string_view cmd) { std::string Crafter::RunCommand(const std::string_view cmd) {
std::array<char, 128> buffer; std::array<char, 128> buffer;
std::string result; std::string result;
@ -46,17 +57,36 @@ std::string RunCommand(const std::string_view cmd) {
return result; return result;
} }
fs::path BuildStdPcm(Configuration& config, fs::path stdPcm) { CommandResult Crafter::RunCommandChecked(std::string_view cmd) {
std::array<char, 128> buffer;
CommandResult result{0, ""};
std::string with = "cmd /C \"" + std::string(cmd) + " 2>&1\"";
FILE* pipe = _popen(with.c_str(), "r");
if (!pipe) {
throw std::runtime_error("_popen() failed!");
}
while (fgets(buffer.data(), static_cast<int>(buffer.size()), pipe) != nullptr) {
result.output += buffer.data();
}
result.exitCode = _pclose(pipe);
return result;
}
std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
std::string libcxx = std::getenv("LIBCXX_DIR"); std::string libcxx = std::getenv("LIBCXX_DIR");
std::string stdPcm = std::format("{}\\{}{}\\std.pcm", exeDir.string(), config.target, config.march);
std::string stdcppm = std::format("{}\\modules\\c++\\v1\\std.cppm", libcxx); std::string stdcppm = std::format("{}\\modules\\c++\\v1\\std.cppm", libcxx);
if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time(stdcppm)) { if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time(stdcppm)) {
return RunCommand(std::format("clang++ --target={} -march={} -mtune={} -isystem %LIBCXX_DIR%\\include\\c++\\v1 -nostdinc++ -nostdlib++ -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile %LIBCXX_DIR%\\modules\\c++\\v1\\std.cppm -o {}", config.target, config.march, config.mtune, stdPcm)); return RunCommand(std::format("clang++ --target={} -march={} -mtune={} -isystem %LIBCXX_DIR%\\include\\c++\\v1 -nostdinc++ -nostdlib++ -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile %LIBCXX_DIR%\\modules\\c++\\v1\\std.cppm -o {}", config.target, config.march, config.mtune, stdPcm.string()));
} }
return "";
} }
fs::path GetCacheDir() { fs::path Crafter::GetCacheDir() {
if (const char* local = std::getenv("LOCALAPPDATA")) { if (const char* local = std::getenv("LOCALAPPDATA")) {
return fs::path(local) / "crafter.build"; return fs::path(local) / "crafter.build";
} }
@ -64,14 +94,121 @@ fs::path GetCacheDir() {
throw std::runtime_error("LOCALAPPDATA not set"); throw std::runtime_error("LOCALAPPDATA not set");
} }
std::string GetBaseCommand(Configuration& config) { std::string Crafter::GetBaseCommand(const Configuration& config) {
return std::format("clang++ -nostdinc++ -nostdlib++ -isystem %LIBCXX_DIR%\\include\\c++\\v1"); return std::format("clang++ -nostdinc++ -nostdlib++ -isystem %LIBCXX_DIR%\\include\\c++\\v1");
} }
namespace {
constexpr std::array<std::string_view, 7> kCrafterBuildModules = {
"Crafter.Build-Shader",
"Crafter.Build-Platform",
"Crafter.Build-Interface",
"Crafter.Build-Implementation",
"Crafter.Build-External",
"Crafter.Build-Clang",
"Crafter.Build",
};
void EnsureCrafterBuildPcms(const fs::path& sourceDir, const fs::path& cacheDir) {
for (std::string_view name : kCrafterBuildModules) {
fs::path cppmPath = sourceDir / (std::string(name) + ".cppm");
fs::path pcmPath = cacheDir / (std::string(name) + ".pcm");
if (!fs::exists(cppmPath)) {
throw std::runtime_error(std::format("module source {} not found in {} (set CRAFTER_BUILD_HOME)", name, sourceDir.string()));
}
if (fs::exists(pcmPath) && fs::last_write_time(cppmPath) < fs::last_write_time(pcmPath)) {
continue;
}
std::string cmd = std::format(
"clang++ --target=x86_64-pc-windows-msvc -march=native -mtune=native "
"-std=c++26 -O3 "
"-nostdinc++ -nostdlib++ -isystem %LIBCXX_DIR%\\include\\c++\\v1 "
"-Wno-reserved-identifier -Wno-reserved-module-identifier "
"-fprebuilt-module-path={} "
"--precompile {} -o {}",
cacheDir.string(), cppmPath.string(), pcmPath.string());
CommandResult r = Crafter::RunCommandChecked(cmd);
if (r.exitCode != 0) {
throw std::runtime_error(std::format("Failed to precompile {} (exit {}): {}", name, r.exitCode, r.output));
}
}
}
}
Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const std::string_view> args) {
fs::path absProject = fs::canonical(projectFile);
fs::path buildDir = absProject.parent_path() / "build";
if (!fs::exists(buildDir)) {
fs::create_directories(buildDir);
}
fs::path dllPath = buildDir / (absProject.stem().string() + ".dll");
char hostExeBuf[MAX_PATH];
DWORD hostExeLen = GetModuleFileNameA(nullptr, hostExeBuf, MAX_PATH);
if (hostExeLen == 0 || hostExeLen == MAX_PATH) {
throw std::runtime_error("GetModuleFileName failed");
}
fs::path hostExe(std::string(hostExeBuf, hostExeLen));
const char* envHome = std::getenv("CRAFTER_BUILD_HOME");
fs::path sourceDir = envHome
? fs::path(envHome)
: hostExe.parent_path().parent_path() / "share" / "crafter-build";
Configuration hostConfig;
hostConfig.target = "x86_64-pc-windows-msvc";
hostConfig.march = "native";
hostConfig.mtune = "native";
fs::path cacheDir = GetCacheDir() / std::format("{}-{}", hostConfig.target, hostConfig.march);
if (!fs::exists(cacheDir)) fs::create_directories(cacheDir);
std::string stdResult = BuildStdPcm(hostConfig, cacheDir / "std.pcm");
if (!stdResult.empty()) {
throw std::runtime_error(std::format("Failed to build std.pcm: {}", stdResult));
}
EnsureCrafterBuildPcms(sourceDir, cacheDir);
bool stale = !fs::exists(dllPath)
|| fs::last_write_time(dllPath) < fs::last_write_time(absProject)
|| fs::last_write_time(dllPath) < fs::last_write_time(hostExe);
if (stale) {
fs::path crafterBuildLib = hostExe.parent_path() / "crafter-build.lib";
std::string compileCmd = std::format(
"clang++ --target=x86_64-pc-windows-msvc -march=native -mtune=native "
"-std=c++26 -shared -O3 -Wno-return-type-c-linkage "
"-nostdinc++ -nostdlib++ -isystem %LIBCXX_DIR%\\include\\c++\\v1 "
"-fprebuilt-module-path={} "
"{} {} -o {} -L %LIBCXX_DIR%\\lib -lc++",
cacheDir.string(),
absProject.string(), crafterBuildLib.string(), dllPath.string());
std::string result = RunCommand(compileCmd);
if (!result.empty()) {
throw std::runtime_error(std::format("Failed to compile project {}: {}", absProject.string(), result));
}
}
HMODULE handle = LoadLibraryA(dllPath.string().c_str());
if (!handle) {
throw std::runtime_error(std::format("Failed to load project {}: error {}", dllPath.string(), GetLastError()));
}
using ProjectFn = Configuration (*)(std::span<const std::string_view>);
auto fn = reinterpret_cast<ProjectFn>(GetProcAddress(handle, "CrafterBuildProject"));
if (!fn) {
throw std::runtime_error(std::format("CrafterBuildProject not found in {}: error {}", dllPath.string(), GetLastError()));
}
return fn(args);
}
#endif #endif
#ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu #ifdef CRAFTER_BUILD_CONFIGURATION_TARGET_x86_64_pc_linux_gnu
std::string RunCommand(const std::string_view cmd) { std::string Crafter::RunCommand(const std::string_view cmd) {
std::array<char, 128> buffer; std::array<char, 128> buffer;
std::string result; std::string result;
@ -90,7 +227,30 @@ std::string RunCommand(const std::string_view cmd) {
return result; return result;
} }
fs::path BuildStdPcm(Configuration& config, fs::path stdPcm) { CommandResult Crafter::RunCommandChecked(std::string_view cmd) {
std::array<char, 128> buffer;
CommandResult result{0, ""};
std::string with = std::string(cmd) + " 2>&1";
FILE* pipe = popen(with.c_str(), "r");
if (!pipe) throw std::runtime_error("popen() failed!");
while (fgets(buffer.data(), buffer.size(), pipe) != nullptr) {
result.output += buffer.data();
}
int status = pclose(pipe);
if (WIFEXITED(status)) {
result.exitCode = WEXITSTATUS(status);
} else if (WIFSIGNALED(status)) {
result.exitCode = 128 + WTERMSIG(status);
} else {
result.exitCode = -1;
}
return result;
}
std::string Crafter::BuildStdPcm(const Configuration& config, fs::path stdPcm) {
if(config.target == "x86_64-w64-mingw32") { if(config.target == "x86_64-w64-mingw32") {
std::vector<std::string> folders; std::vector<std::string> folders;
@ -111,38 +271,20 @@ fs::path BuildStdPcm(Configuration& config, fs::path stdPcm) {
fs::path stdCc = fs::path(std::format("/usr/x86_64-w64-mingw32/include/c++/{}/bits/std.cc", mingWversion)); fs::path stdCc = fs::path(std::format("/usr/x86_64-w64-mingw32/include/c++/{}/bits/std.cc", mingWversion));
if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time(stdCc)) { if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time(stdCc)) {
std::string result = RunCommand(std::format("cp {} {}/{}{}/std.cppm\nclang++ --target={} -march={} -mtune={} -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {}/{}{}/std.cppm -o {}", stdCc.string(), exeDir.string(), config.target, config.march, config.target, config.march, config.mtune, exeDir.string(), config.target, config.march, stdPcm)); return RunCommand(std::format("cp {} {}/std.cppm\nclang++ --target={} -march={} -mtune={} -O3 -std=c++26 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile {}/std.cppm -o {}", stdCc.string(), stdPcm.parent_path().string(), config.target, config.march, config.mtune, stdPcm.parent_path().string(), stdPcm.string()));
if(!result.empty()) { } else {
return result; return "";
}
}
try {
// Iterate over the source directory
for (const auto& entry : fs::directory_iterator("/usr/x86_64-w64-mingw32/bin/")) {
// Check if the file is a regular file and ends with ".dll"
if (fs::is_regular_file(entry) && entry.path().extension() == ".dll") {
// Construct the destination file path
fs::path dest_file = project.binDir / config.name / entry.path().filename();
// Check if the destination file exists and if it is older than the source file
if (!fs::exists(dest_file) || fs::last_write_time(entry.path()) > fs::last_write_time(dest_file)) {
// Copy the file if it doesn't exist or is older
fs::copy(entry.path(), dest_file, fs::copy_options::overwrite_existing);
}
}
}
} catch (const fs::filesystem_error& e) {
return e.what();
} }
} else { } else {
if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time("/usr/share/libc++/v1/std.cppm")) { if(!fs::exists(stdPcm) || fs::last_write_time(stdPcm) < fs::last_write_time("/usr/share/libc++/v1/std.cppm")) {
return RunCommand(std::format("clang++ --target={} -std=c++26 -stdlib=libc++ -march={} -mtune={} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile /usr/share/libc++/v1/std.cppm -o {}", config.target, config.march, config.mtune, stdPcm)); return RunCommand(std::format("clang++ --target={} -std=c++26 -stdlib=libc++ -march={} -mtune={} -O3 -Wno-reserved-identifier -Wno-reserved-module-identifier --precompile /usr/share/libc++/v1/std.cppm -o {}", config.target, config.march, config.mtune, stdPcm.string()));
} else {
return "";
} }
} }
} }
fs::path GetCacheDir() { fs::path Crafter::GetCacheDir() {
if (const char* xdg = std::getenv("XDG_CACHE_HOME"); xdg && *xdg) { if (const char* xdg = std::getenv("XDG_CACHE_HOME"); xdg && *xdg) {
return fs::path(xdg) / "crafter.build"; return fs::path(xdg) / "crafter.build";
} }
@ -154,14 +296,114 @@ fs::path GetCacheDir() {
throw std::runtime_error("Neither XDG_CACHE_HOME nor HOME set"); throw std::runtime_error("Neither XDG_CACHE_HOME nor HOME set");
} }
std::string GetBaseCommand(Configuration& config) { std::string Crafter::GetBaseCommand(const Configuration& config) {
std::string stdlib; std::string stdlib;
if(config.target == "x86_64-w64-mingw32") { if(config.target == "x86_64-w64-mingw32") {
stdlib = ""; stdlib = "";
} else { } else {
stdlib = "-stdlib=libc++"; stdlib = "-stdlib=libc++";
} }
std::string command = std::format("clang++ {}", stdlib); return std::format("clang++ {}", stdlib);
}
namespace {
constexpr std::array<std::string_view, 7> kCrafterBuildModules = {
"Crafter.Build-Shader",
"Crafter.Build-Platform",
"Crafter.Build-Interface",
"Crafter.Build-Implementation",
"Crafter.Build-External",
"Crafter.Build-Clang",
"Crafter.Build",
};
void EnsureCrafterBuildPcms(const fs::path& sourceDir, const fs::path& cacheDir) {
for (std::string_view name : kCrafterBuildModules) {
fs::path cppmPath = sourceDir / (std::string(name) + ".cppm");
fs::path pcmPath = cacheDir / (std::string(name) + ".pcm");
if (!fs::exists(cppmPath)) {
throw std::runtime_error(std::format("module source {} not found in {} (set CRAFTER_BUILD_HOME)", name, sourceDir.string()));
}
if (fs::exists(pcmPath) && fs::last_write_time(cppmPath) < fs::last_write_time(pcmPath)) {
continue;
}
std::string cmd = std::format(
"clang++ --target=x86_64-pc-linux-gnu -march=native -mtune=native "
"-std=c++26 -stdlib=libc++ -O3 "
"-Wno-reserved-identifier -Wno-reserved-module-identifier "
"-fprebuilt-module-path={} "
"--precompile {} -o {}",
cacheDir.string(), cppmPath.string(), pcmPath.string());
CommandResult r = Crafter::RunCommandChecked(cmd);
if (r.exitCode != 0) {
throw std::runtime_error(std::format("Failed to precompile {} (exit {}): {}", name, r.exitCode, r.output));
}
}
}
}
Configuration Crafter::LoadProject(const fs::path& projectFile, std::span<const std::string_view> args) {
fs::path absProject = fs::canonical(projectFile);
fs::path buildDir = absProject.parent_path() / "build";
if (!fs::exists(buildDir)) {
fs::create_directories(buildDir);
}
fs::path soPath = buildDir / (absProject.stem().string() + ".so");
fs::path hostExe = fs::read_symlink("/proc/self/exe");
const char* envHome = std::getenv("CRAFTER_BUILD_HOME");
fs::path sourceDir = envHome
? fs::path(envHome)
: hostExe.parent_path().parent_path() / "share" / "crafter-build";
Configuration hostConfig;
hostConfig.target = "x86_64-pc-linux-gnu";
hostConfig.march = "native";
hostConfig.mtune = "native";
fs::path cacheDir = GetCacheDir() / std::format("{}-{}", hostConfig.target, hostConfig.march);
if (!fs::exists(cacheDir)) fs::create_directories(cacheDir);
std::string stdResult = BuildStdPcm(hostConfig, cacheDir / "std.pcm");
if (!stdResult.empty()) {
throw std::runtime_error(std::format("Failed to build std.pcm: {}", stdResult));
}
EnsureCrafterBuildPcms(sourceDir, cacheDir);
bool stale = !fs::exists(soPath)
|| fs::last_write_time(soPath) < fs::last_write_time(absProject)
|| fs::last_write_time(soPath) < fs::last_write_time(hostExe);
if (stale) {
std::string compileCmd = std::format(
"clang++ --target=x86_64-pc-linux-gnu -march=native -mtune=native "
"-std=c++26 -stdlib=libc++ -shared -fPIC -O3 "
"-Wno-return-type-c-linkage "
"-fprebuilt-module-path={} "
"{} -o {}",
cacheDir.string(),
absProject.string(), soPath.string());
std::string result = RunCommand(compileCmd);
if (!result.empty()) {
throw std::runtime_error(std::format("Failed to compile project {}: {}", absProject.string(), result));
}
}
void* handle = dlopen(soPath.c_str(), RTLD_NOW | RTLD_GLOBAL);
if (!handle) {
throw std::runtime_error(std::format("Failed to load project {}: {}", soPath.string(), dlerror()));
}
using ProjectFn = Configuration (*)(std::span<const std::string_view>);
dlerror();
auto fn = reinterpret_cast<ProjectFn>(dlsym(handle, "CrafterBuildProject"));
if (const char* err = dlerror()) {
throw std::runtime_error(std::format("CrafterBuildProject not found in {}: {}", soPath.string(), err));
}
return fn(args);
} }
#endif #endif

View file

@ -27,14 +27,37 @@ import :Shader;
import std; import std;
namespace fs = std::filesystem; namespace fs = std::filesystem;
namespace {
EShLanguage ToEShLanguage(Crafter::ShaderType t) {
switch (t) {
case Crafter::ShaderType::Vertex: return EShLangVertex;
case Crafter::ShaderType::TessControl: return EShLangTessControl;
case Crafter::ShaderType::TessEvaluation: return EShLangTessEvaluation;
case Crafter::ShaderType::Geometry: return EShLangGeometry;
case Crafter::ShaderType::Fragment: return EShLangFragment;
case Crafter::ShaderType::Compute: return EShLangCompute;
case Crafter::ShaderType::RayGen: return EShLangRayGen;
case Crafter::ShaderType::Intersect: return EShLangIntersect;
case Crafter::ShaderType::AnyHit: return EShLangAnyHit;
case Crafter::ShaderType::ClosestHit: return EShLangClosestHit;
case Crafter::ShaderType::Miss: return EShLangMiss;
case Crafter::ShaderType::Callable: return EShLangCallable;
case Crafter::ShaderType::Task: return EShLangTask;
case Crafter::ShaderType::Mesh: return EShLangMesh;
}
return EShLangVertex;
}
}
namespace Crafter { namespace Crafter {
Shader::Shader(fs::path&& path, std::string&& entrypoint, EShLanguage type) : path(std::move(path)), entrypoint(std::move(entrypoint)), type(type) { Shader::Shader(fs::path&& path, std::string&& entrypoint, ShaderType type) : path(std::move(path)), entrypoint(std::move(entrypoint)), type(type) {
} }
bool Shader::Check(const fs::path& outputDir) const { bool Shader::Check(const fs::path& outputDir) const {
return fs::exists((outputDir/path.filename()).generic_string()+".spv") && fs::last_write_time(path.generic_string()+".glsl") < fs::last_write_time((outputDir/path.filename()).generic_string()+".spv"); return fs::exists((outputDir/path.filename()).generic_string()+".spv") && fs::last_write_time(path.generic_string()+".glsl") < fs::last_write_time((outputDir/path.filename()).generic_string()+".spv");
} }
std::string Shader::Compile(const fs::path& outputDir) const { std::string Shader::Compile(const fs::path& outputDir) const {
EShLanguage glslangType = ToEShLanguage(type);
glslang::InitializeProcess(); glslang::InitializeProcess();
EShMessages messages = static_cast<EShMessages>(EShMsgDefault | EShMsgVulkanRules | EShMsgSpvRules); EShMessages messages = static_cast<EShMessages>(EShMsgDefault | EShMsgVulkanRules | EShMsgSpvRules);
std::ifstream fileStream(path, std::ios::in | std::ios::binary); std::ifstream fileStream(path, std::ios::in | std::ios::binary);
@ -48,7 +71,7 @@ namespace Crafter {
const char *file_name_list[1] = {""}; const char *file_name_list[1] = {""};
const char *shader_source = reinterpret_cast<const char *>(src.data()); const char *shader_source = reinterpret_cast<const char *>(src.data());
glslang::TShader shader(type); glslang::TShader shader(glslangType);
shader.setStringsWithLengthsAndNames(&shader_source, nullptr, file_name_list, 1); shader.setStringsWithLengthsAndNames(&shader_source, nullptr, file_name_list, 1);
shader.setEntryPoint(entrypoint.c_str()); shader.setEntryPoint(entrypoint.c_str());
shader.setSourceEntryPoint(entrypoint.c_str()); shader.setSourceEntryPoint(entrypoint.c_str());
@ -81,7 +104,7 @@ namespace Crafter {
info_log += std::string(program.getInfoLog()) + std::string(program.getInfoDebugLog()); info_log += std::string(program.getInfoLog()) + std::string(program.getInfoDebugLog());
} }
glslang::TIntermediate* intermediate = program.getIntermediate(type); glslang::TIntermediate* intermediate = program.getIntermediate(glslangType);
if (!intermediate) if (!intermediate)
{ {
info_log += "Failed to get shared intermediate code."; info_log += "Failed to get shared intermediate code.";

View file

@ -0,0 +1,25 @@
/*
Crafter® Build
Copyright (C) 2026 Catcrafts®
Catcrafts.net
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License version 3.0 as published by the Free Software Foundation;
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
import std;
import Crafter.Build;
int main(int argc, char** argv) {
return Crafter::Run(argc, argv);
}

View file

@ -20,6 +20,9 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
export module Crafter.Build:Clang; export module Crafter.Build:Clang;
import std; import std;
import :Shader; import :Shader;
import :Interface;
import :Implementation;
import :External;
namespace fs = std::filesystem; namespace fs = std::filesystem;
export namespace Crafter { export namespace Crafter {
@ -27,14 +30,9 @@ export namespace Crafter {
std::string result; std::string result;
bool repack; bool repack;
std::unordered_set<std::string> libs; std::unordered_set<std::string> libs;
} };
struct Dependency { struct Define {
fs::path path;
std::vector<std::pair<std::string, std::string>> arguments;
}
export struct Define {
std::string name; std::string name;
std::string value; std::string value;
}; };
@ -46,20 +44,34 @@ export namespace Crafter {
}; };
struct Configuration { struct Configuration {
fs::path path;
std::string outputName;
std::string name;
std::string march = "native"; std::string march = "native";
std::string mtune = "native"; std::string mtune = "native";
std::string target;
bool debug = false; bool debug = false;
ConfigurationType type = ConfigurationType::Executable; ConfigurationType type = ConfigurationType::Executable;
std::vector<std::path> interfaces; std::vector<std::unique_ptr<Module>> interfaces;
std::vector<std::path> implementations; std::vector<Implementation> implementations;
std::vector<std::path> cFiles; std::vector<fs::path> cFiles;
std::vector<std::path> dependencies; std::vector<fs::path> cuda;
std::vector<std::path> files; std::vector<Configuration*> dependencies;
std::vector<fs::path> files;
std::vector<Define> defines; std::vector<Define> defines;
std::vector<Shader> shaders; std::vector<Shader> shaders;
std::vector<Dependency> deps; std::vector<ExternalDependency> externalDependencies;
} std::vector<std::string> compileFlags;
std::vector<std::string> linkFlags;
void GetInterfacesAndImplementations(std::span<fs::path> interfaces, std::span<fs::path> implementations);
fs::path PcmDir() const {
return path
/ (type == ConfigurationType::Executable ? "build" : "bin")
/ std::format("{}-{}-{}", name, target, march);
}
};
BuildResult Build(fs::path config, std::unordered_set<std::string> depSet); BuildResult Build(Configuration& config, std::unordered_map<fs::path, std::shared_future<BuildResult>>& depResults, std::mutex& depMutex);
BuildResult Build(const Configuration& config, std::unordered_set<std::string> depSet);
int Run(int argc, char** argv);
} }

View file

@ -0,0 +1,55 @@
/*
Crafter® Build
Copyright (C) 2026 Catcrafts®
Catcrafts.net
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License version 3.0 as published by the Free Software Foundation;
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
export module Crafter.Build:External;
import std;
namespace fs = std::filesystem;
export namespace Crafter {
struct GitSource {
std::string url;
std::string branch;
std::string commit;
};
enum class ExternalBuilder {
None,
CMake,
};
struct ExternalDependency {
std::string name;
GitSource source;
ExternalBuilder builder = ExternalBuilder::None;
std::vector<std::string> options;
std::vector<fs::path> includeDirs;
std::vector<std::string> libs;
};
struct ExternalBuildResult {
std::string error;
std::vector<std::string> compileFlags;
std::vector<std::string> linkFlags;
fs::file_time_type latestArtifact = fs::file_time_type::min();
};
ExternalBuildResult BuildExternal(
const ExternalDependency& dep,
std::atomic<bool>& cancelled);
}

View file

@ -18,7 +18,6 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/ */
export module Crafter.Build:Implementation; export module Crafter.Build:Implementation;
import :CompileStatus;
import std; import std;
namespace fs = std::filesystem; namespace fs = std::filesystem;
@ -29,9 +28,10 @@ namespace Crafter {
public: public:
std::vector<Module*> moduleDependencies; std::vector<Module*> moduleDependencies;
std::vector<ModulePartition*> partitionDependencies; std::vector<ModulePartition*> partitionDependencies;
std::vector<std::pair<Module*, fs::path>> externalModuleDependencies;
fs::path path; fs::path path;
Implementation(fs::path&& path); Implementation(fs::path&& path);
bool Check(const fs::path& buildDir, const fs::path& pcmDir) const; bool Check(const fs::path& buildDir, const fs::path& pcmDir, fs::file_time_type sourceFloor = fs::file_time_type::min()) const;
void Compile(const std::string_view clang, const fs::path& buildDir, std::string& result) const; void Compile(const std::string_view clang, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError) const;
}; };
} }

View file

@ -27,13 +27,14 @@ namespace Crafter {
public: public:
std::vector<Module*> moduleDependencies; std::vector<Module*> moduleDependencies;
std::vector<ModulePartition*> partitionDependencies; std::vector<ModulePartition*> partitionDependencies;
std::vector<std::pair<Module*, fs::path>> externalModuleDependencies;
std::atomic<bool> compiled; std::atomic<bool> compiled;
bool needsRecompiling; bool needsRecompiling;
bool checked = false; bool checked = false;
std::string name; std::string name;
fs::path path; fs::path path;
ModulePartition(std::string&& name, fs::path&& path); ModulePartition(std::string&& name, fs::path&& path);
bool Check(const fs::path& pcmDir); bool Check(const fs::path& pcmDir, fs::file_time_type sourceFloor = fs::file_time_type::min());
void Compile(const std::string_view clang, const fs::path& pcmDir, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError); void Compile(const std::string_view clang, const fs::path& pcmDir, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError);
}; };
@ -46,7 +47,7 @@ namespace Crafter {
std::string name; std::string name;
fs::path path; fs::path path;
Module(std::string&& name, fs::path&& path); Module(std::string&& name, fs::path&& path);
bool Check(const fs::path& pcmDir); bool Check(const fs::path& pcmDir, fs::file_time_type sourceFloor = fs::file_time_type::min());
void Compile(const std::string_view clang, const fs::path& pcmDir, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError); void Compile(const std::string_view clang, const fs::path& pcmDir, const fs::path& buildDir, std::atomic<bool>& buildCancelled, std::string& buildError);
}; };
} }

View file

@ -23,8 +23,14 @@ namespace fs = std::filesystem;
namespace Crafter { namespace Crafter {
struct Configuration; struct Configuration;
std::string BuildStdPcm(Configuration& config); struct CommandResult {
int exitCode;
std::string output;
};
std::string BuildStdPcm(const Configuration& config, fs::path stdPcm);
fs::path GetCacheDir(); fs::path GetCacheDir();
std::string RunCommand(); std::string RunCommand(const std::string_view command);
std::string GetBaseCommand(Configuration& config); CommandResult RunCommandChecked(std::string_view command);
std::string GetBaseCommand(const Configuration& config);
export Configuration LoadProject(const fs::path& projectFile, std::span<const std::string_view> args);
} }

View file

@ -17,19 +17,34 @@ License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/ */
module;
#include "glslang/Public/ShaderLang.h"
export module Crafter.Build:Shader; export module Crafter.Build:Shader;
import std; import std;
namespace fs = std::filesystem; namespace fs = std::filesystem;
namespace Crafter { namespace Crafter {
export enum class ShaderType {
Vertex,
TessControl,
TessEvaluation,
Geometry,
Fragment,
Compute,
RayGen,
Intersect,
AnyHit,
ClosestHit,
Miss,
Callable,
Task,
Mesh,
};
export class Shader { export class Shader {
public: public:
fs::path path; fs::path path;
std::string entrypoint; std::string entrypoint;
EShLanguage type; ShaderType type;
Shader(fs::path&& path, std::string&& entrypoint, EShLanguage type); Shader(fs::path&& path, std::string&& entrypoint, ShaderType type);
bool Check(const fs::path& outputDir) const; bool Check(const fs::path& outputDir) const;
std::string Compile(const fs::path& outputDir) const; std::string Compile(const fs::path& outputDir) const;
}; };

View file

@ -22,3 +22,4 @@ export import :Clang;
export import :Platform; export import :Platform;
export import :Implementation; export import :Implementation;
export import :Shader; export import :Shader;
export import :External;

51
project.cpp Normal file
View file

@ -0,0 +1,51 @@
import std;
import Crafter.Build;
namespace fs = std::filesystem;
using namespace Crafter;
extern "C" Configuration CrafterBuildProject(std::span<const std::string_view> args) {
Configuration cfg;
cfg.path = "./";
cfg.name = "crafter.build";
cfg.outputName = "crafter-build";
cfg.target = "x86_64-pc-linux-gnu";
cfg.type = ConfigurationType::Executable;
for (std::string_view arg : args) {
if (arg == "--debug") cfg.debug = true;
}
std::array<fs::path, 7> interfaces = {
"interfaces/Crafter.Build",
"interfaces/Crafter.Build-Shader",
"interfaces/Crafter.Build-Platform",
"interfaces/Crafter.Build-Interface",
"interfaces/Crafter.Build-Implementation",
"interfaces/Crafter.Build-External",
"interfaces/Crafter.Build-Clang",
};
std::array<fs::path, 7> implementations = {
"implementations/Crafter.Build-Shader",
"implementations/Crafter.Build-Platform",
"implementations/Crafter.Build-Interface",
"implementations/Crafter.Build-Implementation",
"implementations/Crafter.Build-External",
"implementations/Crafter.Build-Clang",
"implementations/main",
};
cfg.GetInterfacesAndImplementations(interfaces, implementations);
ExternalDependency& glslang = cfg.externalDependencies.emplace_back();
glslang.name = "glslang";
glslang.source.url = "https://github.com/KhronosGroup/glslang.git";
glslang.source.branch = "main";
glslang.builder = ExternalBuilder::CMake;
glslang.options = { "-DENABLE_OPT=OFF" };
glslang.includeDirs = { "" };
glslang.libs = { "SPIRV", "GenericCodeGen", "glslang", "OSDependent", "MachineIndependent", "glslang-default-resource-limits" };
cfg.linkFlags.push_back("-Wl,--export-dynamic");
cfg.linkFlags.push_back("-ldl");
return cfg;
}