V2: WASI, -r flag, CI pipeline, examples & tests cleanup #1

Merged
jorijnvdgraaf merged 10 commits from V2 into master 2026-04-29 02:35:36 +02:00

Summary

V2 baseline ready for review. Ships:

  • WASI (wasm32-wasip1) target support: auto-sysroot, required compile flags (-fno-exceptions, -mllvm -wasm-enable-sjlj, -D_WASI_EMULATED_SIGNAL), .wasm extension, opt-in EnableWasiBrowserRuntime(cfg) helper that drops index.html + runtime.js next to the .wasm.
  • -r flag: build-then-run for host targets; auto .exe/.wasm handling; rejects libraries.
  • CI pipeline (.forgejo/workflows/ci.yaml): PR/push/dispatch triggers, single arch-latest container, runs tests + cross-compiles for mingw + uploads artifacts; rolling latest release on push to master.
  • mingw cross-compile from Linux: ExternalDependency cache key includes target; cmake cross-flags for wasm32/mingw; -femulated-tls so std::__once_callable and friends resolve against libstdc++'s emutls definitions; -lstdc++exp -lpthread auto-linked.
  • GetCrafterBuildHome() exposed from Platform; LoadProject (Linux + Windows) now share the resolution.
  • Examples reorg: hello-world, library, with-module, wasi, tests — each with a README. Tests reorg: per-test directory with inner/ fixture, no shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes.

Test plan

  • Local: ./build.sh succeeds, ./bin/crafter-build self-rebuilds, crafter-build test shows 13 passed / 2 skipped (SshRunner + WindowsViaSsh skip without env vars set).
  • Local: ./bin/crafter-build --target=x86_64-w64-mingw32 produces a PE32+ executable plus auto-bundled mingw runtime DLLs.
  • Local: WASI example builds, index.html + runtime.js work in a browser (verified via caddy file-server).
  • CI: this PR's first run should mirror the local baseline (13 passed / 2 skipped). The arch-latest runner label needs to be registered on the runner before this can run.
  • CI artifacts: confirm crafter-build-linux-x86_64.tar.gz and crafter-build-windows-x86_64.zip upload as workflow artifacts.
  • CI release: after merge to master, confirm the rolling latest prerelease shows up under Releases with both archives attached.

🤖 Generated with Claude Code

## Summary V2 baseline ready for review. Ships: - **WASI** (`wasm32-wasip1`) target support: auto-sysroot, required compile flags (`-fno-exceptions`, `-mllvm -wasm-enable-sjlj`, `-D_WASI_EMULATED_SIGNAL`), `.wasm` extension, opt-in `EnableWasiBrowserRuntime(cfg)` helper that drops `index.html` + `runtime.js` next to the .wasm. - **`-r` flag**: build-then-run for host targets; auto `.exe`/`.wasm` handling; rejects libraries. - **CI pipeline** (`.forgejo/workflows/ci.yaml`): PR/push/dispatch triggers, single arch-latest container, runs tests + cross-compiles for mingw + uploads artifacts; rolling `latest` release on push to master. - **mingw cross-compile from Linux**: ExternalDependency cache key includes target; cmake cross-flags for `wasm32`/`mingw`; `-femulated-tls` so `std::__once_callable` and friends resolve against libstdc++'s emutls definitions; `-lstdc++exp -lpthread` auto-linked. - **`GetCrafterBuildHome()`** exposed from Platform; `LoadProject` (Linux + Windows) now share the resolution. - **Examples reorg**: hello-world, library, with-module, wasi, tests — each with a README. **Tests reorg**: per-test directory with `inner/` fixture, no shared `tests/fixtures/` tree. New `Wasi` test verifies `.wasm` magic bytes. ## Test plan - [x] Local: `./build.sh` succeeds, `./bin/crafter-build` self-rebuilds, `crafter-build test` shows 13 passed / 2 skipped (SshRunner + WindowsViaSsh skip without env vars set). - [x] Local: `./bin/crafter-build --target=x86_64-w64-mingw32` produces a PE32+ executable plus auto-bundled mingw runtime DLLs. - [x] Local: WASI example builds, `index.html` + `runtime.js` work in a browser (verified via caddy file-server). - [ ] **CI**: this PR's first run should mirror the local baseline (13 passed / 2 skipped). The `arch-latest` runner label needs to be registered on the runner before this can run. - [ ] **CI artifacts**: confirm `crafter-build-linux-x86_64.tar.gz` and `crafter-build-windows-x86_64.zip` upload as workflow artifacts. - [ ] **CI release**: after merge to master, confirm the rolling `latest` prerelease shows up under Releases with both archives attached. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
- subprocess-isolated test runner (replaces V1 dlopen-RunTest);
  Pass/Fail/Crash/Timeout/Skipped outcomes via :Test partition
- TestRunner abstraction with command templates: Local, Ssh,
  SshWin (cmd.exe-shell), QemuUser, FromEnv; probe-based skip
  when runner unreachable
- transitive PCM-path propagation in Build(); resolveImport
  walks deps recursively; depResults cache keyed by PcmDir()
  so per-target builds don't collide
- cfg.sysroot threaded through BuildStdPcm + base compile/link
  command (enables aarch64 cross via Arch Linux ARM rootfs)
- lib + exe split: project.cpp defines crafterBuildLib
  (LibraryStatic) + crafterBuildExe (Executable depending on
  it); build.sh produces lib/libcrafter-build.a alongside
  bin/crafter-build for downstream static-link consumers
- Windows DLL+launcher: CRAFTER_API macro, /EXPORT flag for
  project.dll's CrafterBuildProject; Crafter::Run as the real
  entry point with main.cpp as a thin wrapper
- 18 tests: HelloWorld/WithModule/Defines/CrossProjectModule/
  Diamond × (Linux + sshwin:winvm), plus Incremental,
  BuildError, Libraries, RunnerClassification, QemuUser,
  SshRunner, WindowsViaSsh, CrossArchAarch64
- single ./bin/crafter-build test runs everything; Windows
  variants skip gracefully if winvm SSH alias unreachable

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
V2: WASI, -r flag, CI pipeline, examples & tests cleanup
Some checks failed
CI / build-test-release (pull_request) Failing after 44s
eaee502e8c
WASI / wasm32 target support
- Auto-detect /usr/share/wasi-sysroot on Linux when target starts_with("wasm32")
- Skip -march/-mtune for wasm (clang rejects them)
- Apply -fno-exceptions -fno-c++-static-destructors -mllvm -wasm-enable-sjlj
  -D_WASI_EMULATED_SIGNAL to wasm builds (compile + std PCM, kept in sync)
- .wasm output extension in expectedOutputFor and link command
- EnableWasiBrowserRuntime(cfg): opt-in helper that drops index.html +
  runtime.js next to the .wasm; runtime.js reads window.CRAFTER_WASM_URL
  set in the templated index.html so a single shim handles any output name

-r run flag in the CLI: build then exec the artifact (host targets only;
  rejects libraries; auto .exe/.wasm extension handling)

CI pipeline (.forgejo/workflows/ci.yaml)
- Triggers: PR/push to master + manual dispatch
- Single arch-latest container job: install deps, bootstrap, self-rebuild,
  run tests, cross-compile mingw, package both archives, upload artifacts
- Rolling 'latest' release published only on push/dispatch to master

mingw cross-compile from Linux now works end-to-end:
- ExternalDependency cache key includes target so per-target glslang builds
  don't collide; CMAKE_BUILD_TYPE=Release pinned (otherwise glslang appends
  'd' to lib names and breaks linking); cross-compile cmake flags
  (CMAKE_SYSTEM_NAME=Windows, CMAKE_*_COMPILER_TARGET=...)
- project.cpp accepts --target=<triple>; Linux-only -Wl,--export-dynamic
  and -ldl are gated; mingw glslang skips the standalone exe (its libgcc_eh
  link pulls pthread which mingw doesn't link by default)
- mingw compile uses -femulated-tls so std::__once_callable etc reference
  the same emutls symbols libstdc++ provides
- mingw link auto-adds -lstdc++exp -lpthread

GetCrafterBuildHome() exposed from the Platform module; LoadProject (Linux
+ Windows) now both use it instead of duplicating the resolution.

Examples reorg: hello-world, library, with-module, wasi, tests — each with
its own README. Tests reorg: per-test directory with inner/ fixture, no
shared tests/fixtures/ tree. New Wasi test verifies .wasm magic bytes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci: refresh archlinux-keyring before installing deps
Some checks failed
CI / build-test-release (pull_request) Failing after 18s
fa202c49f1
archlinux:latest container ships a snapshot keyring; packages signed by
keys added after the snapshot date fail PGP verification (zip-3.0-13 hit
this with a "signature from Robin Candau is unknown trust" error). Update
the keyring first via pacman -Sy archlinux-keyring, then -Syu the rest.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci: pacman-key --init/--populate before keyring upgrade
Some checks failed
CI / build-test-release (pull_request) Failing after 12m22s
f0b1fd899c
archlinux:latest slim image has no local pacman master key and an
unpopulated upstream keyring, so:
  - the archlinux-keyring upgrade fails with "no secret key available to
    sign with" because pacman can't sign the keyring it's rewriting
  - falling through to -Syu hits the original "unknown trust" errors on
    libseccomp and zip

Run pacman-key --init then --populate archlinux before any pacman -S.
This is the documented bootstrap for slim Arch CI containers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci: pin march to x86-64-v2, drop upload-artifact to v3
Some checks failed
CI / build-test-release (pull_request) Failing after 12m27s
87a64d3b3c
Two CI fixes from run #276 (got all the way through tests + mingw cross-
compile + packaging, only failed on artifact upload):

march: workflow now sets CRAFTER_BUILD_MARCH=x86-64-v2 / MTUNE=generic;
project.cpp reads both and applies them to the lib + exe Configurations
so the self-rebuild and mingw cross-compile honor the same baseline.
v3 is unusable on the runner — Intel N5105 (Tremont) has no AVX2, so
a v3 bootstrap binary wouldn't even start. v2 (SSE4.2) runs on the SBC
and on every x86_64 CPU since ~2011.

upload-artifact: pinned to v3. v4+ uses a GHES-only API that Forgejo
Actions doesn't implement; the v3 action stays on the older API that
Forgejo supports.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci: package mingw output dir by march, not 'native'
All checks were successful
CI / build-test-release (pull_request) Successful in 12m30s
f1199429b7
Setting CRAFTER_BUILD_MARCH=x86-64-v2 made crafter-build emit the
mingw cross-compile to bin/crafter.build-exe-x86_64-w64-mingw32-x86-64-v2/
but the package step still hardcoded -native, causing cp to fail.

Use ${CRAFTER_BUILD_MARCH} in the path so workflow env and packaging
stay in sync.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cross-compiled mingw artifact: full DLL+launcher pattern + MSVC target
Some checks failed
CI / build-test-release (pull_request) Has been cancelled
bed4a7c9e4
Linux→mingw cross-compile now produces the same architectural shape as
build.cmd (DLL + import lib + launcher exe) instead of a single static
binary. The CI Windows artifact becomes a first-class drop-in: a user
on Windows can run crafter-build.exe against any project.cpp and have
it produce real Windows binaries — for either mingw or MSVC ABI.

What changed:

project.cpp: when target=mingw or target=msvc, crafter.build-lib is
built as LibraryDynamic instead of LibraryStatic so the link emits a
DLL + import lib (matching what build.cmd produces natively).

Crafter.Build-Clang.cpp Build():
- LibraryDynamic now branches per target — mingw emits <name>.dll +
  lib<name>.dll.a via lld --out-implib; msvc emits <name>.dll +
  <name>.lib via /IMPLIB; unix unchanged.
- expectedOutputFor returns .dll for Windows-target dynamic libs.
- Executable on Windows host now branches per target: mingw target
  uses simple link (no -lc++/-nostdlib++/LIBCXX_DIR), msvc target keeps
  the existing path. Both auto-copy LibraryDynamic dep DLLs + import
  libs alongside the launcher exe (Windows resolves DLLs from the exe's
  own directory at load time).
- Mingw-target Executables get -D CRAFTER_BUILD_DLL_IMPORT so
  CRAFTER_API resolves to dllimport in their PCMs.
- mingw link adds -static-libstdc++ -static-libgcc -Wl,-Bstatic
  -lpthread so produced .exe/.dll don't depend on a particular
  libstdc++-6.dll / libwinpthread-1.dll being on the consumer's PATH
  (avoids the Arch UCRT vs msys2 UCRT vs msys2 MSVCRT ABI rabbit hole).
  Drops the old auto-copy of /usr/x86_64-w64-mingw32/bin/*.dll which
  is now dead weight.
- -r flag resolves to an absolute path before std::system, otherwise
  cmd.exe rejects "./bin/..." with "'.' is not recognized...".

Crafter.Build-Platform.cpp:
- Split the Windows-host block into shared shell helpers (#if MSVC ||
  MINGW) plus separate #if MSVC and #if MINGW blocks for LoadProject /
  EnsureCrafterBuildPcms / GetBaseCommand / BuildStdPcm.
- Mingw-host LoadProject compiles project.cpp with --target=mingw,
  --sysroot=C:\msys64\ucrt64 (default; override with CRAFTER_MINGW_DIR),
  -femulated-tls, -Wl,--export-all-symbols (mingw-lld doesn't accept
  /EXPORT:NAME), and links against libcrafter-build.dll.a from the
  launcher's directory.
- Mingw-host GetBaseCommand and BuildStdPcm dispatch on config.target
  so a mingw-host crafter-build can also build msvc-target outputs
  (uses LIBCXX_DIR + libc++ headers, same as native build.cmd) when
  the user sets cfg.target = "x86_64-pc-windows-msvc".

README adds a Quick start (Windows) section covering both build paths
(native MSVC via build.cmd and the cross-compiled mingw artifact),
documenting the msys2 UCRT toolchain prerequisite.

Verified end-to-end on the winvm:
- mingw target: cross-compiled crafter-build.exe builds hello-world's
  project.cpp, compiles main.cpp, links a hello.exe that runs without
  any custom PATH (only Windows system DLLs needed).
- msvc target: same crafter-build.exe builds an MSVC-ABI hello.exe
  linked against c++.dll (auto-copied from LIBCXX_DIR), runs cleanly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci: ship per-march variants (v2/v3/v4) as individual artifacts
Some checks failed
CI / build-test-release (pull_request) Failing after 14m50s
CI / build-test-release (push) Failing after 14m45s
af41ee5084
Replaces the single Linux + single Windows tarball with six self-
contained variants — three µarch levels × two OSes. End users on the
Releases page see them as six individually-downloadable archives:

  crafter-build-{linux,windows}-x86_64-{v2,v3,v4}.{tar.gz,zip}

Pick the v-level your CPU supports:
  v2: SSE4.2 baseline (every x86_64 since ~2011)
  v3: AVX2 + FMA + BMI (Intel Haswell+, AMD Excavator+, ~2013+)
  v4: AVX-512 (Skylake-X+, recent server-class)

How: the bootstrap binary stays at v2 because the CI SBC (N5105 / Tremont,
no AVX) can't execute v3 or v4 instructions and would crash before it
could rebuild itself for higher levels. Once bootstrap's done, the same
v2 binary is re-invoked with CRAFTER_BUILD_MARCH overridden per variant
to produce v2/v3/v4 outputs. Same again with --target=x86_64-w64-mingw32
for the Windows variants. ExternalDependency cache keys on (url, target,
march) already so glslang gets built per variant; subsequent CI runs hit
the cache.

Also drops the wrapper zip from workflow artifacts: instead of one
upload-artifact call producing crafter-build.zip containing six archives,
there's now one call per archive. The PR / run page shows six small
downloads named for what they are.

forgejo-release continues to upload release-dir/* as individual assets,
so the Releases page already has the right shape — this just brings the
workflow artifact UX in line.

Bumped the glslang cache key to v2 (one-time invalidation) since the
cache layout now needs space for six glslang builds rather than two.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign in to join this conversation.
No description provided.