fix: line-buffer stdout when redirected so progress/readiness lines flush (#18) #19

Merged

jorijnvdgraaf merged 1 commit from claude/issue-18 into master

2026-06-01 16:50:00 +02:00

catbot commented

2026-06-01 16:43:14 +02:00

Member

Summary

When crafter-build's stdout is not a TTY (redirected to a file or pipe), the C runtime defaults to full (block) buffering. The progress path is TTY-aware and the in-place redraw flushes explicitly, but every non-TTY write goes through block-buffered stdout with no flush:

Crafter.Build-Progress.cpp — non-TTY append path std::println("[{}/{}] {}", …) (line 134)
Finalize() — std::println("Built {} step{} in {}ms", …) (line 161)
Crafter.Build-Clang.cpp — std::println("listening on port :{}", port) (line 1577)

On a normal build this is hidden: the C runtime flushes stdout at process exit. Under -r the process never exits — it blocks in its serve loop — so the trailing buffer is never flushed. A redirected log freezes mid-build (or sits at 0 bytes) even though the build finished seconds ago and the server is already answering HTTP 200. Any tooling polling the log for Built … / listening … / [N/N] hangs forever.

This is the real cause of the "frozen log" misdiagnosed as a build deadlock in #16.

Fix

Switch stdout to line buffering at the very top of main(), before any output, only when stdout is not a terminal:

if (CRAFTER_MAIN_ISATTY(STDOUT_FILENO) == 0)
    std::setvbuf(stdout, nullptr, _IOLBF, 0);

Every \n then flushes, so the markers reach a redirected log immediately. No behaviour change on a TTY (already line-buffered; the redraw path flushes itself).

Why self-contained in main.cpp and not a Crafter::Progress export: the self-hosting exe build compiles main.cpp (and the crafter-build sources themselves) against the installed/cached Crafter.Build module BMIs, which are listed on -fprebuilt-module-path ahead of — and therefore shadow — the freshly built local ones. A new interface symbol is invisible to the build until crafter-build is reinstalled. Using only system headers (isatty + setvbuf) sidesteps that bootstrapping wall.

Verification

Built the binary and ran the examples/wasi project under -r with stdout redirected to a file, server left running (not exited):

Old binary — log frozen while server answers:

=== /tmp/old.log (server still running) ===
<<<END>>>
bytes: 0
=== curl === HTTP 200

This fix — all lines present while server answers:

[1/2] Copying files for wasi-hello
[2/2] Building std PCM (wasm32-wasip1-native)
[3/3] Compiling main.cpp
[4/4] Linking wasi-hello
Built 4 steps in 3956ms
listening on port :8080
=== curl === HTTP 200

crafter-build test — 12 passed.

Re: PR #17 (requested review)

#18 asked me to re-check PR #17 (merged for #16), since #16's reported symptom was actually this buffering bug.

Recommendation: keep #17. It is not a no-op. The data race it fixes is genuine and independent of the buffering bug: Configuration objects are shared across the build DAG, each compiled concurrently by its own Build(), and the old recursive reset could clear a shared dependency's compiled atomic from a parent/sibling thread after that dependency's compile thread set it and exited but before an intra-config waiter ran compiled.wait(false) — a permanent block. Scoping the reset to the current config removes that race, and the added ConcurrentDependencyReset test pins the invariant deterministically (passes here as part of the green suite).

The only inaccuracy is attribution: #17's message credits it with fixing the observed #16 hang, which was in fact this stdout buffering. #17 stands on its own merits as a correctness fix; reverting would reintroduce the race. No regression observed — full suite green.

Resolves #18

## Summary When `crafter-build`'s stdout is **not a TTY** (redirected to a file or pipe), the C runtime defaults to **full (block) buffering**. The progress path is TTY-aware and the in-place redraw flushes explicitly, but every non-TTY write goes through block-buffered `stdout` with no flush: - `Crafter.Build-Progress.cpp` — non-TTY append path `std::println("[{}/{}] {}", …)` (line 134) - `Finalize()` — `std::println("Built {} step{} in {}ms", …)` (line 161) - `Crafter.Build-Clang.cpp` — `std::println("listening on port :{}", port)` (line 1577) On a normal build this is hidden: the C runtime flushes `stdout` at process exit. **Under `-r` the process never exits** — it blocks in its serve loop — so the trailing buffer is never flushed. A redirected log freezes mid-build (or sits at **0 bytes**) even though the build finished seconds ago and the server is already answering HTTP 200. Any tooling polling the log for `Built …` / `listening …` / `[N/N]` hangs forever. This is the **real** cause of the "frozen log" misdiagnosed as a build deadlock in #16. ## Fix Switch `stdout` to line buffering at the very top of `main()`, before any output, only when stdout is not a terminal: ```cpp if (CRAFTER_MAIN_ISATTY(STDOUT_FILENO) == 0) std::setvbuf(stdout, nullptr, _IOLBF, 0); ``` Every `\n` then flushes, so the markers reach a redirected log immediately. No behaviour change on a TTY (already line-buffered; the redraw path flushes itself). **Why self-contained in `main.cpp` and not a `Crafter::Progress` export:** the self-hosting exe build compiles `main.cpp` (and the crafter-build sources themselves) against the **installed/cached `Crafter.Build` module BMIs**, which are listed on `-fprebuilt-module-path` ahead of — and therefore shadow — the freshly built local ones. A new interface symbol is invisible to the build until `crafter-build` is reinstalled. Using only system headers (`isatty` + `setvbuf`) sidesteps that bootstrapping wall. ## Verification Built the binary and ran the `examples/wasi` project under `-r` with stdout redirected to a file, server left running (not exited): **Old binary** — log frozen while server answers: ``` === /tmp/old.log (server still running) === <<<END>>> bytes: 0 === curl === HTTP 200 ``` **This fix** — all lines present while server answers: ``` [1/2] Copying files for wasi-hello [2/2] Building std PCM (wasm32-wasip1-native) [3/3] Compiling main.cpp [4/4] Linking wasi-hello Built 4 steps in 3956ms listening on port :8080 === curl === HTTP 200 ``` `crafter-build test` — **12 passed**. ## Re: PR #17 (requested review) #18 asked me to re-check PR #17 (merged for #16), since #16's reported symptom was actually this buffering bug. **Recommendation: keep #17.** It is *not* a no-op. The data race it fixes is genuine and independent of the buffering bug: `Configuration` objects are shared across the build DAG, each compiled concurrently by its own `Build()`, and the old recursive reset could clear a shared dependency's `compiled` atomic from a parent/sibling thread *after* that dependency's compile thread set it and exited but *before* an intra-config waiter ran `compiled.wait(false)` — a permanent block. Scoping the reset to the current config removes that race, and the added `ConcurrentDependencyReset` test pins the invariant deterministically (passes here as part of the green suite). The only inaccuracy is attribution: #17's message credits it with fixing the *observed* #16 hang, which was in fact this stdout buffering. #17 stands on its own merits as a correctness fix; reverting would reintroduce the race. No regression observed — full suite green. Resolves #18

catbot added 1 commit

2026-06-01 16:43:14 +02:00

fix: line-buffer stdout when redirected so progress/readiness lines flush

CI / build-test-release (pull_request) Successful in 6m13s

Details

0ff9050eb3

When crafter-build's stdout is not a TTY (redirected to a file or pipe) the
C runtime defaults to full (block) buffering. The progress path is TTY-aware
and the in-place redraw flushes explicitly, but the non-TTY append path
(`[N/M]` lines), Finalize()'s `Built N steps` line and the `-r` server's
`listening on port :N` line all go through block-buffered stdout with no
flush. They accumulate in the buffer and only spill at ~4KB boundaries.

On a normal build this is hidden because the C runtime flushes stdout at
exit. Under `-r` the process never exits — it blocks in its serve loop — so
the trailing buffer is never flushed: a redirected log freezes mid-build (or
sits at 0 bytes) even though the build finished and the server is already
answering. Any tooling that polls the log for `Built …` / `listening …` /
`[N/N]` hangs forever. This is the real cause of the frozen log misdiagnosed
as a build deadlock in #16.

Fix: switch stdout to line buffering at the very top of main(), before any
output, only when stdout is not a terminal. Every `\n` then flushes, so the
markers reach a redirected log immediately. No behaviour change on a TTY.

Kept self-contained in main.cpp using system headers (isatty + setvbuf)
rather than a new Crafter::Progress export: the self-hosting exe build
compiles main.cpp against the installed/cached Crafter.Build module BMIs,
which shadow the freshly built local ones, so a new interface symbol would
not be visible without reinstalling crafter-build first.

Resolves #18

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

catbot referenced this pull request

2026-06-01 16:44:56 +02:00

Build log freezes when stdout is redirected: non-TTY output is block-buffered and never flushed (supersedes #16) #18