Add HTTP/1.1 client and server #2

Merged

catbot merged 4 commits from claude/issue-1 into master

2026-07-27 01:05:16 +00:00

catbot commented

2026-07-27 01:05:11 +00:00

Member

Adds an HTTP/1.1 client and server so the library can talk to the large part of the world that is not ready for HTTP/3 — old proxies, load balancers, CI tooling, curl scripts.

Resolves #1

What this adds

ClientHTTP1 and ListenerHTTP1 use the same HTTPRequest/HTTPResponse types and the same route-map shape as the HTTP/3 stack, so a handler or call site moves between the two protocols by changing the class name:

Crafter::ListenerAsyncHTTP1 listener(8080, std::move(routes));   // same routes as ListenerHTTP
Crafter::ClientHTTP1 client("localhost", 8080);
auto response = client.Send(Crafter::CreateRequestHTTP("GET", "/", "localhost"));

Three new module partitions, all native-only (a browser page cannot open a raw TCP socket, and fetch() behind ClientHTTP already negotiates whatever version the server offers):

:HTTP1 — the wire format on its own, with no transport dependency. Serialisation with the framing headers owned by the serialiser rather than the caller, plus an incremental MessageParser that takes arbitrary socket chunks and yields one message at a time.
:ClientHTTP1 — persistent connection, redialling once when a pooled connection turns out to have been closed by the peer. Nothing is replayed after a response byte has arrived, and a freshly dialled connection is never retried on, so a genuinely broken server surfaces as an exception rather than a retry loop.
:ListenerHTTP1 — one thread per connection. Deliberately not a ThreadPool task: keep-alive connections are idle most of their life and would otherwise pin every pool thread.

Covered: keep-alive, pipelining, content-length and chunked bodies with trailers, Expect: 100-continue, HEAD, absolute-form targets, obs-fold, interim 1xx responses, read-to-EOF responses, HTTP/1.0 peers, automatic Date, handler-requested close, per-connection timeouts, and 400/404/500. Routing falls back to the query-stripped path, so /thing?x=1 reaches the handler for /thing while the handler still sees the full target.

Ambiguous framing is rejected rather than guessed at, because guessing is how request smuggling happens (RFC 9112 §11.2): Content-Length with Transfer-Encoding, disagreeing duplicate Content-Length values, and whitespace before a header's colon are all 400s. CR/LF in a header value we are asked to send throws instead of splitting the message.

No TLS. This is http:// only, and the README says so plainly: for encrypted traffic use HTTP/3 over QUIC, or terminate TLS in a proxy in front of it.

Fixes to the TCP layer underneath

The HTTP/1.1 stack sits on ClientTCP/ListenerTCP and each of these bit it:

gethostbyname() returning null on an unresolvable host was dereferenced straight into a crash, and it is not thread safe → getaddrinfo.
A failed socket()/connect() only printed to stderr and handed back an unusable ClientTCP, so the real error surfaced much later as an unrelated errno.
send() was assumed to accept everything offered. It does not once a buffer outgrows the socket's send buffer, which silently truncated multi-megabyte bodies. Now loops, and passes MSG_NOSIGNAL so a vanished peer raises EPIPE instead of killing the process.
ClientTCP's move constructor closed the socket it had just taken ownership of, and it and the destructor both tested socketid != 1 where they meant != -1.
ListenerTCP ignored bind()'s result — leaving a listener that accepted nothing with no explanation — and did not set SO_REUSEADDR, so a restart hit EADDRINUSE for the length of TIME_WAIT.

One bug in the new code was found by its own test and is fixed in ea1310f: a finished connection's socket was held until the next accept reaped it, so a peer we had finished with never saw EOF and sat waiting for a server that was done talking.

Tests

Six new tests, all passing:

Test	Covers
`ShouldParseHTTP1`	The codec, fed one byte at a time: chunked bodies with trailers, pipelining, interim 1xx, HTTP/1.0 and HEAD framing, serialise-then-parse round-trip, and every malformed input the parser must reject
`ShouldSendRecieveHTTP1`	Round-trip: routing, query strings, HEAD, 404, a throwing handler, and pipelining driven from a raw socket
`ShouldSendRecieveKeepaliveHTTP1`	Connection reuse, handler-requested close, and recovery from a pooled connection the server closed underneath the client
`ShouldSendRecieveLargeHTTP1`	10 MiB in both directions on one connection, with a non-uniform payload so a mangled offset cannot compare equal
`ShouldInteropCurlHTTP1`	Real peers both ways: `curl` against the listener (keep-alive reuse, chunked upload, `Expect: 100-continue` verified from curl's own trace, HEAD, status codes), and the client against python3's `http.server`, which answers HTTP/1.0 with `Connection: close`
`ShouldSurviveAbuseHTTP1`	24 concurrent keep-alive clients × 8 requests, peers that vanish mid-request or send garbage, an over-long request line, and a stalled peer that must be timed out — then the server must still serve correctly and stop promptly

13 passed, 1 timed out

About that one: ShouldSend is the pre-existing live-interop test against cloudflare-quic.com:443. It fails here because this sandbox's egress allowlist blocks outbound UDP/443 — curl --http3-only https://cloudflare-quic.com/ cannot connect either, and the test times out identically on master before any change in this branch. The README already documents that this test requires outbound UDP/443. Nothing in this PR touches the QUIC or HTTP/3 path.

Note for reviewers

While working on this I hit — and filed upstream as Crafter.Build#26 — a build-system bug: when a library's module interface changes, crafter-build test relinks the test but does not recompile its main.cpp, producing ABI-mismatch crashes that look like bugs in the code under test. If you see unexplained SIGSEGVs after editing a .cppm, rm -rf build bin first. The numbers above are from a clean build.

Adds an HTTP/1.1 client and server so the library can talk to the large part of the world that is not ready for HTTP/3 — old proxies, load balancers, CI tooling, `curl` scripts. Resolves #1 ## What this adds `ClientHTTP1` and `ListenerHTTP1` use the same `HTTPRequest`/`HTTPResponse` types and the same route-map shape as the HTTP/3 stack, so a handler or call site moves between the two protocols by changing the class name: ```cpp Crafter::ListenerAsyncHTTP1 listener(8080, std::move(routes)); // same routes as ListenerHTTP Crafter::ClientHTTP1 client("localhost", 8080); auto response = client.Send(Crafter::CreateRequestHTTP("GET", "/", "localhost")); ``` Three new module partitions, all native-only (a browser page cannot open a raw TCP socket, and `fetch()` behind `ClientHTTP` already negotiates whatever version the server offers): - **`:HTTP1`** — the wire format on its own, with no transport dependency. Serialisation with the framing headers owned by the serialiser rather than the caller, plus an incremental `MessageParser` that takes arbitrary socket chunks and yields one message at a time. - **`:ClientHTTP1`** — persistent connection, redialling once when a pooled connection turns out to have been closed by the peer. Nothing is replayed after a response byte has arrived, and a freshly dialled connection is never retried on, so a genuinely broken server surfaces as an exception rather than a retry loop. - **`:ListenerHTTP1`** — one thread per connection. Deliberately not a ThreadPool task: keep-alive connections are idle most of their life and would otherwise pin every pool thread. Covered: keep-alive, pipelining, `content-length` and `chunked` bodies with trailers, `Expect: 100-continue`, HEAD, absolute-form targets, obs-fold, interim 1xx responses, read-to-EOF responses, HTTP/1.0 peers, automatic `Date`, handler-requested close, per-connection timeouts, and 400/404/500. Routing falls back to the query-stripped path, so `/thing?x=1` reaches the handler for `/thing` while the handler still sees the full target. Ambiguous framing is rejected rather than guessed at, because guessing is how request smuggling happens (RFC 9112 §11.2): `Content-Length` with `Transfer-Encoding`, disagreeing duplicate `Content-Length` values, and whitespace before a header's colon are all 400s. CR/LF in a header value we are asked to *send* throws instead of splitting the message. **No TLS.** This is `http://` only, and the README says so plainly: for encrypted traffic use HTTP/3 over QUIC, or terminate TLS in a proxy in front of it. ## Fixes to the TCP layer underneath The HTTP/1.1 stack sits on `ClientTCP`/`ListenerTCP` and each of these bit it: - `gethostbyname()` returning null on an unresolvable host was dereferenced straight into a crash, and it is not thread safe → `getaddrinfo`. - A failed `socket()`/`connect()` only printed to stderr and handed back an unusable `ClientTCP`, so the real error surfaced much later as an unrelated errno. - `send()` was assumed to accept everything offered. It does not once a buffer outgrows the socket's send buffer, which silently truncated multi-megabyte bodies. Now loops, and passes `MSG_NOSIGNAL` so a vanished peer raises `EPIPE` instead of killing the process. - `ClientTCP`'s move constructor closed the socket it had just taken ownership of, and it and the destructor both tested `socketid != 1` where they meant `!= -1`. - `ListenerTCP` ignored `bind()`'s result — leaving a listener that accepted nothing with no explanation — and did not set `SO_REUSEADDR`, so a restart hit `EADDRINUSE` for the length of TIME_WAIT. One bug in the new code was found by its own test and is fixed in `ea1310f`: a finished connection's socket was held until the *next* accept reaped it, so a peer we had finished with never saw EOF and sat waiting for a server that was done talking. ## Tests Six new tests, all passing: | Test | Covers | |---|---| | `ShouldParseHTTP1` | The codec, fed one byte at a time: chunked bodies with trailers, pipelining, interim 1xx, HTTP/1.0 and HEAD framing, serialise-then-parse round-trip, and every malformed input the parser must reject | | `ShouldSendRecieveHTTP1` | Round-trip: routing, query strings, HEAD, 404, a throwing handler, and pipelining driven from a raw socket | | `ShouldSendRecieveKeepaliveHTTP1` | Connection reuse, handler-requested close, and recovery from a pooled connection the server closed underneath the client | | `ShouldSendRecieveLargeHTTP1` | 10 MiB in both directions on one connection, with a non-uniform payload so a mangled offset cannot compare equal | | `ShouldInteropCurlHTTP1` | Real peers both ways: `curl` against the listener (keep-alive reuse, chunked upload, `Expect: 100-continue` verified from curl's own trace, HEAD, status codes), and the client against python3's `http.server`, which answers HTTP/1.0 with `Connection: close` | | `ShouldSurviveAbuseHTTP1` | 24 concurrent keep-alive clients × 8 requests, peers that vanish mid-request or send garbage, an over-long request line, and a stalled peer that must be timed out — then the server must still serve correctly and stop promptly | ``` 13 passed, 1 timed out ``` **About that one:** `ShouldSend` is the pre-existing live-interop test against `cloudflare-quic.com:443`. It fails here because this sandbox's egress allowlist blocks outbound UDP/443 — `curl --http3-only https://cloudflare-quic.com/` cannot connect either, and the test times out identically on `master` before any change in this branch. The README already documents that this test requires outbound UDP/443. Nothing in this PR touches the QUIC or HTTP/3 path. ## Note for reviewers While working on this I hit — and filed upstream as [Crafter.Build#26](https://forgejo.catcrafts.net/Catcrafts/Crafter.Build/issues/26) — a build-system bug: when a library's module interface changes, `crafter-build test` relinks the test but does not recompile its `main.cpp`, producing ABI-mismatch crashes that look like bugs in the code under test. If you see unexplained SIGSEGVs after editing a `.cppm`, `rm -rf build bin` first. The numbers above are from a clean build.

catbot added 4 commits

2026-07-27 01:05:11 +00:00

fix(tcp): resolve, connect, bind and send failures were silent or wrong b758419007

The HTTP/1.1 stack sits directly on these two classes and each of these
bit it:

- gethostbyname() returning null on an unresolvable host was dereferenced
  straight into a crash, and it is not thread safe; use getaddrinfo.
- A failed socket()/connect() only printed to stderr and handed back an
  unusable ClientTCP, so the real error surfaced much later as an
  unrelated errno from send().
- send() was assumed to accept everything it was offered. It does not
  once a buffer outgrows the socket's send buffer, which silently
  truncated multi-megabyte bodies. Loop, and pass MSG_NOSIGNAL so a
  vanished peer raises EPIPE instead of killing the process.
- ClientTCP's move constructor closed the socket it had just taken
  ownership of, and both it and the destructor tested `socketid != 1`
  where they meant `!= -1`.
- ListenerTCP ignored bind()'s result, leaving a listener that accepted
  nothing with no explanation, and did not set SO_REUSEADDR, so a
  restart hit EADDRINUSE for the length of TIME_WAIT.

accept() failing during Stop() is expected and no longer logged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

feat(http1): add an HTTP/1.1 client and listener 337ce32eca

HTTP/3-only is not a deployable position yet: plenty of clients, proxies
and CI tooling still speak nothing but HTTP/1.1. This adds that path
using the request/response types the HTTP/3 stack already uses, so a
route handler or call site moves between the two protocols by changing
the class name.

- :HTTP1 — transport-free wire format. Serialisation with the framing
  headers owned by the serialiser, and an incremental parser that takes
  arbitrary socket chunks and yields one message at a time: keep-alive,
  pipelining, content-length and chunked bodies (with trailers),
  read-to-EOF responses, interim 1xx skipping, HEAD/204/304 framing and
  Expect: 100-continue. Ambiguous framing is rejected rather than
  guessed at (content-length with transfer-encoding, disagreeing
  content-lengths, whitespace before a colon), and CR/LF in a value we
  are asked to serialise is refused.
- ClientHTTP1 — persistent connection, redialling once when a pooled
  connection turns out to have been closed by the peer, which is the
  race HTTP/1.1 keep-alive cannot avoid. Nothing is replayed after a
  response byte has arrived.
- ListenerHTTP1 — one thread per connection (keep-alive connections are
  idle most of their life and would pin every ThreadPool thread),
  automatic Date, HEAD, 100-continue, handler-requested close, idle and
  request timeouts, and 400/404/500 responses. Routes fall back to the
  query-stripped path so `/thing?x=1` reaches the handler for `/thing`.

No TLS: this is `http://` only. Encrypted traffic still goes over
HTTP/3, or through a TLS-terminating proxy.

Tests: codec unit tests including the malformed inputs above, a
client/server round-trip, keep-alive and stale-connection recovery, a
10 MiB body both ways, and interop both directions against curl and
python3's http.server (skipped when those are not installed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(http1): close a finished connection instead of holding it until reap ea1310faec

A connection's socket was owned by the registry entry and only released
when the next accept() reaped it, so a peer we had finished with — after
a 408, a 400, or a `connection: close` — never saw EOF and sat waiting
for a server that was done talking. On a server that goes quiet it also
held every descriptor from the last burst indefinitely.

The connection thread now closes its own socket the moment Serve()
returns, under the registry lock so Stop()'s shutdown() can never name a
descriptor that has already been released, and Stop() waits on a
condition variable for the last thread rather than assuming the vector
it moved out is quiescent. Adopt() is also fully guarded: it runs on
ListenerTCP's accept loop, which has no handler, so anything escaping it
would abort the process.

Found by ShouldSurviveAbuseHTTP1, added here: 24 concurrent keep-alive
clients, peers that vanish mid-request or send garbage, and a peer that
stalls forever — the server must keep serving and still stop promptly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs(http1): document the HTTP/1.1 stack, and expose the client's timeout a79ab6a024

README: HTTP/1.1 in the intro, feature list, module list, browser-build
exclusions, dependencies and test list, plus a Components section
covering both classes, the standalone codec, what is and is not
implemented, the smuggling-shaped inputs that are rejected, and an
explicit note that this path is plaintext and belongs behind a TLS
terminator.

ClientHTTP1::timeout was hard-coded and invisible; make it a public
member alongside `limits`, mirroring the listener's timeouts.

Also pipelining coverage in ShouldSendRecieveHTTP1: two requests written
before either is answered, driven from a raw socket since ClientHTTP1
waits for each response.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

catbot merged commit 219a31a8f7 into master

2026-07-27 01:05:16 +00:00

catbot deleted branch claude/issue-1

2026-07-27 01:05:16 +00:00

catbot referenced this pull request from a commit

2026-07-27 01:05:17 +00:00

Merge pull request 'Add HTTP/1.1 client and server' (#2) from claude/issue-1 into master