Add HTTP/1.1 client and server #2

Merged
catbot merged 4 commits from claude/issue-1 into master 2026-07-27 01:05:16 +00:00
4 changed files with 102 additions and 8 deletions
Showing only changes of commit a79ab6a024 - Show all commits

docs(http1): document the HTTP/1.1 stack, and expose the client's timeout

README: HTTP/1.1 in the intro, feature list, module list, browser-build
exclusions, dependencies and test list, plus a Components section
covering both classes, the standalone codec, what is and is not
implemented, the smuggling-shaped inputs that are rejected, and an
explicit note that this path is plaintext and belongs behind a TLS
terminator.

ClientHTTP1::timeout was hard-coded and invisible; make it a public
member alongside `limits`, mirroring the listener's timeouts.

Also pipelining coverage in ShouldSendRecieveHTTP1: two requests written
before either is answered, driven from a raw socket since ClientHTTP1
waits for each response.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
catbot 2026-07-27 01:02:37 +00:00

View file

@ -1,16 +1,17 @@
# Crafter.Network
A cross-platform C++ networking library providing TCP, QUIC, HTTP/3, and WebTransport client/server functionality with modern C++ features. Builds for native Linux and for the browser (wasm32-wasip1).
A cross-platform C++ networking library providing TCP, QUIC, HTTP/3, HTTP/1.1, and WebTransport client/server functionality with modern C++ features. Builds for native Linux and for the browser (wasm32-wasip1).
## Overview
Crafter.Network is a C++ networking library designed for modern C++ applications. It provides TCP, QUIC, HTTP/3, and WebTransport-over-HTTP/3 capabilities with support for synchronous and asynchronous operations, making it suitable for a wide range of networking tasks including real-time multiplayer games. The same source compiles for native Linux (via msquic + POSIX sockets) and for the browser (via `fetch()` + `WebTransport` JS APIs); see [Browser build](#browser-build).
Crafter.Network is a C++ networking library designed for modern C++ applications. It provides TCP, QUIC, HTTP/3, HTTP/1.1, and WebTransport-over-HTTP/3 capabilities with support for synchronous and asynchronous operations, making it suitable for a wide range of networking tasks including real-time multiplayer games. The same source compiles for native Linux (via msquic + POSIX sockets) and for the browser (via `fetch()` + `WebTransport` JS APIs); see [Browser build](#browser-build).
## Features
- **TCP Networking**: Client and server implementations for raw TCP connections (native only).
- **QUIC Networking**: Encrypted, multi-stream transport via msquic — reliable streams for control plane, unreliable datagrams for low-latency state sync.
- **HTTP/3**: Client and server implementations on top of QUIC. Uses ALPN `h3`, QUIC bidi streams for requests/responses, the mandatory unidirectional control stream + SETTINGS frame (RFC 9114 §6.2.1), the (empty) QPACK encoder + decoder unidi streams required by stricter peers like Chromium, and a built-in QPACK codec (RFC 9204) with the full static table, Huffman *decoding* (RFC 7541), and literal-only emission. The QPACK dynamic table is unused. The client is interoperable with mainstream public h3 endpoints (cloudflare, nghttp3-based servers, etc.).
- **HTTP/1.1**: Client and server over plain TCP (RFC 9112), for the large part of the world that is not ready for HTTP/3 — old proxies, CI tooling, load balancers, `curl` scripts. Shares the `HTTPRequest`/`HTTPResponse` types and the route-map API with the HTTP/3 stack, so a handler or call site moves between the two by changing the class name. Keep-alive and pipelining, `content-length` and `chunked` bodies with trailers, `Expect: 100-continue`, HEAD, automatic `Date`, and per-connection timeouts. Plaintext only — see [HTTP/1.1 Components](#http11-components).
- **WebTransport (server)**: `ListenerHTTP` accepts extended-CONNECT sessions (`:method=CONNECT, :protocol=webtransport`) negotiated on the existing h3 listener — no separate port or alternate stack. Both draft-02 and draft-07+ identifier sets are advertised in SETTINGS so current Chrome/Edge browsers connect out of the box. Per-route handlers receive a `WebTransportSession&` and can multiplex bidirectional streams over the session.
- **Browser client**: Same C++ API compiled to wasm32-wasip1 and routed through `fetch()` (for `ClientHTTP`) and `WebTransport` (for `ClientQUIC`). Listeners and raw TCP are not compiled in the browser build — the browser is client-only.
- **Asynchronous Operations**: Thread poolbased async operations on native; the same `*Async` API on the browser side, where it's required (no synchronous I/O in the browser event loop).
@ -27,7 +28,10 @@ The library follows a modular design using C++20 modules:
- `Crafter.Network:ListenerTCP`: TCP server implementation (native only)
- `Crafter.Network:ClientHTTP`: HTTP/3 client (ALPN `h3`). On browser builds this maps to `fetch()`.
- `Crafter.Network:ListenerHTTP`: HTTP/3 + WebTransport server (ALPN `h3`, native only)
- `Crafter.Network:HTTP`: HTTP request/response types and constructors
- `Crafter.Network:HTTP`: HTTP request/response types and constructors, shared by every HTTP version
- `Crafter.Network:ClientHTTP1`: HTTP/1.1 client over TCP (native only)
- `Crafter.Network:ListenerHTTP1`: HTTP/1.1 server over TCP (native only)
- `Crafter.Network:HTTP1`: HTTP/1.1 wire format — serialisation plus an incremental parser (RFC 9112). Transport-free; usable on its own to speak HTTP/1.1 over some other byte stream. Native only, for the same reason as `:HTTP3`
- `Crafter.Network:ClientQUIC`: QUIC connection (client + accepted-server side) with reliable streams and unreliable datagrams. On browser builds this maps to the `WebTransport` JS API.
- `Crafter.Network:ListenerQUIC`: QUIC listener accepting incoming connections (native only). Also exports `ComputeCertificateHashSHA256()` and `GetSelfSignedCertificatePath()` for browser-peer cert pinning.
- `Crafter.Network:WebTransport`: `WebTransportSession` type — the per-session handle handed to `ListenerHTTP` WT route handlers.
@ -89,6 +93,58 @@ listener.Listen();
The `HTTPRequest` exposes the four HTTP/3 pseudo-headers (`method`, `scheme`, `authority`, `path`) as named struct fields rather than mixing them into the regular `headers` map. Routes are dispatched by exact match on `path`; unmatched paths return a synthetic 404.
### HTTP/1.1 Components
HTTP/3 is not something every peer can be made to speak. `ClientHTTP1` and `ListenerHTTP1` provide the same API over plain TCP, using the same `HTTPRequest`/`HTTPResponse` types and the same route-map shape, so a handler can be registered with both and served over either protocol.
#### ClientHTTP1
```cpp
Crafter::ClientHTTP1 client("localhost", 8080);
Crafter::HTTPResponse response = client.Send(
Crafter::CreateRequestHTTP("GET", "/", "localhost")
);
```
The connection is persistent: the first `Send()` dials, and later calls reuse the socket unless the peer asked for it to be closed. A pooled connection that the peer closed while it looked idle — the race HTTP/1.1 keep-alive cannot avoid — is redialled once and the request replayed; nothing is replayed once a response byte has arrived, and a freshly dialled connection is never retried on, so a genuinely broken server surfaces as an exception rather than a retry loop. `authority` defaults to the host:port the client was constructed with (the port is elided when it is 80).
#### ListenerHTTP1
```cpp
std::unordered_map<std::string,
std::function<Crafter::HTTPResponse(const Crafter::HTTPRequest&)>> routes;
routes["/hello"] = [](const Crafter::HTTPRequest&) {
return Crafter::CreateResponseHTTP("200", "Hello World!");
};
Crafter::ListenerAsyncHTTP1 listener(8080, std::move(routes));
```
Each accepted connection gets its own thread and is served sequentially until the peer closes it, a `Connection: close` is seen, or a timeout expires (`keepAliveTimeout`, default 15 s between requests; `requestTimeout`, default 30 s for one request to arrive). A dedicated thread rather than a ThreadPool task is deliberate: keep-alive connections are idle most of their life and would otherwise pin every pool thread.
Routing matches `path` exactly and then falls back to the query-stripped path, so `/thing?x=1` reaches the handler registered for `/thing` while the handler still sees the full target in `request.path`. A handler that throws becomes a 500; an unknown path a 404; a request we refuse to parse a 400. `Date` is stamped automatically unless the handler set one, HEAD returns the headers a GET would have produced with no body, and a handler can end the connection by answering with a `connection: close` header.
Implemented: keep-alive, pipelining, `content-length` and `chunked` request bodies with trailers, `Expect: 100-continue`, absolute-form request targets, obs-fold, and HTTP/1.0 peers (which only get connection reuse when they ask for it). Not implemented: TLS, CONNECT tunnels, `Upgrade`, and chunked *responses* — handlers return a complete body, so responses are always `content-length` framed.
Ambiguous framing is rejected rather than guessed at, because guessing is how request smuggling happens (RFC 9112 §11.2): `Content-Length` together with `Transfer-Encoding`, disagreeing duplicate `Content-Length` values, and whitespace between a field name and its colon are all 400s. CR/LF in a header value we are asked to *send* throws instead of splitting the message.
#### No TLS
`ClientHTTP1`/`ListenerHTTP1` speak `http://` only. There is no `https://` support and no plan to link a TLS stack into this path: for encrypted traffic use `ClientHTTP`/`ListenerHTTP` (HTTP/3 over QUIC, which is always encrypted), or terminate TLS in a proxy in front of the HTTP/1.1 endpoint. Do not expose an HTTP/1.1 listener directly to the internet.
#### HTTP/1.1 wire format on its own
`Crafter::HTTP1` exposes the codec without the transport — `SerializeRequest` / `SerializeResponse` and an incremental `MessageParser` that takes arbitrary byte chunks and yields one message at a time. Useful for speaking HTTP/1.1 over a byte stream this library does not own.
```cpp
Crafter::HTTP1::MessageParser parser(Crafter::HTTP1::MessageKind::Response);
parser.SetRequestMethod("GET"); // framing depends on the request method
parser.Feed(chunk.data(), chunk.size());
if (parser.Complete()) {
Crafter::HTTPResponse response = parser.TakeResponse();
parser.Reset(); // rearm; keeps any pipelined bytes
}
```
### WebTransport Components
`ListenerHTTP` has a WT-aware constructor overload that takes a second route map keyed by `:path`. When the map is non-empty the listener advertises both draft-02 (`SETTINGS_ENABLE_WEBTRANSPORT = 0x2b603742`) and draft-07+ (`SETTINGS_WT_MAX_SESSIONS = 0xc671706a`) identifiers in its SETTINGS frame so current browsers connect. An extended-CONNECT request (`:method=CONNECT, :protocol=webtransport`) whose `:path` matches a registered route is accepted with a `200` (no FIN), upgraded into a `WebTransportSession`, and dispatched on the ThreadPool. Plain HTTP/3 routes and WT routes coexist on the same listener and port.
@ -135,7 +191,7 @@ Crafter.Network compiles for `wasm32-wasip1` via [Crafter.Build](https://forgejo
- `CRAFTER_NETWORK_BROWSER` is defined. Synchronous methods on `ClientHTTP` / `QUICStream` / `ClientQUIC` are not compiled — only the `*Async` variants are available.
- `ClientHTTP` calls into `crafterNetworkFetch` (JS) which delegates to `fetch()`. An empty `host` is a same-origin sentinel: the path is passed through as the URL, so `ClientHTTP("", 0).SendAsync({.path="/data.json"}, ...)` fetches from the page origin.
- `ClientQUIC` calls into `crafterNetworkWtConnect` which constructs a `WebTransport(url, opts)` against `https://{host}:{port}/{alpn}` (i.e. `alpn` is the WebTransport URL path on this target). `QUICClientCredentials::serverCertificateHash` is forwarded as `serverCertificateHashes`; leaving it zeroed makes the browser fall back to its normal trust store.
- `ListenerTCP`, `ListenerHTTP`, `ListenerQUIC`, `ClientTCP`, and the sync receive/send paths are excluded — the browser is client-only.
- `ListenerTCP`, `ListenerHTTP`, `ListenerQUIC`, `ClientTCP`, and the sync receive/send paths are excluded — the browser is client-only. So are `ClientHTTP1` / `ListenerHTTP1` / the `HTTP1` codec: a page cannot open a raw TCP socket anyway, and `fetch()` behind `ClientHTTP` already negotiates whatever HTTP version the server offers.
- `additional/network-env.js` is shipped alongside the produced `.wasm` and merged into the runtime's `env` import object by `EnableWasiBrowserRuntime`.
A worked example pairing a wasm browser client with a native server lives in [examples/SimpleClient/](examples/SimpleClient/). Build the server with `crafter-build --target=x86_64-pc-linux-gnu`, run it, then run `crafter-build` (no `--target`) to produce the wasm and serve it over HTTPS.
@ -158,8 +214,14 @@ The library includes tests covering:
- QUIC reliable streams (`ShouldSendRecieveQUICStream`)
- QUIC unreliable datagrams (`ShouldSendRecieveQUICDatagram`)
- WebTransport echo (`ShouldEchoWebTransport`) — extended-CONNECT acceptance, draft-02 SETTINGS, bidi data stream framing (`WT_STREAM 0x41` + session-id varint), and byte-for-byte echo
- HTTP/1.1 wire format (`ShouldParseHTTP1`) — serialisation, incremental parsing fed one byte at a time, chunked bodies with trailers, pipelining, interim 1xx, HTTP/1.0 and HEAD framing, and the malformed inputs the parser is required to reject
- HTTP/1.1 round-trip (`ShouldSendRecieveHTTP1`) — routing, query strings, HEAD, 404 and a throwing handler
- HTTP/1.1 keep-alive (`ShouldSendRecieveKeepaliveHTTP1`) — connection reuse, handler-requested close, and recovery from a pooled connection the server closed
- HTTP/1.1 large body transfer (`ShouldSendRecieveLargeHTTP1`) — 10 MiB in both directions on one connection
- HTTP/1.1 interop (`ShouldInteropCurlHTTP1`) — `curl` against `ListenerHTTP1` (keep-alive reuse, chunked upload, `Expect: 100-continue`, HEAD) and `ClientHTTP1` against python3's `http.server`, which answers HTTP/1.0 with `Connection: close`
- HTTP/1.1 under abuse (`ShouldSurviveAbuseHTTP1`) — 24 concurrent keep-alive clients, peers that vanish mid-request or send garbage, and a stalled peer that must be timed out
The external-interop test requires outbound UDP/443; if your network blocks it the test will fail.
The external-interop test requires outbound UDP/443; if your network blocks it the test will fail. `ShouldInteropCurlHTTP1` skips whichever half is unavailable when `curl` or `python3` is not installed, so it passes on a bare machine — install both to actually exercise it.
## Dependencies
@ -167,6 +229,7 @@ The external-interop test requires outbound UDP/443; if your network blocks it t
- **msquic** (native target only) — fetched and built automatically as a Crafter `ExternalDependency` (no system install required). The build clones `microsoft/msquic` recursively into the per-project external cache, configures it via CMake (`QUIC_TLS_LIB=quictls`, tests/tools/perf disabled), and links the produced `libmsquic` into the QUIC and HTTP/3 modules. Skipped entirely on browser builds.
- On Linux msquic links against `libnuma` (provided by the `numactl` package on most distros).
- The self-signed-cert path used by tests/dev shells out to the `openssl` CLI; install `openssl` if you intend to use `QUICServerCredentials{selfSigned = true}`. The same path also produces the cert hash that browser peers need for `serverCertificateHashes`.
- The HTTP/1.1 stack has no dependencies beyond POSIX sockets — no TLS library, no msquic.
- **Browser build** has no extra dependencies beyond Crafter.Build's `wasi-browser` runtime: HTTP delegates to the browser's `fetch()`, QUIC to its `WebTransport`. The JS glue lives in `additional/network-env.js` and is shipped alongside the produced `.wasm`.
## Usage Example

View file

@ -54,7 +54,6 @@ struct ClientHTTP1::Impl {
std::string host;
std::uint16_t port;
std::unique_ptr<ClientTCP> tcp;
std::chrono::milliseconds timeout{30000};
void Connect() {
tcp = std::make_unique<ClientTCP>(host, port);
@ -68,7 +67,8 @@ struct ClientHTTP1::Impl {
// is set as soon as any response byte arrives, which is what decides
// whether a failure is safe to replay on a new connection.
HTTPResponse Exchange(const std::string& wire, std::string_view method,
const HTTP1::MessageLimits& limits, bool& received) {
const HTTP1::MessageLimits& limits,
std::chrono::milliseconds timeout, bool& received) {
tcp->Send(wire.data(), static_cast<std::uint32_t>(wire.size()));
HTTP1::MessageParser parser(HTTP1::MessageKind::Response, limits);
@ -127,7 +127,7 @@ HTTPResponse ClientHTTP1::Send(const HTTPRequest& request) {
bool received = false;
try {
return impl->Exchange(wire, method, limits, received);
return impl->Exchange(wire, method, limits, timeout, received);
} catch (const std::exception&) {
impl->Close();
if (attempt == 0 && reused && !received) continue;

View file

@ -60,6 +60,9 @@ namespace Crafter {
// Limits applied to responses. Set before the first Send().
HTTP1::MessageLimits limits;
// How long to wait for the next piece of a response before giving
// up on a server that accepted the connection and then went quiet.
std::chrono::milliseconds timeout{30000};
private:
struct Impl;

View file

@ -73,6 +73,34 @@ int main() {
// Every exchange above shared one connection.
Check(listener.listener.AcceptedCount() == 1, "the whole test used a single connection");
// Pipelining: two requests written in one go, before either is
// answered. The responses must come back in order, on that same
// connection. ClientHTTP1 never does this — it waits for each
// response — so drive it from a raw socket.
{
ClientTCP socket("localhost", 8090);
const std::string pipelined =
"GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"
"GET /nope HTTP/1.1\r\nHost: localhost\r\n\r\n";
socket.Send(pipelined.data(), static_cast<std::uint32_t>(pipelined.size()));
HTTP1::MessageParser parser(HTTP1::MessageKind::Response);
std::string first;
std::string second;
while (second.empty()) {
std::vector<char> chunk = socket.RecieveSync();
parser.Feed(chunk.data(), chunk.size());
while (parser.Complete()) {
HTTPResponse response = parser.TakeResponse();
(first.empty() ? first : second) = response.status;
parser.Reset();
if (!second.empty()) break;
}
}
Check(first == "200", "first pipelined response");
Check(second == "404", "second pipelined response, in order");
}
listener.Stop();
} catch (const std::exception& error) {
std::println("threw: {}", error.what());