Crafter.Network/interfaces/Crafter.Network-ClientQUIC.cppm

340 lines
17 KiB
Text
Raw Normal View History

2026-07-22 17:55:45 +02:00
//SPDX-License-Identifier: LGPL-3.0-only
//SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
2026-05-06 04:06:17 +02:00
module;
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
2026-05-06 04:06:17 +02:00
#include <msquic.h>
2026-05-19 02:53:50 +02:00
#endif
2026-05-06 04:06:17 +02:00
export module Crafter.Network:ClientQUIC;
import std;
namespace Crafter {
export class ListenerQUIC;
export class QUICException : public std::runtime_error {
public:
using std::runtime_error::runtime_error;
};
export class QUICClosedException : public std::exception {
public:
const char* what() const noexcept override { return "QUIC connection closed"; }
};
2026-08-25 19:17:22 +02:00
// Transport knobs applied to the msquic Configuration. The defaults
// reproduce the values these were hardcoded to before they were made
// configurable, so leaving this alone keeps the previous behaviour.
//
// peerUnidiStreamCount / peerBidiStreamCount cap how many streams the
// *peer* may open toward us. msquic defaults both to 0; a server's
// HTTP/3 control + QPACK streams alone need 3, and a protocol that
// fans data out over many concurrent streams needs correspondingly
// more. Raising these costs per-stream flow-control state, so raise
// them to what a protocol actually needs rather than to a large
// round number.
//
// keepAliveIntervalMs is client-only (the listener does not set it).
// Keep it below idleTimeoutMs or the peer's idle timer will expire.
// handshakeIdleTimeoutMs bounds how long a connect attempt to an
// unreachable peer takes to fail. This one is set explicitly where it
// previously was not: ClientQUIC's constructor waits on a condition
// variable with no timeout of its own, so without a bound here a client
// dialling a dead port blocks for minutes rather than failing and
// retrying. 10s matches msquic's own documented default.
export struct QUICSettings {
std::uint32_t idleTimeoutMs = 120'000;
std::uint32_t handshakeIdleTimeoutMs = 10'000;
std::uint32_t keepAliveIntervalMs = 10'000;
std::uint16_t peerUnidiStreamCount = 16;
std::uint16_t peerBidiStreamCount = 16;
bool datagramReceiveEnabled = true;
};
2026-05-06 04:06:17 +02:00
// Server certificate sources. Pick one of (filePaths) or (selfSigned).
fix(listener): make the selfSigned QUIC certificate per user, in-process, without a race QUICServerCredentials{selfSigned} shelled out to the openssl CLI and kept the pair in the fixed /tmp/crafter-network-quic-cert/, guarded by an in-process mutex only. Three things went wrong with that: - Processes started at once (crafter-build runs tests in parallel, and inside the Flatpak they share one /tmp) each found no pair and each ran `openssl req` over the same key.pem and cert.pem, which it writes in place one after the other. A process could load one run's key with another's certificate and fail in ConfigurationLoadCredential. - /tmp is shared by every user of the machine: whoever made the directory first could keep everyone else from writing there, or plant a pair in it. - Without the openssl CLI there was no certificate at all, and ComputeCertificateHashSHA256 piped the file through it as well. The pair now lives in $XDG_CACHE_HOME/crafter-network/quic-cert/ (else ~/.cache/..., else $XDG_RUNTIME_DIR/...). Missing directories are made 0700, and the last is opened O_NOFOLLOW, refused unless the user owns it, and made 0700 if it was not. A POSIX record lock on a file in it, given up on after 10 s, covers both the check of the pair on disk and its replacement, so one process makes the pair and the rest read it; each file is written under a temporary name, fsynced and renamed into place; and the listener hands msquic the pair from memory as PKCS#12 (QUIC_CREDENTIAL_TYPE_CERTIFICATE_PKCS12) instead of the two paths, so a replacement cannot land between its reads of the key and the certificate. A pair that does not parse, whose key does not match, or that has less than a day left is replaced, which also repairs one a crash tore between the two renames. A record lock rather than flock: an flock is shared with a child forked while another thread holds it, and the child holds it for its whole life. Threads of one process are kept apart by the existing mutex, and the process keeps the pair it loaded, so all its listeners serve one certificate until it nears its end. The certificate is made in-process by :TLS in the shape the CLI produced (ECDSA P-256, ecdsa-with-SHA256, 10 days, BasicConstraints CA:FALSE, KeyUsage digitalSignature, EKU serverAuth, SANs localhost, 127.0.0.1 and ::1), with a random serial where the CLI also drew one; ComputeCertificateHashSHA256 is X509_digest over the file's certificate. The OpenSSL helpers are module-internal declarations in :TLS, so libssl stays behind that one partition, and the TLS partition's own development certificate is unchanged. As before, the pair is reused across restarts while valid, so a browser's pinned hash survives one; it is now replaced a day before it expires rather than after, so a server is not started on a certificate about to lapse. In the Flatpak, where /tmp went away with the last instance, it now survives between runs too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 01:56:44 +01:00
// selfSigned serves a development certificate made in-process and kept
// per user under $XDG_CACHE_HOME/crafter-network/quic-cert/, reused
// while valid (see GetSelfSignedCertificatePath) — fine for dev/LAN.
2026-05-06 04:06:17 +02:00
export struct QUICServerCredentials {
std::string certPath;
std::string keyPath;
bool selfSigned = false;
2026-08-25 19:17:22 +02:00
QUICSettings settings;
2026-05-06 04:06:17 +02:00
};
// Client-side credential validation. By default we require a real cert.
2026-05-19 02:53:50 +02:00
// insecureNoServerValidation disables peer cert checks — only for dev,
// and silently ignored in the browser build (browsers enforce their own
// certificate policy). For browser dev against a self-signed listener,
// populate serverCertificateHash with the SHA-256 of the server's DER
// certificate; on the browser it is forwarded to WebTransport's
// serverCertificateHashes option. A zeroed array means "unused" — the
// browser will then require a publicly trusted cert.
2026-05-06 04:06:17 +02:00
export struct QUICClientCredentials {
bool insecureNoServerValidation = false;
2026-05-19 02:53:50 +02:00
std::array<std::uint8_t, 32> serverCertificateHash{};
2026-08-25 19:17:22 +02:00
QUICSettings settings;
2026-05-06 04:06:17 +02:00
};
export class ClientQUIC;
2026-05-07 00:06:44 +02:00
// A reliable, ordered stream within a QUIC connection. May be
// bidirectional or unidirectional; for unidi streams either canSend or
// canReceive will be false depending on which side initiated. Owned by
// ClientQUIC; do not destroy directly. Obtain via ClientQUIC::OpenStream
// (optionally with unidirectional=true) or via the on-stream callback
// for inbound streams initiated by the peer.
2026-05-06 04:06:17 +02:00
export class QUICStream {
public:
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
2026-05-06 04:06:17 +02:00
// Underlying msquic HQUIC handle. Treated as opaque by callers.
HQUIC handle = nullptr;
2026-05-19 02:53:50 +02:00
#endif
2026-05-06 04:06:17 +02:00
// The connection that owns this stream (non-owning).
ClientQUIC* connection = nullptr;
2026-05-07 00:06:44 +02:00
// Direction flags. Bidi streams have both true; outgoing unidi sets
// canReceive=false; incoming unidi (peer-initiated) sets canSend=false.
bool canSend = true;
bool canReceive = true;
QUICStream();
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
2026-05-06 04:06:17 +02:00
QUICStream(HQUIC handle, ClientQUIC* connection);
2026-05-19 02:53:50 +02:00
#else
// Browser-only constructor: wraps a JS-side WebTransport stream
// identified by its integer handle. Used by ClientQUIC::OpenStream
// and by the incoming-stream dispatcher in the JS bridge — not
// intended for direct use.
QUICStream(std::int32_t handle, ClientQUIC* connection,
bool canSend, bool canReceive);
#endif
2026-05-06 04:06:17 +02:00
~QUICStream();
QUICStream(const QUICStream&) = delete;
QUICStream(QUICStream&&) noexcept;
QUICStream& operator=(QUICStream&&) noexcept;
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
2026-05-06 04:06:17 +02:00
// Send a buffer. If finish=true, the send-side of the stream is closed
// after the buffer is delivered (peer will see graceful shutdown).
feat(quic): QUICStream::SendAsync holds no thread and reports from SEND_COMPLETE Native SendAsync was SendSync run as a pool task. That held a worker of Crafter.Thread's pool, which the host shares and which runs each worker's queue in order, for as long as msquic took to accept the buffer, and a peer that reads slowly stretches that to its own pace: a server streaming to a slow follower stalled the tasks queued behind it. The task also reached back through `this`, so moving or destroying the wrapper before it ran was a use-after-free, and two sends could reach msquic out of order from two workers. Now SendAsync copies the buffer into a SendRequest, hands it to msquic and returns. SEND_COMPLETE, which msquic delivers exactly once for every send it took and before the stream's SHUTDOWN_COMPLETE, frees the copy and calls onSent on the msquic worker: true when msquic took the whole buffer (in its send buffer or acknowledged, which is what SendSync's return has always meant), false when it was canceled because the stream or the connection ended first. Sends queue in msquic in call order and none waits for another. A send msquic never took (a closed stream, a moved-from wrapper, StreamSend refusing it) reports false from a pool task, never from inside SendAsync, so a caller may hold a lock its own callback takes. Only `impl` is read, not `handle`, so SendAsync is callable from any thread, a callback included; on a stopped stream msquic cancels the send. BytesInFlight() and SendsInFlight() count what SendAsync and SendSync handed msquic whose SEND_COMPLETE has not come, so a server can drop a follower that falls too far behind. Rewritten rather than added beside it: nothing in this repository calls the native SendAsync (ListenerHTTP, WebTransport and ClientHTTP use SendSync; the example's SendAsync is the browser build's), nor does any consumer found here (3DForts uses SendSync natively); its contract is unchanged, copy now and onSent once with whether it went; and the old one's pool thread, `this` capture and reordering were defects, not behaviour to keep. What changes is onSent's thread, msquic's worker rather than a pool task, so a callback that blocks, a *Sync call above all, now stalls its connection rather than a pool worker. The browser build is untouched. Breaking: native onSent runs on msquic's worker thread, not as a pool task, and must not block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:24:15 +01:00
// Blocks until msquic accepts the buffer; throws QUICClosedException
// when the stream or connection closed before it did. SendAsync is
// the same without the thread.
2026-05-06 04:06:17 +02:00
void SendSync(const void* buffer, std::uint32_t size, bool finish = false);
// Block until at least one byte is received; returns the received
// chunk. Throws QUICClosedException once the peer has closed the
// send-side and the buffer is drained.
std::vector<char> RecieveSync();
// Read until the peer closes the send-side, accumulating all chunks.
std::vector<char> RecieveUntilCloseSync();
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived - --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with them msquic: a consumer that only needs TCP builds no msquic. The flag isn't passed on to dependencies. - --local takes Crafter.Thread from the sibling checkout, as the other Crafter projects do. - ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and SocketTimeoutException. - TCP receives return the bytes that arrived: - RecieveSync(n) returned the whole buffer when fewer had come; - RecieveUntilCloseSync truncated what it had read; - a signal (EINTR) during a receive threw where it should wait on; - RecieveSync(size, buffer) returns the count and throws like the others; - a receive of no bytes returns at once: recv's 0 for it isn't a close. - Every TCP *Async receive takes an onError(std::exception_ptr): an exception escaping a ThreadPool task ended the program. An empty onError drops the error. - QUIC streams: - async receives deliver an empty chunk on close, as the browser build does; - RecieveUntilFullAsync gives what arrived on a stream that closes early; - RecieveUntilFullSync takes all it asked for or nothing, under one hold of the lock, so what did arrive stays for the next receive and no other receive can take bytes from among it; - SendAsync's callback hears whether the send went out (bool sent): native, browser, the JS bridge and the example. - msquic is pinned to v2.6.3 and built against the system OpenSSL (QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with config_diagnostics on) made it fail to initialise. The README says to use --no-quic where the distro's OpenSSL is older. - Tests now register on any Linux triple: they never ran under the Flatpak's x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP, ShouldRecieveTCP, ShouldReportClosedQUICStream. Breaking: SendAsync's callback takes a bool, the TCP *Async receives take an onError, and RecieveSync(size, buffer) returns the count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
// Read exactly bufferSize bytes. Throws if the peer closes early,
// leaving what did arrive for the next receive.
2026-05-06 04:06:17 +02:00
std::vector<char> RecieveUntilFullSync(std::uint32_t bufferSize);
2026-05-19 02:53:50 +02:00
#endif
2026-05-06 04:06:17 +02:00
2026-05-19 02:53:50 +02:00
// Send a buffer. If finish=true, the send-side is closed after the
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived - --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with them msquic: a consumer that only needs TCP builds no msquic. The flag isn't passed on to dependencies. - --local takes Crafter.Thread from the sibling checkout, as the other Crafter projects do. - ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and SocketTimeoutException. - TCP receives return the bytes that arrived: - RecieveSync(n) returned the whole buffer when fewer had come; - RecieveUntilCloseSync truncated what it had read; - a signal (EINTR) during a receive threw where it should wait on; - RecieveSync(size, buffer) returns the count and throws like the others; - a receive of no bytes returns at once: recv's 0 for it isn't a close. - Every TCP *Async receive takes an onError(std::exception_ptr): an exception escaping a ThreadPool task ended the program. An empty onError drops the error. - QUIC streams: - async receives deliver an empty chunk on close, as the browser build does; - RecieveUntilFullAsync gives what arrived on a stream that closes early; - RecieveUntilFullSync takes all it asked for or nothing, under one hold of the lock, so what did arrive stays for the next receive and no other receive can take bytes from among it; - SendAsync's callback hears whether the send went out (bool sent): native, browser, the JS bridge and the example. - msquic is pinned to v2.6.3 and built against the system OpenSSL (QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with config_diagnostics on) made it fail to initialise. The README says to use --no-quic where the distro's OpenSSL is older. - Tests now register on any Linux triple: they never ran under the Flatpak's x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP, ShouldRecieveTCP, ShouldReportClosedQUICStream. Breaking: SendAsync's callback takes a bool, the TCP *Async receives take an onError, and RecieveSync(size, buffer) returns the count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
// buffer is delivered. onSent fires with true once the transport has
// accepted the buffer (native) or the WritableStream writer has
// resolved (browser), and with false when the stream was closed or
// the send failed. Available on both native and browser builds.
feat(quic): QUICStream::SendAsync holds no thread and reports from SEND_COMPLETE Native SendAsync was SendSync run as a pool task. That held a worker of Crafter.Thread's pool, which the host shares and which runs each worker's queue in order, for as long as msquic took to accept the buffer, and a peer that reads slowly stretches that to its own pace: a server streaming to a slow follower stalled the tasks queued behind it. The task also reached back through `this`, so moving or destroying the wrapper before it ran was a use-after-free, and two sends could reach msquic out of order from two workers. Now SendAsync copies the buffer into a SendRequest, hands it to msquic and returns. SEND_COMPLETE, which msquic delivers exactly once for every send it took and before the stream's SHUTDOWN_COMPLETE, frees the copy and calls onSent on the msquic worker: true when msquic took the whole buffer (in its send buffer or acknowledged, which is what SendSync's return has always meant), false when it was canceled because the stream or the connection ended first. Sends queue in msquic in call order and none waits for another. A send msquic never took (a closed stream, a moved-from wrapper, StreamSend refusing it) reports false from a pool task, never from inside SendAsync, so a caller may hold a lock its own callback takes. Only `impl` is read, not `handle`, so SendAsync is callable from any thread, a callback included; on a stopped stream msquic cancels the send. BytesInFlight() and SendsInFlight() count what SendAsync and SendSync handed msquic whose SEND_COMPLETE has not come, so a server can drop a follower that falls too far behind. Rewritten rather than added beside it: nothing in this repository calls the native SendAsync (ListenerHTTP, WebTransport and ClientHTTP use SendSync; the example's SendAsync is the browser build's), nor does any consumer found here (3DForts uses SendSync natively); its contract is unchanged, copy now and onSent once with whether it went; and the old one's pool thread, `this` capture and reordering were defects, not behaviour to keep. What changes is onSent's thread, msquic's worker rather than a pool task, so a callback that blocks, a *Sync call above all, now stalls its connection rather than a pool worker. The browser build is untouched. Breaking: native onSent runs on msquic's worker thread, not as a pool task, and must not block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:24:15 +01:00
//
// Native: holds no thread. It copies the buffer, hands the copy to
// msquic and returns; msquic's SEND_COMPLETE for it frees the copy and
// fires onSent, exactly once: true when msquic took the whole buffer
// (it is in msquic's send buffer or acknowledged, what SendSync's
// return means; not a receipt from the peer's application), false
// when the stream or the connection closed first. Callable from any
// thread, inside this stream's own callbacks included, and as often
// as you like: the sends queue in msquic in call order, none waits for
// another, and BytesInFlight says how far behind they are.
// - onSent runs on the connection's msquic worker thread. A send
// that fails before msquic takes it (a closed stream, a moved-from
// wrapper) reports from a pool task instead, never from inside
// SendAsync, so the caller may hold a lock onSent takes.
// - In onSent: SendAsync and the other non-blocking calls are fine;
// nothing that blocks, no *Sync call (it waits for this very
// worker), no ~ClientQUIC; it must not throw.
// - It fires even after the wrapper is gone: destroying it lets the
// sends already made go out, then the FIN.
2026-05-19 02:53:50 +02:00
void SendAsync(const void* buffer, std::uint32_t size, bool finish,
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived - --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with them msquic: a consumer that only needs TCP builds no msquic. The flag isn't passed on to dependencies. - --local takes Crafter.Thread from the sibling checkout, as the other Crafter projects do. - ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and SocketTimeoutException. - TCP receives return the bytes that arrived: - RecieveSync(n) returned the whole buffer when fewer had come; - RecieveUntilCloseSync truncated what it had read; - a signal (EINTR) during a receive threw where it should wait on; - RecieveSync(size, buffer) returns the count and throws like the others; - a receive of no bytes returns at once: recv's 0 for it isn't a close. - Every TCP *Async receive takes an onError(std::exception_ptr): an exception escaping a ThreadPool task ended the program. An empty onError drops the error. - QUIC streams: - async receives deliver an empty chunk on close, as the browser build does; - RecieveUntilFullAsync gives what arrived on a stream that closes early; - RecieveUntilFullSync takes all it asked for or nothing, under one hold of the lock, so what did arrive stays for the next receive and no other receive can take bytes from among it; - SendAsync's callback hears whether the send went out (bool sent): native, browser, the JS bridge and the example. - msquic is pinned to v2.6.3 and built against the system OpenSSL (QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with config_diagnostics on) made it fail to initialise. The README says to use --no-quic where the distro's OpenSSL is older. - Tests now register on any Linux triple: they never ran under the Flatpak's x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP, ShouldRecieveTCP, ShouldReportClosedQUICStream. Breaking: SendAsync's callback takes a bool, the TCP *Async receives take an onError, and RecieveSync(size, buffer) returns the count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
std::function<void(bool sent)> onSent);
2026-05-19 02:53:50 +02:00
// Async receive variants. Dispatched on Crafter.Thread's ThreadPool
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived - --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with them msquic: a consumer that only needs TCP builds no msquic. The flag isn't passed on to dependencies. - --local takes Crafter.Thread from the sibling checkout, as the other Crafter projects do. - ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and SocketTimeoutException. - TCP receives return the bytes that arrived: - RecieveSync(n) returned the whole buffer when fewer had come; - RecieveUntilCloseSync truncated what it had read; - a signal (EINTR) during a receive threw where it should wait on; - RecieveSync(size, buffer) returns the count and throws like the others; - a receive of no bytes returns at once: recv's 0 for it isn't a close. - Every TCP *Async receive takes an onError(std::exception_ptr): an exception escaping a ThreadPool task ended the program. An empty onError drops the error. - QUIC streams: - async receives deliver an empty chunk on close, as the browser build does; - RecieveUntilFullAsync gives what arrived on a stream that closes early; - RecieveUntilFullSync takes all it asked for or nothing, under one hold of the lock, so what did arrive stays for the next receive and no other receive can take bytes from among it; - SendAsync's callback hears whether the send went out (bool sent): native, browser, the JS bridge and the example. - msquic is pinned to v2.6.3 and built against the system OpenSSL (QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with config_diagnostics on) made it fail to initialise. The README says to use --no-quic where the distro's OpenSSL is older. - Tests now register on any Linux triple: they never ran under the Flatpak's x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP, ShouldRecieveTCP, ShouldReportClosedQUICStream. Breaking: SendAsync's callback takes a bool, the TCP *Async receives take an onError, and RecieveSync(size, buffer) returns the count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
// (native) or driven by a per-stream JS reader loop (browser). A
// stream that has closed gives the callback an empty chunk, and
// RecieveUntilFullAsync on one that closes early gives what arrived:
// the browser build has no exceptions to report it with, and on the
// ThreadPool an exception would end the program.
2026-05-06 04:06:17 +02:00
void RecieveAsync(std::function<void(std::vector<char>)> recieveCallback);
void RecieveUntilCloseAsync(std::function<void(std::vector<char>)> recieveCallback);
void RecieveUntilFullAsync(std::uint32_t bufferSize, std::function<void(std::vector<char>)> recieveCallback);
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
// Advanced: re-inject already-consumed bytes at the front of the
// receive queue so the next Recieve* call sees them. Used by
// protocol demuxers (e.g. the WebTransport stream router in
// ListenerHTTP) that need to peek a prefix off the wire, then hand
// the stream to user code as if the prefix had never been read.
void PrependReceived(std::vector<char> bytes);
// Underlying QUIC stream id. Stable for the stream's lifetime.
// Browsers identify WT streams by the session's CONNECT stream id,
// so the server has to query and remember it at session creation
// time. Throws if the stream is not yet started.
std::uint64_t GetStreamId() const;
feat(quic): QUICStream::SendAsync holds no thread and reports from SEND_COMPLETE Native SendAsync was SendSync run as a pool task. That held a worker of Crafter.Thread's pool, which the host shares and which runs each worker's queue in order, for as long as msquic took to accept the buffer, and a peer that reads slowly stretches that to its own pace: a server streaming to a slow follower stalled the tasks queued behind it. The task also reached back through `this`, so moving or destroying the wrapper before it ran was a use-after-free, and two sends could reach msquic out of order from two workers. Now SendAsync copies the buffer into a SendRequest, hands it to msquic and returns. SEND_COMPLETE, which msquic delivers exactly once for every send it took and before the stream's SHUTDOWN_COMPLETE, frees the copy and calls onSent on the msquic worker: true when msquic took the whole buffer (in its send buffer or acknowledged, which is what SendSync's return has always meant), false when it was canceled because the stream or the connection ended first. Sends queue in msquic in call order and none waits for another. A send msquic never took (a closed stream, a moved-from wrapper, StreamSend refusing it) reports false from a pool task, never from inside SendAsync, so a caller may hold a lock its own callback takes. Only `impl` is read, not `handle`, so SendAsync is callable from any thread, a callback included; on a stopped stream msquic cancels the send. BytesInFlight() and SendsInFlight() count what SendAsync and SendSync handed msquic whose SEND_COMPLETE has not come, so a server can drop a follower that falls too far behind. Rewritten rather than added beside it: nothing in this repository calls the native SendAsync (ListenerHTTP, WebTransport and ClientHTTP use SendSync; the example's SendAsync is the browser build's), nor does any consumer found here (3DForts uses SendSync natively); its contract is unchanged, copy now and onSent once with whether it went; and the old one's pool thread, `this` capture and reordering were defects, not behaviour to keep. What changes is onSent's thread, msquic's worker rather than a pool task, so a callback that blocks, a *Sync call above all, now stalls its connection rather than a pool worker. The browser build is untouched. Breaking: native onSent runs on msquic's worker thread, not as a pool task, and must not block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:24:15 +01:00
// Bytes, and sends, handed to SendAsync or SendSync whose completion
// has not come yet: how far behind a slow peer is. A server streaming
// to a follower can drop one whose backlog passes a cap. Any thread.
std::uint64_t BytesInFlight() const;
std::size_t SendsInFlight() const;
2026-05-19 02:53:50 +02:00
#endif
fix(quic): close stream handles whose async StreamStart lost the teardown race A client that sends on a stream while its connection is being torn down — the shape any long-lived client with periodic acks or keepalives has — would hang at exit inside MsQuicRegistrationClose, and occasionally bugcheck on an msquic worker thread instead. Root cause: StreamStart is asynchronous. It returns QUIC_STATUS_PENDING and queues the start onto the connection, so it can still fail later, and it fails with QUIC_STATUS_INVALID_STATE whenever the connection is shut down before the queued start runs. Passed QUIC_STREAM_START_FLAG_NONE, msquic leaves such a stream neither started nor shut down, so SHUTDOWN_COMPLETE never arrives. The stream close is driven entirely off that event, so the handle stays open: ~ClientQUIC's bounded drain expires, the connection is closed with a stream msquic still considers open, and the registration's rundown at exit() waits forever for the handle. Instrumenting the handle lifecycle over the reproducer shows one leaked locally-opened stream per failing run, correlating 1:1 with a START_COMPLETE carrying 0x1 (INVALID_STATE). Fixes: - OpenStream passes QUIC_STREAM_START_FLAG_SHUTDOWN_ON_FAIL, so a start that loses the race is followed by SHUTDOWN_COMPLETE like any other stream and its handle is closed. - QUICStream::Stop also aborts the receive direction. GRACEFUL closes only the send side and msquic rejects it combined with any other flag, so a bidirectional stream whose peer keeps its send side open never reached SHUTDOWN_COMPLETE either. - The msquic stream handle is now refcounted, held by the callback plus every app thread inside an msquic call on it, and closed by whoever lets go last. StreamSend/StreamShutdown/GetParam could previously run against a handle a worker thread had just closed — a use-after-free msquic reports as a bugcheck. A refcount rather than an exclusion lock because GetParam blocks on the connection's worker, which must never be the thread waiting. - ~ClientQUIC drops undispatched peer streams before closing the connection rather than with the rest of Impl afterwards, so the drain covers them. The reproducer (tests/ShouldSurviveConnectionChurn) grows a client-opened bidirectional control stream with a thread sending across the teardown, and no longer ends in std::_Exit — returning from main runs the static MsQuicRuntime destructor, and its RegistrationClose is where a leaked handle shows up. Before: 5 of 15 runs clean, 10 hung. After: 90 of 90 clean, with every opened handle observed closed and the drain never expiring. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 23:18:01 +00:00
// Shut the stream down in both directions: our send side gracefully
// (the peer sees a FIN), the receive side abortively. Aborting the
// receive side matters — a bidirectional stream whose peer keeps its
// send side open otherwise never completes its shutdown, and so never
// releases its transport handle. Returns immediately; completion is
// asynchronous. Called by the destructor, so an explicit call is only
// needed to shut a stream down ahead of its wrapper going away.
2026-05-06 04:06:17 +02:00
void Stop();
private:
struct Impl;
2026-07-22 17:55:45 +02:00
// shared_ptr (not unique_ptr): the msquic stream callback holds its own
// ref so it can never run on a freed Impl if the wrapper is destroyed
// before SHUTDOWN_COMPLETE fires. See Crafter.Network-ClientQUIC.cpp.
std::shared_ptr<Impl> impl;
2026-05-06 04:06:17 +02:00
friend class ClientQUIC;
};
// A QUIC connection. On the client side, constructing one initiates the
2026-05-19 02:53:50 +02:00
// handshake and (on native) blocks until it succeeds, or throws on
// failure. On the server side, ListenerQUIC instantiates these for
// accepted peers.
2026-05-06 04:06:17 +02:00
//
// A connection multiplexes:
// - Reliable, ordered streams (open via OpenStream() / observe inbound
// via OnStream()).
// - Unreliable, unordered datagrams (SendDatagram() / OnDatagram()).
//
fix(quic): stop ~ClientQUIC racing the callback to close the connection MsQuicConnectionClose was reachable twice for one handle: ~ClientQUIC called it unconditionally, and the SHUTDOWN_COMPLETE callback also called it whenever AppCloseInProgress was clear -- which it is for the whole window between the destructor's ConnectionShutdown and its ConnectionClose. The second call trips CXPLAT_TEL_ASSERT(!Connection->State.HandleClosed) and aborts the process. Clearing impl->connection from the callback did not help: the destructor had already loaded it. `connection` is now a claim token, taken under the mutex, and only whoever wins the claim closes. The callback keeps closing peer-dropped connections so they do not leak while their ClientQUIC lives on. Separately, the destructor now waits (bounded, 5s) for the connection's streams to reach StreamClose before closing it. QUICStream::Stop only initiates a shutdown whose async completion does the close, so a connection could otherwise be closed with streams msquic still considers open. The count lives in a shared StreamRegistry rather than behind the ClientQUIC*, so a stream finalising after the wait gave up decrements a live object. A failed handshake also closes its connection and configuration now. Throwing from the constructor body means ~ClientQUIC never runs, so both handles leaked -- and the connection callback holds the impl that was about to be destroyed with the half-built object. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 17:06:51 +00:00
// Lifetime: ~ClientQUIC shuts the connection down and closes it, waiting
// (bounded — 5s) for every stream on it to finish closing first, since
fix(quic): let ~ClientQUIC's CONNECTION_CLOSE reach the peer ~ClientQUIC asked msquic to shut the connection down, waited for its streams to drain, and closed the handle. With no stream open the drain returns at once, so ConnectionClose followed ConnectionShutdown within microseconds. msquic sends the CONNECTION_CLOSE frame from a FLUSH_SEND operation it queues while it processes the shutdown; the close the destructor had queued meanwhile reached the worker first, forced the shutdown to complete silently (QuicConnTryClose: "silent close forced after we already started the close process"), and the flush, which is ignored once ShutdownComplete is set, sent nothing. The peer learned of the close only at its idle timeout, two minutes by default: a server whose clients each live for one request, such as a debug server driven by a CLI that connects once per command, kept every one of them that long. The destructor now also waits, after the drain, for the connection's SHUTDOWN_COMPLETE before closing the handle. That comes about one round trip later once the peer answers the close, or after msquic's closing period (three PTOs) when it does not; the wait is bounded at 1 s, since the frame went out at the start either way and a peer that is gone must not hold the destructor for the whole closing period. The handler sets a flag for it before it looks at the handle, which the destructor holds, so the close stays the destructor's. Measured with ShouldTellPeerOfCloseQUIC: before, the server was not told of the first of 20 client closes within 2 s (and, counted another way, 8 of 100 peer closes reached the server within 10 s in one run, 0 of 100 in the next); after, all 20 within a millisecond. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:06:39 +01:00
// QUICStream::Stop only initiates a shutdown that completes asynchronously,
// and then (bounded — 1s, about a round trip when the peer answers) for
// the shutdown to complete, so the peer is sent the close rather than
// left to time the connection out.
fix(quic): stop ~ClientQUIC racing the callback to close the connection MsQuicConnectionClose was reachable twice for one handle: ~ClientQUIC called it unconditionally, and the SHUTDOWN_COMPLETE callback also called it whenever AppCloseInProgress was clear -- which it is for the whole window between the destructor's ConnectionShutdown and its ConnectionClose. The second call trips CXPLAT_TEL_ASSERT(!Connection->State.HandleClosed) and aborts the process. Clearing impl->connection from the callback did not help: the destructor had already loaded it. `connection` is now a claim token, taken under the mutex, and only whoever wins the claim closes. The callback keeps closing peer-dropped connections so they do not leak while their ClientQUIC lives on. Separately, the destructor now waits (bounded, 5s) for the connection's streams to reach StreamClose before closing it. QUICStream::Stop only initiates a shutdown whose async completion does the close, so a connection could otherwise be closed with streams msquic still considers open. The count lives in a shared StreamRegistry rather than behind the ClientQUIC*, so a stream finalising after the wait gave up decrements a live object. A failed handshake also closes its connection and configuration now. Throwing from the constructor body means ~ClientQUIC never runs, so both handles leaked -- and the connection callback holds the impl that was about to be destroyed with the half-built object. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 17:06:51 +00:00
// Streams outliving the connection are therefore safe to hold, though they
// are dead once it goes; prefer destroying or moving them out first.
2026-05-19 02:53:50 +02:00
//
// Browser build: the only QUIC-shaped API the browser exposes is
// WebTransport, which is HTTP/3-based and reached at a fixed URL. Here:
// - The constructor returns immediately; the connection is opened in
// the background. Operations issued before the connection is ready
// are queued JS-side until WebTransport's "ready" promise resolves
// (or fail with QUICClosedException if the connection rejects).
// - `alpn` is mapped to the URL path: new WebTransport(
// `https://${host}:${port}/${alpn}`). The QUIC-layer ALPN itself
// is fixed to "h3" by the browser and cannot be customised.
// - The server side must accept WebTransport sessions (HTTP/3 extended
// CONNECT) on the path equal to `alpn`. Plain QUIC with a custom
// ALPN — what ListenerQUIC offers today — is not reachable from a
// browser.
// - Synchronous send/receive methods are not compiled. Use the *Async
// variants instead.
2026-05-06 04:06:17 +02:00
export class ClientQUIC {
public:
// ALPN identifier exchanged in the handshake. Both peers must agree.
// For 3DForts use e.g. "f3d/1" or similar — a short stable token.
2026-05-19 02:53:50 +02:00
// On the browser build, this is the WebTransport URL path instead
// of an ALPN token; see the class comment above.
2026-05-06 04:06:17 +02:00
std::string alpn;
// Client constructor: connects to host:port using QUIC. ALPN must
// match the listener. Throws QUICException on connect failure.
// Blocks for at most the handshake idle timeout (10s), so a host that
// never answers fails in seconds rather than at the idle timeout.
2026-05-06 04:06:17 +02:00
ClientQUIC(const char* host, std::uint16_t port, std::string alpn,
QUICClientCredentials creds = {});
ClientQUIC(std::string host, std::uint16_t port, std::string alpn,
QUICClientCredentials creds = {});
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
2026-05-06 04:06:17 +02:00
// Server-side constructor used by ListenerQUIC for accepted peers.
// Takes ownership of an already-accepted msquic connection handle
// and the server configuration handle. Not intended for direct use.
ClientQUIC(HQUIC connectionHandle, HQUIC serverConfiguration, std::string alpn);
2026-05-19 02:53:50 +02:00
#endif
2026-05-06 04:06:17 +02:00
~ClientQUIC();
ClientQUIC(const ClientQUIC&) = delete;
ClientQUIC(ClientQUIC&&) noexcept;
2026-05-07 00:06:44 +02:00
// Open a new stream initiated by this side. Defaults to bidirectional;
// pass unidirectional=true to open a one-way send stream (used for
// HTTP/3's control + QPACK encoder/decoder streams).
2026-05-06 04:06:17 +02:00
// Blocks until the stream is started; throws on failure.
2026-05-07 00:06:44 +02:00
QUICStream OpenStream(bool unidirectional = false);
2026-05-06 04:06:17 +02:00
// Send a datagram. Best-effort: may be silently dropped under loss
// or congestion. Size must fit within the path MTU (msquic surfaces
// the maximum via QUIC_PARAM_CONN_DATAGRAM_SEND_ENABLED — typically
// ~1200 bytes safely on the open internet).
void SendDatagram(const void* buffer, std::uint32_t size);
// Register a handler for streams the peer initiates toward us.
// Called on the msquic worker; offload heavy work to ThreadPool.
void OnStream(std::function<void(QUICStream)> callback);
// Register a handler for datagrams from the peer. Called on the
// msquic worker; copy/queue and return promptly.
void OnDatagram(std::function<void(std::vector<char>)> callback);
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
2026-05-06 04:06:17 +02:00
// Block the caller until the next datagram arrives; returns it.
// Throws QUICClosedException if the connection closes first.
std::vector<char> RecieveDatagramSync();
feat(quic): ClientQUIC::OnClosed tells the owner the connection is over A ListenerQUIC hands each accepted connection to its callback as a heap-allocated ClientQUIC the callback owns, and nothing said when to delete it: the SHUTDOWN_COMPLETE handler closed the msquic handle of a connection the peer or the idle timeout ended, but the object stayed, so a server either leaked its peers or deleted them blind, and one still alive at exit holds msquic's registration open until its idle timeout. OnClosed(callback) is told once when the connection has shut down for any reason but this object's destruction: the peer closed it, the idle timeout or a transport error ended it, or Stop() was called. It runs on the msquic worker, in the SHUTDOWN_COMPLETE handler, after that handler has claimed and closed the handle, so deleting the ClientQUIC from inside it is safe: ~ClientQUIC then finds no handle to claim and waits for nothing. Delivering it there, rather than as a pool task, keeps it independent of a pool the host shares and possibly saturates; the cost is the usual worker rule, which the interface spells out (no blocking, no *Sync call, no deleting another ClientQUIC from inside it). A callback set after the close, which happens when the peer goes before a ListenerQUIC's connectCallback runs, fires as a pool task instead, never from inside OnClosed, where the caller is still using the object. Its state is a shared CloseNotice rather than part of the Impl, since the handler fires it after giving up the handle, when ~ClientQUIC on another thread no longer waits for the worker and may free the Impl. The rules are std::stop_callback's: ~ClientQUIC marks the owner gone first, after which the callback never starts; one already running on another thread is waited for, so the owner may free what it uses as soon as the destructor returns; one on the destructor's own thread, a callback deleting its connection, is not. The handler takes the handle and the notice under one lock, so the destructor cannot slip between. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:18:50 +01:00
// Told once when the connection has shut down for any reason other
// than this object being destroyed: the peer closed it, the idle
// timeout or a transport error ended it, or Stop() was called. This
// is how the owner of a connection a ListenerQUIC accepted learns
// that it is time to delete it.
//
// - It runs on the connection's msquic worker thread, once the
// connection's handle is closed; if the connection had already
// closed when OnClosed was called, it runs as a pool task
// instead, never inside OnClosed itself.
// - Deleting this ClientQUIC from inside it is safe, and the usual
// thing to do. Do nothing else that blocks: deleting a
// *different* ClientQUIC, or any *Sync call, waits on msquic
// workers, this one perhaps among them. It must not throw; an
// exception leaving it ends the program.
// - Once ~ClientQUIC has begun it never starts, and a run already
// under way on another thread finishes before ~ClientQUIC goes
// on, as with std::stop_callback (one on this thread, deleting
// its own connection, is not waited for). So do not destroy the
// ClientQUIC while holding a lock the callback takes.
// - Calling OnClosed again replaces the callback, and an empty one
// unregisters it; once it has fired, later calls do nothing.
void OnClosed(std::function<void()> callback);
2026-05-19 02:53:50 +02:00
#endif
2026-05-06 04:06:17 +02:00
// Cleanly shut down the connection.
void Stop();
2026-05-19 02:53:50 +02:00
#ifndef CRAFTER_NETWORK_BROWSER
2026-05-06 04:06:17 +02:00
// Underlying handle for advanced use (parameter queries, etc.).
HQUIC GetHandle() const;
2026-05-19 02:53:50 +02:00
#endif
2026-05-06 04:06:17 +02:00
private:
struct Impl;
std::unique_ptr<Impl> impl;
friend class ListenerQUIC;
friend class QUICStream;
};
}