2026-07-22 17:55:45 +02:00
|
|
|
//SPDX-License-Identifier: LGPL-3.0-only
|
|
|
|
|
//SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
|
2026-05-06 04:06:17 +02:00
|
|
|
|
|
|
|
|
module;
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
2026-05-06 04:06:17 +02:00
|
|
|
#include <msquic.h>
|
2026-05-19 02:53:50 +02:00
|
|
|
#endif
|
2026-05-06 04:06:17 +02:00
|
|
|
export module Crafter.Network:ClientQUIC;
|
|
|
|
|
import std;
|
|
|
|
|
|
|
|
|
|
namespace Crafter {
|
|
|
|
|
export class ListenerQUIC;
|
|
|
|
|
|
|
|
|
|
export class QUICException : public std::runtime_error {
|
|
|
|
|
public:
|
|
|
|
|
using std::runtime_error::runtime_error;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
export class QUICClosedException : public std::exception {
|
|
|
|
|
public:
|
|
|
|
|
const char* what() const noexcept override { return "QUIC connection closed"; }
|
|
|
|
|
};
|
|
|
|
|
|
2026-08-25 19:17:22 +02:00
|
|
|
// Transport knobs applied to the msquic Configuration. The defaults
|
|
|
|
|
// reproduce the values these were hardcoded to before they were made
|
|
|
|
|
// configurable, so leaving this alone keeps the previous behaviour.
|
|
|
|
|
//
|
|
|
|
|
// peerUnidiStreamCount / peerBidiStreamCount cap how many streams the
|
|
|
|
|
// *peer* may open toward us. msquic defaults both to 0; a server's
|
|
|
|
|
// HTTP/3 control + QPACK streams alone need 3, and a protocol that
|
|
|
|
|
// fans data out over many concurrent streams needs correspondingly
|
|
|
|
|
// more. Raising these costs per-stream flow-control state, so raise
|
|
|
|
|
// them to what a protocol actually needs rather than to a large
|
|
|
|
|
// round number.
|
|
|
|
|
//
|
|
|
|
|
// keepAliveIntervalMs is client-only (the listener does not set it).
|
|
|
|
|
// Keep it below idleTimeoutMs or the peer's idle timer will expire.
|
|
|
|
|
// handshakeIdleTimeoutMs bounds how long a connect attempt to an
|
|
|
|
|
// unreachable peer takes to fail. This one is set explicitly where it
|
|
|
|
|
// previously was not: ClientQUIC's constructor waits on a condition
|
|
|
|
|
// variable with no timeout of its own, so without a bound here a client
|
|
|
|
|
// dialling a dead port blocks for minutes rather than failing and
|
|
|
|
|
// retrying. 10s matches msquic's own documented default.
|
|
|
|
|
export struct QUICSettings {
|
|
|
|
|
std::uint32_t idleTimeoutMs = 120'000;
|
|
|
|
|
std::uint32_t handshakeIdleTimeoutMs = 10'000;
|
|
|
|
|
std::uint32_t keepAliveIntervalMs = 10'000;
|
|
|
|
|
std::uint16_t peerUnidiStreamCount = 16;
|
|
|
|
|
std::uint16_t peerBidiStreamCount = 16;
|
|
|
|
|
bool datagramReceiveEnabled = true;
|
|
|
|
|
};
|
|
|
|
|
|
2026-05-06 04:06:17 +02:00
|
|
|
// Server certificate sources. Pick one of (filePaths) or (selfSigned).
|
fix(listener): make the selfSigned QUIC certificate per user, in-process, without a race
QUICServerCredentials{selfSigned} shelled out to the openssl CLI and kept
the pair in the fixed /tmp/crafter-network-quic-cert/, guarded by an
in-process mutex only. Three things went wrong with that:
- Processes started at once (crafter-build runs tests in parallel, and
inside the Flatpak they share one /tmp) each found no pair and each
ran `openssl req` over the same key.pem and cert.pem, which it writes
in place one after the other. A process could load one run's key with
another's certificate and fail in ConfigurationLoadCredential.
- /tmp is shared by every user of the machine: whoever made the
directory first could keep everyone else from writing there, or plant
a pair in it.
- Without the openssl CLI there was no certificate at all, and
ComputeCertificateHashSHA256 piped the file through it as well.
The pair now lives in $XDG_CACHE_HOME/crafter-network/quic-cert/ (else
~/.cache/..., else $XDG_RUNTIME_DIR/...). Missing directories are made
0700, and the last is opened O_NOFOLLOW, refused unless the user owns
it, and made 0700 if it was not. A POSIX record lock on a file in it,
given up on after 10 s, covers both the check of the pair on disk and
its replacement, so one process makes the pair and the rest read it;
each file is written under a temporary name, fsynced and renamed into
place; and the listener hands msquic the pair from memory as PKCS#12
(QUIC_CREDENTIAL_TYPE_CERTIFICATE_PKCS12) instead of the two paths, so
a replacement cannot land between its reads of the key and the
certificate. A pair that does not parse, whose key does not match, or
that has less than a day left is replaced, which also repairs one a
crash tore between the two renames. A record lock rather than flock: an
flock is shared with a child forked while another thread holds it, and
the child holds it for its whole life. Threads of one process are kept
apart by the existing mutex, and the process keeps the pair it loaded,
so all its listeners serve one certificate until it nears its end.
The certificate is made in-process by :TLS in the shape the CLI
produced (ECDSA P-256, ecdsa-with-SHA256, 10 days, BasicConstraints
CA:FALSE, KeyUsage digitalSignature, EKU serverAuth, SANs localhost,
127.0.0.1 and ::1), with a random serial where the CLI also drew one;
ComputeCertificateHashSHA256 is X509_digest over the file's
certificate. The OpenSSL helpers are module-internal declarations in
:TLS, so libssl stays behind that one partition, and the TLS
partition's own development certificate is unchanged.
As before, the pair is reused across restarts while valid, so a
browser's pinned hash survives one; it is now replaced a day before it
expires rather than after, so a server is not started on a certificate
about to lapse. In the Flatpak, where /tmp went away with the last
instance, it now survives between runs too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 01:56:44 +01:00
|
|
|
// selfSigned serves a development certificate made in-process and kept
|
|
|
|
|
// per user under $XDG_CACHE_HOME/crafter-network/quic-cert/, reused
|
|
|
|
|
// while valid (see GetSelfSignedCertificatePath) — fine for dev/LAN.
|
2026-05-06 04:06:17 +02:00
|
|
|
export struct QUICServerCredentials {
|
|
|
|
|
std::string certPath;
|
|
|
|
|
std::string keyPath;
|
|
|
|
|
bool selfSigned = false;
|
2026-08-25 19:17:22 +02:00
|
|
|
QUICSettings settings;
|
2026-05-06 04:06:17 +02:00
|
|
|
};
|
|
|
|
|
|
|
|
|
|
// Client-side credential validation. By default we require a real cert.
|
2026-05-19 02:53:50 +02:00
|
|
|
// insecureNoServerValidation disables peer cert checks — only for dev,
|
|
|
|
|
// and silently ignored in the browser build (browsers enforce their own
|
|
|
|
|
// certificate policy). For browser dev against a self-signed listener,
|
|
|
|
|
// populate serverCertificateHash with the SHA-256 of the server's DER
|
|
|
|
|
// certificate; on the browser it is forwarded to WebTransport's
|
|
|
|
|
// serverCertificateHashes option. A zeroed array means "unused" — the
|
|
|
|
|
// browser will then require a publicly trusted cert.
|
2026-05-06 04:06:17 +02:00
|
|
|
export struct QUICClientCredentials {
|
|
|
|
|
bool insecureNoServerValidation = false;
|
2026-05-19 02:53:50 +02:00
|
|
|
std::array<std::uint8_t, 32> serverCertificateHash{};
|
2026-08-25 19:17:22 +02:00
|
|
|
QUICSettings settings;
|
2026-05-06 04:06:17 +02:00
|
|
|
};
|
|
|
|
|
|
|
|
|
|
export class ClientQUIC;
|
|
|
|
|
|
2026-05-07 00:06:44 +02:00
|
|
|
// A reliable, ordered stream within a QUIC connection. May be
|
|
|
|
|
// bidirectional or unidirectional; for unidi streams either canSend or
|
|
|
|
|
// canReceive will be false depending on which side initiated. Owned by
|
|
|
|
|
// ClientQUIC; do not destroy directly. Obtain via ClientQUIC::OpenStream
|
|
|
|
|
// (optionally with unidirectional=true) or via the on-stream callback
|
|
|
|
|
// for inbound streams initiated by the peer.
|
2026-05-06 04:06:17 +02:00
|
|
|
export class QUICStream {
|
|
|
|
|
public:
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
2026-05-06 04:06:17 +02:00
|
|
|
// Underlying msquic HQUIC handle. Treated as opaque by callers.
|
|
|
|
|
HQUIC handle = nullptr;
|
2026-05-19 02:53:50 +02:00
|
|
|
#endif
|
2026-05-06 04:06:17 +02:00
|
|
|
|
|
|
|
|
// The connection that owns this stream (non-owning).
|
|
|
|
|
ClientQUIC* connection = nullptr;
|
|
|
|
|
|
2026-05-07 00:06:44 +02:00
|
|
|
// Direction flags. Bidi streams have both true; outgoing unidi sets
|
|
|
|
|
// canReceive=false; incoming unidi (peer-initiated) sets canSend=false.
|
|
|
|
|
bool canSend = true;
|
|
|
|
|
bool canReceive = true;
|
|
|
|
|
|
|
|
|
|
QUICStream();
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
2026-05-06 04:06:17 +02:00
|
|
|
QUICStream(HQUIC handle, ClientQUIC* connection);
|
2026-05-19 02:53:50 +02:00
|
|
|
#else
|
|
|
|
|
// Browser-only constructor: wraps a JS-side WebTransport stream
|
|
|
|
|
// identified by its integer handle. Used by ClientQUIC::OpenStream
|
|
|
|
|
// and by the incoming-stream dispatcher in the JS bridge — not
|
|
|
|
|
// intended for direct use.
|
|
|
|
|
QUICStream(std::int32_t handle, ClientQUIC* connection,
|
|
|
|
|
bool canSend, bool canReceive);
|
|
|
|
|
#endif
|
2026-05-06 04:06:17 +02:00
|
|
|
~QUICStream();
|
|
|
|
|
QUICStream(const QUICStream&) = delete;
|
|
|
|
|
QUICStream(QUICStream&&) noexcept;
|
|
|
|
|
QUICStream& operator=(QUICStream&&) noexcept;
|
|
|
|
|
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
2026-05-06 04:06:17 +02:00
|
|
|
// Send a buffer. If finish=true, the send-side of the stream is closed
|
|
|
|
|
// after the buffer is delivered (peer will see graceful shutdown).
|
feat(quic): QUICStream::SendAsync holds no thread and reports from SEND_COMPLETE
Native SendAsync was SendSync run as a pool task. That held a worker of
Crafter.Thread's pool, which the host shares and which runs each
worker's queue in order, for as long as msquic took to accept the
buffer, and a peer that reads slowly stretches that to its own pace: a
server streaming to a slow follower stalled the tasks queued behind it.
The task also reached back through `this`, so moving or destroying the
wrapper before it ran was a use-after-free, and two sends could reach
msquic out of order from two workers.
Now SendAsync copies the buffer into a SendRequest, hands it to msquic
and returns. SEND_COMPLETE, which msquic delivers exactly once for every
send it took and before the stream's SHUTDOWN_COMPLETE, frees the copy
and calls onSent on the msquic worker: true when msquic took the whole
buffer (in its send buffer or acknowledged, which is what SendSync's
return has always meant), false when it was canceled because the stream
or the connection ended first. Sends queue in msquic in call order and
none waits for another. A send msquic never took (a closed stream, a
moved-from wrapper, StreamSend refusing it) reports false from a pool
task, never from inside SendAsync, so a caller may hold a lock its own
callback takes. Only `impl` is read, not `handle`, so SendAsync is
callable from any thread, a callback included; on a stopped stream
msquic cancels the send.
BytesInFlight() and SendsInFlight() count what SendAsync and SendSync
handed msquic whose SEND_COMPLETE has not come, so a server can drop a
follower that falls too far behind.
Rewritten rather than added beside it: nothing in this repository calls
the native SendAsync (ListenerHTTP, WebTransport and ClientHTTP use
SendSync; the example's SendAsync is the browser build's), nor does any
consumer found here (3DForts uses SendSync natively); its contract is
unchanged, copy now and onSent once with whether it went; and the old
one's pool thread, `this` capture and reordering were defects, not
behaviour to keep. What changes is onSent's thread, msquic's worker
rather than a pool task, so a callback that blocks, a *Sync call above
all, now stalls its connection rather than a pool worker. The browser
build is untouched.
Breaking: native onSent runs on msquic's worker thread, not as a pool
task, and must not block.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:24:15 +01:00
|
|
|
// Blocks until msquic accepts the buffer; throws QUICClosedException
|
|
|
|
|
// when the stream or connection closed before it did. SendAsync is
|
|
|
|
|
// the same without the thread.
|
2026-05-06 04:06:17 +02:00
|
|
|
void SendSync(const void* buffer, std::uint32_t size, bool finish = false);
|
|
|
|
|
|
|
|
|
|
// Block until at least one byte is received; returns the received
|
|
|
|
|
// chunk. Throws QUICClosedException once the peer has closed the
|
|
|
|
|
// send-side and the buffer is drained.
|
|
|
|
|
std::vector<char> RecieveSync();
|
|
|
|
|
|
|
|
|
|
// Read until the peer closes the send-side, accumulating all chunks.
|
|
|
|
|
std::vector<char> RecieveUntilCloseSync();
|
|
|
|
|
|
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived
- --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with
them msquic: a consumer that only needs TCP builds no msquic. The flag
isn't passed on to dependencies.
- --local takes Crafter.Thread from the sibling checkout, as the other
Crafter projects do.
- ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and
SocketTimeoutException.
- TCP receives return the bytes that arrived:
- RecieveSync(n) returned the whole buffer when fewer had come;
- RecieveUntilCloseSync truncated what it had read;
- a signal (EINTR) during a receive threw where it should wait on;
- RecieveSync(size, buffer) returns the count and throws like the others;
- a receive of no bytes returns at once: recv's 0 for it isn't a close.
- Every TCP *Async receive takes an onError(std::exception_ptr): an
exception escaping a ThreadPool task ended the program. An empty onError
drops the error.
- QUIC streams:
- async receives deliver an empty chunk on close, as the browser build does;
- RecieveUntilFullAsync gives what arrived on a stream that closes early;
- RecieveUntilFullSync takes all it asked for or nothing, under one hold
of the lock, so what did arrive stays for the next receive and no
other receive can take bytes from among it;
- SendAsync's callback hears whether the send went out (bool sent): native,
browser, the JS bridge and the example.
- msquic is pinned to v2.6.3 and built against the system OpenSSL
(QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's
openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with
config_diagnostics on) made it fail to initialise. The README says to use
--no-quic where the distro's OpenSSL is older.
- Tests now register on any Linux triple: they never ran under the Flatpak's
x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP,
ShouldRecieveTCP, ShouldReportClosedQUICStream.
Breaking: SendAsync's callback takes a bool, the TCP *Async receives take
an onError, and RecieveSync(size, buffer) returns the count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
|
|
|
// Read exactly bufferSize bytes. Throws if the peer closes early,
|
|
|
|
|
// leaving what did arrive for the next receive.
|
2026-05-06 04:06:17 +02:00
|
|
|
std::vector<char> RecieveUntilFullSync(std::uint32_t bufferSize);
|
2026-05-19 02:53:50 +02:00
|
|
|
#endif
|
2026-05-06 04:06:17 +02:00
|
|
|
|
2026-05-19 02:53:50 +02:00
|
|
|
// Send a buffer. If finish=true, the send-side is closed after the
|
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived
- --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with
them msquic: a consumer that only needs TCP builds no msquic. The flag
isn't passed on to dependencies.
- --local takes Crafter.Thread from the sibling checkout, as the other
Crafter projects do.
- ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and
SocketTimeoutException.
- TCP receives return the bytes that arrived:
- RecieveSync(n) returned the whole buffer when fewer had come;
- RecieveUntilCloseSync truncated what it had read;
- a signal (EINTR) during a receive threw where it should wait on;
- RecieveSync(size, buffer) returns the count and throws like the others;
- a receive of no bytes returns at once: recv's 0 for it isn't a close.
- Every TCP *Async receive takes an onError(std::exception_ptr): an
exception escaping a ThreadPool task ended the program. An empty onError
drops the error.
- QUIC streams:
- async receives deliver an empty chunk on close, as the browser build does;
- RecieveUntilFullAsync gives what arrived on a stream that closes early;
- RecieveUntilFullSync takes all it asked for or nothing, under one hold
of the lock, so what did arrive stays for the next receive and no
other receive can take bytes from among it;
- SendAsync's callback hears whether the send went out (bool sent): native,
browser, the JS bridge and the example.
- msquic is pinned to v2.6.3 and built against the system OpenSSL
(QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's
openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with
config_diagnostics on) made it fail to initialise. The README says to use
--no-quic where the distro's OpenSSL is older.
- Tests now register on any Linux triple: they never ran under the Flatpak's
x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP,
ShouldRecieveTCP, ShouldReportClosedQUICStream.
Breaking: SendAsync's callback takes a bool, the TCP *Async receives take
an onError, and RecieveSync(size, buffer) returns the count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
|
|
|
// buffer is delivered. onSent fires with true once the transport has
|
|
|
|
|
// accepted the buffer (native) or the WritableStream writer has
|
|
|
|
|
// resolved (browser), and with false when the stream was closed or
|
|
|
|
|
// the send failed. Available on both native and browser builds.
|
feat(quic): QUICStream::SendAsync holds no thread and reports from SEND_COMPLETE
Native SendAsync was SendSync run as a pool task. That held a worker of
Crafter.Thread's pool, which the host shares and which runs each
worker's queue in order, for as long as msquic took to accept the
buffer, and a peer that reads slowly stretches that to its own pace: a
server streaming to a slow follower stalled the tasks queued behind it.
The task also reached back through `this`, so moving or destroying the
wrapper before it ran was a use-after-free, and two sends could reach
msquic out of order from two workers.
Now SendAsync copies the buffer into a SendRequest, hands it to msquic
and returns. SEND_COMPLETE, which msquic delivers exactly once for every
send it took and before the stream's SHUTDOWN_COMPLETE, frees the copy
and calls onSent on the msquic worker: true when msquic took the whole
buffer (in its send buffer or acknowledged, which is what SendSync's
return has always meant), false when it was canceled because the stream
or the connection ended first. Sends queue in msquic in call order and
none waits for another. A send msquic never took (a closed stream, a
moved-from wrapper, StreamSend refusing it) reports false from a pool
task, never from inside SendAsync, so a caller may hold a lock its own
callback takes. Only `impl` is read, not `handle`, so SendAsync is
callable from any thread, a callback included; on a stopped stream
msquic cancels the send.
BytesInFlight() and SendsInFlight() count what SendAsync and SendSync
handed msquic whose SEND_COMPLETE has not come, so a server can drop a
follower that falls too far behind.
Rewritten rather than added beside it: nothing in this repository calls
the native SendAsync (ListenerHTTP, WebTransport and ClientHTTP use
SendSync; the example's SendAsync is the browser build's), nor does any
consumer found here (3DForts uses SendSync natively); its contract is
unchanged, copy now and onSent once with whether it went; and the old
one's pool thread, `this` capture and reordering were defects, not
behaviour to keep. What changes is onSent's thread, msquic's worker
rather than a pool task, so a callback that blocks, a *Sync call above
all, now stalls its connection rather than a pool worker. The browser
build is untouched.
Breaking: native onSent runs on msquic's worker thread, not as a pool
task, and must not block.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:24:15 +01:00
|
|
|
//
|
|
|
|
|
// Native: holds no thread. It copies the buffer, hands the copy to
|
|
|
|
|
// msquic and returns; msquic's SEND_COMPLETE for it frees the copy and
|
|
|
|
|
// fires onSent, exactly once: true when msquic took the whole buffer
|
|
|
|
|
// (it is in msquic's send buffer or acknowledged, what SendSync's
|
|
|
|
|
// return means; not a receipt from the peer's application), false
|
|
|
|
|
// when the stream or the connection closed first. Callable from any
|
|
|
|
|
// thread, inside this stream's own callbacks included, and as often
|
|
|
|
|
// as you like: the sends queue in msquic in call order, none waits for
|
|
|
|
|
// another, and BytesInFlight says how far behind they are.
|
feat(quic): QUICStream::OnRecieve and Abort, callbacks that hold no thread
Every way to wait for a stream's bytes held a thread: RecieveSync and
RecieveUntilCloseSync block their caller with no bound, and
RecieveAsync and RecieveUntilCloseAsync are those run as pool tasks, so
they block a worker of the pool the host shares, for good on a stream a
client keeps open. A server embedded in an app, answering requests from
the app's main loop and streaming logs to followers, can afford neither.
OnRecieve(onData(span), onEnd(clean)) delivers the bytes as they arrive
and the end once: clean for the peer's FIN (PEER_SEND_SHUTDOWN, which
msquic raises only once every byte before it has been taken), not clean
for a reset by either side or a connection that went first. Bytes that
arrived before the call are delivered by it, before it returns, and so
is the end; from then on the callbacks run on the msquic worker inside
the event that made them due, with msquic's own buffers handed over
when nothing is queued, so nothing is copied and no thread waits.
The callbacks of one stream, SendAsync's onSent included, never run at
once. Whichever thread finds nobody delivering becomes the delivering
thread and runs what is ready, picking up what others queue meanwhile;
the others queue and return, the worker never waiting. A mutex held
across the callbacks was the alternative: it would make the worker wait
whenever the registering thread was delivering, and deadlock a callback
that called into msquic and waited for that worker. A failed SendAsync
now reports after the callback that made it, or from a pool task.
Abort(errorCode) resets both directions with an application error code
(ABORT_SEND | ABORT_RECEIVE | IMMEDIATE, so SHUTDOWN_COMPLETE, and
onEnd(false), follow without waiting for the peer's acknowledgement).
From inside onData it is how a server enforces a byte cap: no onData
follows, what was queued is dropped, onEnd(false) comes once.
Destroying the wrapper, or Stop(), unregisters the callbacks the way
std::stop_callback does: once it returns neither runs again, one
running on another thread has finished, and one on the destructor's own
thread, a callback destroying its own wrapper, is not waited for. After
the end or the unregistering the callbacks are released, so a callback
holding its own stream does not keep it forever. The callbacks use the
shared Impl only, so moving the wrapper under them is safe.
Breaking: none. Existing receives are unchanged, and must not be mixed
with OnRecieve on one stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:32:48 +01:00
|
|
|
// - onSent runs on the connection's msquic worker thread, never at
|
|
|
|
|
// the same time as another callback of this stream (see
|
|
|
|
|
// OnRecieve). A send that fails before msquic takes it (a closed
|
|
|
|
|
// stream, a moved-from wrapper) reports from a pool task, or
|
|
|
|
|
// after the callback that made it returns, never from inside
|
feat(quic): QUICStream::SendAsync holds no thread and reports from SEND_COMPLETE
Native SendAsync was SendSync run as a pool task. That held a worker of
Crafter.Thread's pool, which the host shares and which runs each
worker's queue in order, for as long as msquic took to accept the
buffer, and a peer that reads slowly stretches that to its own pace: a
server streaming to a slow follower stalled the tasks queued behind it.
The task also reached back through `this`, so moving or destroying the
wrapper before it ran was a use-after-free, and two sends could reach
msquic out of order from two workers.
Now SendAsync copies the buffer into a SendRequest, hands it to msquic
and returns. SEND_COMPLETE, which msquic delivers exactly once for every
send it took and before the stream's SHUTDOWN_COMPLETE, frees the copy
and calls onSent on the msquic worker: true when msquic took the whole
buffer (in its send buffer or acknowledged, which is what SendSync's
return has always meant), false when it was canceled because the stream
or the connection ended first. Sends queue in msquic in call order and
none waits for another. A send msquic never took (a closed stream, a
moved-from wrapper, StreamSend refusing it) reports false from a pool
task, never from inside SendAsync, so a caller may hold a lock its own
callback takes. Only `impl` is read, not `handle`, so SendAsync is
callable from any thread, a callback included; on a stopped stream
msquic cancels the send.
BytesInFlight() and SendsInFlight() count what SendAsync and SendSync
handed msquic whose SEND_COMPLETE has not come, so a server can drop a
follower that falls too far behind.
Rewritten rather than added beside it: nothing in this repository calls
the native SendAsync (ListenerHTTP, WebTransport and ClientHTTP use
SendSync; the example's SendAsync is the browser build's), nor does any
consumer found here (3DForts uses SendSync natively); its contract is
unchanged, copy now and onSent once with whether it went; and the old
one's pool thread, `this` capture and reordering were defects, not
behaviour to keep. What changes is onSent's thread, msquic's worker
rather than a pool task, so a callback that blocks, a *Sync call above
all, now stalls its connection rather than a pool worker. The browser
build is untouched.
Breaking: native onSent runs on msquic's worker thread, not as a pool
task, and must not block.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:24:15 +01:00
|
|
|
// SendAsync, so the caller may hold a lock onSent takes.
|
|
|
|
|
// - In onSent: SendAsync and the other non-blocking calls are fine;
|
|
|
|
|
// nothing that blocks, no *Sync call (it waits for this very
|
|
|
|
|
// worker), no ~ClientQUIC; it must not throw.
|
|
|
|
|
// - It fires even after the wrapper is gone: destroying it lets the
|
|
|
|
|
// sends already made go out, then the FIN.
|
2026-05-19 02:53:50 +02:00
|
|
|
void SendAsync(const void* buffer, std::uint32_t size, bool finish,
|
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived
- --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with
them msquic: a consumer that only needs TCP builds no msquic. The flag
isn't passed on to dependencies.
- --local takes Crafter.Thread from the sibling checkout, as the other
Crafter projects do.
- ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and
SocketTimeoutException.
- TCP receives return the bytes that arrived:
- RecieveSync(n) returned the whole buffer when fewer had come;
- RecieveUntilCloseSync truncated what it had read;
- a signal (EINTR) during a receive threw where it should wait on;
- RecieveSync(size, buffer) returns the count and throws like the others;
- a receive of no bytes returns at once: recv's 0 for it isn't a close.
- Every TCP *Async receive takes an onError(std::exception_ptr): an
exception escaping a ThreadPool task ended the program. An empty onError
drops the error.
- QUIC streams:
- async receives deliver an empty chunk on close, as the browser build does;
- RecieveUntilFullAsync gives what arrived on a stream that closes early;
- RecieveUntilFullSync takes all it asked for or nothing, under one hold
of the lock, so what did arrive stays for the next receive and no
other receive can take bytes from among it;
- SendAsync's callback hears whether the send went out (bool sent): native,
browser, the JS bridge and the example.
- msquic is pinned to v2.6.3 and built against the system OpenSSL
(QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's
openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with
config_diagnostics on) made it fail to initialise. The README says to use
--no-quic where the distro's OpenSSL is older.
- Tests now register on any Linux triple: they never ran under the Flatpak's
x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP,
ShouldRecieveTCP, ShouldReportClosedQUICStream.
Breaking: SendAsync's callback takes a bool, the TCP *Async receives take
an onError, and RecieveSync(size, buffer) returns the count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
|
|
|
std::function<void(bool sent)> onSent);
|
2026-05-19 02:53:50 +02:00
|
|
|
|
|
|
|
|
// Async receive variants. Dispatched on Crafter.Thread's ThreadPool
|
feat: TCP-only builds; TCP options and timeouts; receives that report and keep what arrived
- --no-quic leaves QUIC, HTTP/3 and WebTransport out of the build, and with
them msquic: a consumer that only needs TCP builds no msquic. The flag
isn't passed on to dependencies.
- --local takes Crafter.Thread from the sibling checkout, as the other
Crafter projects do.
- ClientTCP gains SetNoDelay, SetReceiveTimeout, a connect timeout and
SocketTimeoutException.
- TCP receives return the bytes that arrived:
- RecieveSync(n) returned the whole buffer when fewer had come;
- RecieveUntilCloseSync truncated what it had read;
- a signal (EINTR) during a receive threw where it should wait on;
- RecieveSync(size, buffer) returns the count and throws like the others;
- a receive of no bytes returns at once: recv's 0 for it isn't a close.
- Every TCP *Async receive takes an onError(std::exception_ptr): an
exception escaping a ThreadPool task ended the program. An empty onError
drops the error.
- QUIC streams:
- async receives deliver an empty chunk on close, as the browser build does;
- RecieveUntilFullAsync gives what arrived on a stream that closes early;
- RecieveUntilFullSync takes all it asked for or nothing, under one hold
of the lock, so what did arrive stays for the next receive and no
other receive can take bytes from among it;
- SendAsync's callback hears whether the send went out (bool sent): native,
browser, the JS bridge and the example.
- msquic is pinned to v2.6.3 and built against the system OpenSSL
(QUIC_TLS_LIB=openssl, 3.5 or newer). Its bundled quictls read the system's
openssl.cnf, and the Flatpak runtime's (a PKCS#11 engine, with
config_diagnostics on) made it fail to initialise. The README says to use
--no-quic where the distro's OpenSSL is older.
- Tests now register on any Linux triple: they never ran under the Flatpak's
x86_64-unknown-linux-gnu. New: ShouldBuildWithoutQUIC, ShouldConfigureTCP,
ShouldRecieveTCP, ShouldReportClosedQUICStream.
Breaking: SendAsync's callback takes a bool, the TCP *Async receives take
an onError, and RecieveSync(size, buffer) returns the count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 19:07:29 +02:00
|
|
|
// (native) or driven by a per-stream JS reader loop (browser). A
|
|
|
|
|
// stream that has closed gives the callback an empty chunk, and
|
|
|
|
|
// RecieveUntilFullAsync on one that closes early gives what arrived:
|
|
|
|
|
// the browser build has no exceptions to report it with, and on the
|
|
|
|
|
// ThreadPool an exception would end the program.
|
2026-05-06 04:06:17 +02:00
|
|
|
void RecieveAsync(std::function<void(std::vector<char>)> recieveCallback);
|
|
|
|
|
void RecieveUntilCloseAsync(std::function<void(std::vector<char>)> recieveCallback);
|
|
|
|
|
void RecieveUntilFullAsync(std::uint32_t bufferSize, std::function<void(std::vector<char>)> recieveCallback);
|
|
|
|
|
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
|
|
|
|
// Advanced: re-inject already-consumed bytes at the front of the
|
|
|
|
|
// receive queue so the next Recieve* call sees them. Used by
|
|
|
|
|
// protocol demuxers (e.g. the WebTransport stream router in
|
|
|
|
|
// ListenerHTTP) that need to peek a prefix off the wire, then hand
|
|
|
|
|
// the stream to user code as if the prefix had never been read.
|
|
|
|
|
void PrependReceived(std::vector<char> bytes);
|
|
|
|
|
|
|
|
|
|
// Underlying QUIC stream id. Stable for the stream's lifetime.
|
|
|
|
|
// Browsers identify WT streams by the session's CONNECT stream id,
|
|
|
|
|
// so the server has to query and remember it at session creation
|
|
|
|
|
// time. Throws if the stream is not yet started.
|
|
|
|
|
std::uint64_t GetStreamId() const;
|
feat(quic): QUICStream::SendAsync holds no thread and reports from SEND_COMPLETE
Native SendAsync was SendSync run as a pool task. That held a worker of
Crafter.Thread's pool, which the host shares and which runs each
worker's queue in order, for as long as msquic took to accept the
buffer, and a peer that reads slowly stretches that to its own pace: a
server streaming to a slow follower stalled the tasks queued behind it.
The task also reached back through `this`, so moving or destroying the
wrapper before it ran was a use-after-free, and two sends could reach
msquic out of order from two workers.
Now SendAsync copies the buffer into a SendRequest, hands it to msquic
and returns. SEND_COMPLETE, which msquic delivers exactly once for every
send it took and before the stream's SHUTDOWN_COMPLETE, frees the copy
and calls onSent on the msquic worker: true when msquic took the whole
buffer (in its send buffer or acknowledged, which is what SendSync's
return has always meant), false when it was canceled because the stream
or the connection ended first. Sends queue in msquic in call order and
none waits for another. A send msquic never took (a closed stream, a
moved-from wrapper, StreamSend refusing it) reports false from a pool
task, never from inside SendAsync, so a caller may hold a lock its own
callback takes. Only `impl` is read, not `handle`, so SendAsync is
callable from any thread, a callback included; on a stopped stream
msquic cancels the send.
BytesInFlight() and SendsInFlight() count what SendAsync and SendSync
handed msquic whose SEND_COMPLETE has not come, so a server can drop a
follower that falls too far behind.
Rewritten rather than added beside it: nothing in this repository calls
the native SendAsync (ListenerHTTP, WebTransport and ClientHTTP use
SendSync; the example's SendAsync is the browser build's), nor does any
consumer found here (3DForts uses SendSync natively); its contract is
unchanged, copy now and onSent once with whether it went; and the old
one's pool thread, `this` capture and reordering were defects, not
behaviour to keep. What changes is onSent's thread, msquic's worker
rather than a pool task, so a callback that blocks, a *Sync call above
all, now stalls its connection rather than a pool worker. The browser
build is untouched.
Breaking: native onSent runs on msquic's worker thread, not as a pool
task, and must not block.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:24:15 +01:00
|
|
|
|
|
|
|
|
// Bytes, and sends, handed to SendAsync or SendSync whose completion
|
|
|
|
|
// has not come yet: how far behind a slow peer is. A server streaming
|
|
|
|
|
// to a follower can drop one whose backlog passes a cap. Any thread.
|
|
|
|
|
std::uint64_t BytesInFlight() const;
|
|
|
|
|
std::size_t SendsInFlight() const;
|
feat(quic): QUICStream::OnRecieve and Abort, callbacks that hold no thread
Every way to wait for a stream's bytes held a thread: RecieveSync and
RecieveUntilCloseSync block their caller with no bound, and
RecieveAsync and RecieveUntilCloseAsync are those run as pool tasks, so
they block a worker of the pool the host shares, for good on a stream a
client keeps open. A server embedded in an app, answering requests from
the app's main loop and streaming logs to followers, can afford neither.
OnRecieve(onData(span), onEnd(clean)) delivers the bytes as they arrive
and the end once: clean for the peer's FIN (PEER_SEND_SHUTDOWN, which
msquic raises only once every byte before it has been taken), not clean
for a reset by either side or a connection that went first. Bytes that
arrived before the call are delivered by it, before it returns, and so
is the end; from then on the callbacks run on the msquic worker inside
the event that made them due, with msquic's own buffers handed over
when nothing is queued, so nothing is copied and no thread waits.
The callbacks of one stream, SendAsync's onSent included, never run at
once. Whichever thread finds nobody delivering becomes the delivering
thread and runs what is ready, picking up what others queue meanwhile;
the others queue and return, the worker never waiting. A mutex held
across the callbacks was the alternative: it would make the worker wait
whenever the registering thread was delivering, and deadlock a callback
that called into msquic and waited for that worker. A failed SendAsync
now reports after the callback that made it, or from a pool task.
Abort(errorCode) resets both directions with an application error code
(ABORT_SEND | ABORT_RECEIVE | IMMEDIATE, so SHUTDOWN_COMPLETE, and
onEnd(false), follow without waiting for the peer's acknowledgement).
From inside onData it is how a server enforces a byte cap: no onData
follows, what was queued is dropped, onEnd(false) comes once.
Destroying the wrapper, or Stop(), unregisters the callbacks the way
std::stop_callback does: once it returns neither runs again, one
running on another thread has finished, and one on the destructor's own
thread, a callback destroying its own wrapper, is not waited for. After
the end or the unregistering the callbacks are released, so a callback
holding its own stream does not keep it forever. The callbacks use the
shared Impl only, so moving the wrapper under them is safe.
Breaking: none. Existing receives are unchanged, and must not be mixed
with OnRecieve on one stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:32:48 +01:00
|
|
|
|
|
|
|
|
// Receive by callback, holding no thread while nothing arrives.
|
|
|
|
|
// onData gets each run of bytes as it arrives, as a span valid only
|
|
|
|
|
// for the call (copy what you keep); onEnd fires once, after the last
|
|
|
|
|
// onData: clean=true when the peer finished its side (FIN), false
|
|
|
|
|
// when either side aborted the stream (Abort(), the peer's reset) or
|
|
|
|
|
// the connection went first.
|
|
|
|
|
// - What arrived before the call is delivered first, on the calling
|
|
|
|
|
// thread before OnRecieve returns, and so is onEnd if the stream
|
|
|
|
|
// had already ended; from then on the callbacks run on the
|
|
|
|
|
// connection's msquic worker thread. No two callbacks of one
|
|
|
|
|
// stream (onData, onEnd, SendAsync's onSent) ever run at once.
|
|
|
|
|
// - Inside them: SendAsync, Abort, BytesInFlight, SendsInFlight,
|
|
|
|
|
// and destroying or moving this wrapper are fine. Nothing that
|
|
|
|
|
// blocks: no *Sync call on any stream (it waits for this very
|
|
|
|
|
// worker), no ~ClientQUIC (it waits for its streams to drain), no
|
|
|
|
|
// waiting on a thread that may be waiting on this one. A callback
|
|
|
|
|
// that throws ends the program.
|
|
|
|
|
// - Destroying the wrapper, or Stop(), unregisters both: once that
|
|
|
|
|
// returns neither runs again, a callback running on another
|
|
|
|
|
// thread has finished (as with std::stop_callback; one on this
|
|
|
|
|
// thread, destroying its own wrapper, is not waited for), and an
|
|
|
|
|
// onEnd not delivered by then never is. So do not destroy it
|
|
|
|
|
// while holding a lock its callbacks take. Sends still complete.
|
|
|
|
|
// - Once per stream: a second call throws QUICException. Do not mix
|
|
|
|
|
// with the Recieve* calls, which then get nothing. On a stream
|
|
|
|
|
// that cannot receive, or a closed or moved-from wrapper, onEnd
|
|
|
|
|
// (false) fires at once.
|
|
|
|
|
void OnRecieve(std::function<void(std::span<const char> data)> onData,
|
|
|
|
|
std::function<void(bool clean)> onEnd);
|
|
|
|
|
|
|
|
|
|
// Reset both directions at once with a QUIC application error code
|
|
|
|
|
// (62 bits; larger values are clamped): the peer's receive side sees
|
|
|
|
|
// a reset and its send side a stop-sending. Returns at once. Sends
|
|
|
|
|
// not yet delivered report false, no onData follows (not even the
|
|
|
|
|
// rest of a run being delivered when called from inside one), and
|
|
|
|
|
// onEnd(false) follows unless the end was delivered already or the
|
|
|
|
|
// wrapper is destroyed first. Callable from any thread and from
|
|
|
|
|
// inside this stream's callbacks, which is how a server enforces a
|
|
|
|
|
// byte cap. A no-op on a closed or moved-from wrapper.
|
|
|
|
|
void Abort(std::uint64_t errorCode = 0);
|
2026-05-19 02:53:50 +02:00
|
|
|
#endif
|
|
|
|
|
|
fix(quic): close stream handles whose async StreamStart lost the teardown race
A client that sends on a stream while its connection is being torn down —
the shape any long-lived client with periodic acks or keepalives has —
would hang at exit inside MsQuicRegistrationClose, and occasionally
bugcheck on an msquic worker thread instead.
Root cause: StreamStart is asynchronous. It returns QUIC_STATUS_PENDING
and queues the start onto the connection, so it can still fail later, and
it fails with QUIC_STATUS_INVALID_STATE whenever the connection is shut
down before the queued start runs. Passed QUIC_STREAM_START_FLAG_NONE,
msquic leaves such a stream neither started nor shut down, so
SHUTDOWN_COMPLETE never arrives. The stream close is driven entirely off
that event, so the handle stays open: ~ClientQUIC's bounded drain expires,
the connection is closed with a stream msquic still considers open, and
the registration's rundown at exit() waits forever for the handle.
Instrumenting the handle lifecycle over the reproducer shows one leaked
locally-opened stream per failing run, correlating 1:1 with a
START_COMPLETE carrying 0x1 (INVALID_STATE).
Fixes:
- OpenStream passes QUIC_STREAM_START_FLAG_SHUTDOWN_ON_FAIL, so a start
that loses the race is followed by SHUTDOWN_COMPLETE like any other
stream and its handle is closed.
- QUICStream::Stop also aborts the receive direction. GRACEFUL closes
only the send side and msquic rejects it combined with any other flag,
so a bidirectional stream whose peer keeps its send side open never
reached SHUTDOWN_COMPLETE either.
- The msquic stream handle is now refcounted, held by the callback plus
every app thread inside an msquic call on it, and closed by whoever
lets go last. StreamSend/StreamShutdown/GetParam could previously run
against a handle a worker thread had just closed — a use-after-free
msquic reports as a bugcheck. A refcount rather than an exclusion lock
because GetParam blocks on the connection's worker, which must never
be the thread waiting.
- ~ClientQUIC drops undispatched peer streams before closing the
connection rather than with the rest of Impl afterwards, so the drain
covers them.
The reproducer (tests/ShouldSurviveConnectionChurn) grows a client-opened
bidirectional control stream with a thread sending across the teardown,
and no longer ends in std::_Exit — returning from main runs the static
MsQuicRuntime destructor, and its RegistrationClose is where a leaked
handle shows up. Before: 5 of 15 runs clean, 10 hung. After: 90 of 90
clean, with every opened handle observed closed and the drain never
expiring.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 23:18:01 +00:00
|
|
|
// Shut the stream down in both directions: our send side gracefully
|
|
|
|
|
// (the peer sees a FIN), the receive side abortively. Aborting the
|
|
|
|
|
// receive side matters — a bidirectional stream whose peer keeps its
|
|
|
|
|
// send side open otherwise never completes its shutdown, and so never
|
|
|
|
|
// releases its transport handle. Returns immediately; completion is
|
|
|
|
|
// asynchronous. Called by the destructor, so an explicit call is only
|
|
|
|
|
// needed to shut a stream down ahead of its wrapper going away.
|
feat(quic): QUICStream::OnRecieve and Abort, callbacks that hold no thread
Every way to wait for a stream's bytes held a thread: RecieveSync and
RecieveUntilCloseSync block their caller with no bound, and
RecieveAsync and RecieveUntilCloseAsync are those run as pool tasks, so
they block a worker of the pool the host shares, for good on a stream a
client keeps open. A server embedded in an app, answering requests from
the app's main loop and streaming logs to followers, can afford neither.
OnRecieve(onData(span), onEnd(clean)) delivers the bytes as they arrive
and the end once: clean for the peer's FIN (PEER_SEND_SHUTDOWN, which
msquic raises only once every byte before it has been taken), not clean
for a reset by either side or a connection that went first. Bytes that
arrived before the call are delivered by it, before it returns, and so
is the end; from then on the callbacks run on the msquic worker inside
the event that made them due, with msquic's own buffers handed over
when nothing is queued, so nothing is copied and no thread waits.
The callbacks of one stream, SendAsync's onSent included, never run at
once. Whichever thread finds nobody delivering becomes the delivering
thread and runs what is ready, picking up what others queue meanwhile;
the others queue and return, the worker never waiting. A mutex held
across the callbacks was the alternative: it would make the worker wait
whenever the registering thread was delivering, and deadlock a callback
that called into msquic and waited for that worker. A failed SendAsync
now reports after the callback that made it, or from a pool task.
Abort(errorCode) resets both directions with an application error code
(ABORT_SEND | ABORT_RECEIVE | IMMEDIATE, so SHUTDOWN_COMPLETE, and
onEnd(false), follow without waiting for the peer's acknowledgement).
From inside onData it is how a server enforces a byte cap: no onData
follows, what was queued is dropped, onEnd(false) comes once.
Destroying the wrapper, or Stop(), unregisters the callbacks the way
std::stop_callback does: once it returns neither runs again, one
running on another thread has finished, and one on the destructor's own
thread, a callback destroying its own wrapper, is not waited for. After
the end or the unregistering the callbacks are released, so a callback
holding its own stream does not keep it forever. The callbacks use the
shared Impl only, so moving the wrapper under them is safe.
Breaking: none. Existing receives are unchanged, and must not be mixed
with OnRecieve on one stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:32:48 +01:00
|
|
|
// Natively it also unregisters OnRecieve's callbacks (see there);
|
|
|
|
|
// sends already made still go out, ahead of the FIN.
|
2026-05-06 04:06:17 +02:00
|
|
|
void Stop();
|
|
|
|
|
|
|
|
|
|
private:
|
|
|
|
|
struct Impl;
|
2026-07-22 17:55:45 +02:00
|
|
|
// shared_ptr (not unique_ptr): the msquic stream callback holds its own
|
|
|
|
|
// ref so it can never run on a freed Impl if the wrapper is destroyed
|
|
|
|
|
// before SHUTDOWN_COMPLETE fires. See Crafter.Network-ClientQUIC.cpp.
|
|
|
|
|
std::shared_ptr<Impl> impl;
|
2026-05-06 04:06:17 +02:00
|
|
|
friend class ClientQUIC;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
// A QUIC connection. On the client side, constructing one initiates the
|
2026-05-19 02:53:50 +02:00
|
|
|
// handshake and (on native) blocks until it succeeds, or throws on
|
|
|
|
|
// failure. On the server side, ListenerQUIC instantiates these for
|
|
|
|
|
// accepted peers.
|
2026-05-06 04:06:17 +02:00
|
|
|
//
|
|
|
|
|
// A connection multiplexes:
|
|
|
|
|
// - Reliable, ordered streams (open via OpenStream() / observe inbound
|
|
|
|
|
// via OnStream()).
|
|
|
|
|
// - Unreliable, unordered datagrams (SendDatagram() / OnDatagram()).
|
|
|
|
|
//
|
2026-08-25 17:06:51 +00:00
|
|
|
// Lifetime: ~ClientQUIC shuts the connection down and closes it, waiting
|
|
|
|
|
// (bounded — 5s) for every stream on it to finish closing first, since
|
fix(quic): let ~ClientQUIC's CONNECTION_CLOSE reach the peer
~ClientQUIC asked msquic to shut the connection down, waited for its
streams to drain, and closed the handle. With no stream open the drain
returns at once, so ConnectionClose followed ConnectionShutdown within
microseconds. msquic sends the CONNECTION_CLOSE frame from a FLUSH_SEND
operation it queues while it processes the shutdown; the close the
destructor had queued meanwhile reached the worker first, forced the
shutdown to complete silently (QuicConnTryClose: "silent close forced
after we already started the close process"), and the flush, which is
ignored once ShutdownComplete is set, sent nothing. The peer learned of
the close only at its idle timeout, two minutes by default: a server
whose clients each live for one request, such as a debug server driven
by a CLI that connects once per command, kept every one of them that
long.
The destructor now also waits, after the drain, for the connection's
SHUTDOWN_COMPLETE before closing the handle. That comes about one round
trip later once the peer answers the close, or after msquic's closing
period (three PTOs) when it does not; the wait is bounded at 1 s, since
the frame went out at the start either way and a peer that is gone must
not hold the destructor for the whole closing period. The handler sets a
flag for it before it looks at the handle, which the destructor holds,
so the close stays the destructor's.
Measured with ShouldTellPeerOfCloseQUIC: before, the server was not told
of the first of 20 client closes within 2 s (and, counted another way,
8 of 100 peer closes reached the server within 10 s in one run, 0 of
100 in the next); after, all 20 within a millisecond.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:06:39 +01:00
|
|
|
// QUICStream::Stop only initiates a shutdown that completes asynchronously,
|
|
|
|
|
// and then (bounded — 1s, about a round trip when the peer answers) for
|
|
|
|
|
// the shutdown to complete, so the peer is sent the close rather than
|
|
|
|
|
// left to time the connection out.
|
2026-08-25 17:06:51 +00:00
|
|
|
// Streams outliving the connection are therefore safe to hold, though they
|
|
|
|
|
// are dead once it goes; prefer destroying or moving them out first.
|
2026-05-19 02:53:50 +02:00
|
|
|
//
|
|
|
|
|
// Browser build: the only QUIC-shaped API the browser exposes is
|
|
|
|
|
// WebTransport, which is HTTP/3-based and reached at a fixed URL. Here:
|
|
|
|
|
// - The constructor returns immediately; the connection is opened in
|
|
|
|
|
// the background. Operations issued before the connection is ready
|
|
|
|
|
// are queued JS-side until WebTransport's "ready" promise resolves
|
|
|
|
|
// (or fail with QUICClosedException if the connection rejects).
|
|
|
|
|
// - `alpn` is mapped to the URL path: new WebTransport(
|
|
|
|
|
// `https://${host}:${port}/${alpn}`). The QUIC-layer ALPN itself
|
|
|
|
|
// is fixed to "h3" by the browser and cannot be customised.
|
|
|
|
|
// - The server side must accept WebTransport sessions (HTTP/3 extended
|
|
|
|
|
// CONNECT) on the path equal to `alpn`. Plain QUIC with a custom
|
|
|
|
|
// ALPN — what ListenerQUIC offers today — is not reachable from a
|
|
|
|
|
// browser.
|
|
|
|
|
// - Synchronous send/receive methods are not compiled. Use the *Async
|
|
|
|
|
// variants instead.
|
2026-05-06 04:06:17 +02:00
|
|
|
export class ClientQUIC {
|
|
|
|
|
public:
|
|
|
|
|
// ALPN identifier exchanged in the handshake. Both peers must agree.
|
|
|
|
|
// For 3DForts use e.g. "f3d/1" or similar — a short stable token.
|
2026-05-19 02:53:50 +02:00
|
|
|
// On the browser build, this is the WebTransport URL path instead
|
|
|
|
|
// of an ALPN token; see the class comment above.
|
2026-05-06 04:06:17 +02:00
|
|
|
std::string alpn;
|
|
|
|
|
|
|
|
|
|
// Client constructor: connects to host:port using QUIC. ALPN must
|
|
|
|
|
// match the listener. Throws QUICException on connect failure.
|
2026-08-25 17:06:51 +00:00
|
|
|
// Blocks for at most the handshake idle timeout (10s), so a host that
|
|
|
|
|
// never answers fails in seconds rather than at the idle timeout.
|
2026-05-06 04:06:17 +02:00
|
|
|
ClientQUIC(const char* host, std::uint16_t port, std::string alpn,
|
|
|
|
|
QUICClientCredentials creds = {});
|
|
|
|
|
ClientQUIC(std::string host, std::uint16_t port, std::string alpn,
|
|
|
|
|
QUICClientCredentials creds = {});
|
|
|
|
|
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
2026-05-06 04:06:17 +02:00
|
|
|
// Server-side constructor used by ListenerQUIC for accepted peers.
|
|
|
|
|
// Takes ownership of an already-accepted msquic connection handle
|
|
|
|
|
// and the server configuration handle. Not intended for direct use.
|
|
|
|
|
ClientQUIC(HQUIC connectionHandle, HQUIC serverConfiguration, std::string alpn);
|
2026-05-19 02:53:50 +02:00
|
|
|
#endif
|
2026-05-06 04:06:17 +02:00
|
|
|
|
|
|
|
|
~ClientQUIC();
|
|
|
|
|
ClientQUIC(const ClientQUIC&) = delete;
|
|
|
|
|
ClientQUIC(ClientQUIC&&) noexcept;
|
|
|
|
|
|
2026-05-07 00:06:44 +02:00
|
|
|
// Open a new stream initiated by this side. Defaults to bidirectional;
|
|
|
|
|
// pass unidirectional=true to open a one-way send stream (used for
|
|
|
|
|
// HTTP/3's control + QPACK encoder/decoder streams).
|
2026-05-06 04:06:17 +02:00
|
|
|
// Blocks until the stream is started; throws on failure.
|
2026-05-07 00:06:44 +02:00
|
|
|
QUICStream OpenStream(bool unidirectional = false);
|
2026-05-06 04:06:17 +02:00
|
|
|
|
|
|
|
|
// Send a datagram. Best-effort: may be silently dropped under loss
|
|
|
|
|
// or congestion. Size must fit within the path MTU (msquic surfaces
|
|
|
|
|
// the maximum via QUIC_PARAM_CONN_DATAGRAM_SEND_ENABLED — typically
|
|
|
|
|
// ~1200 bytes safely on the open internet).
|
|
|
|
|
void SendDatagram(const void* buffer, std::uint32_t size);
|
|
|
|
|
|
|
|
|
|
// Register a handler for streams the peer initiates toward us.
|
|
|
|
|
// Called on the msquic worker; offload heavy work to ThreadPool.
|
|
|
|
|
void OnStream(std::function<void(QUICStream)> callback);
|
|
|
|
|
|
|
|
|
|
// Register a handler for datagrams from the peer. Called on the
|
|
|
|
|
// msquic worker; copy/queue and return promptly.
|
|
|
|
|
void OnDatagram(std::function<void(std::vector<char>)> callback);
|
|
|
|
|
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
2026-05-06 04:06:17 +02:00
|
|
|
// Block the caller until the next datagram arrives; returns it.
|
|
|
|
|
// Throws QUICClosedException if the connection closes first.
|
|
|
|
|
std::vector<char> RecieveDatagramSync();
|
feat(quic): ClientQUIC::OnClosed tells the owner the connection is over
A ListenerQUIC hands each accepted connection to its callback as a
heap-allocated ClientQUIC the callback owns, and nothing said when to
delete it: the SHUTDOWN_COMPLETE handler closed the msquic handle of a
connection the peer or the idle timeout ended, but the object stayed, so
a server either leaked its peers or deleted them blind, and one still
alive at exit holds msquic's registration open until its idle timeout.
OnClosed(callback) is told once when the connection has shut down for
any reason but this object's destruction: the peer closed it, the idle
timeout or a transport error ended it, or Stop() was called. It runs on
the msquic worker, in the SHUTDOWN_COMPLETE handler, after that handler
has claimed and closed the handle, so deleting the ClientQUIC from
inside it is safe: ~ClientQUIC then finds no handle to claim and waits
for nothing. Delivering it there, rather than as a pool task, keeps it
independent of a pool the host shares and possibly saturates; the cost
is the usual worker rule, which the interface spells out (no blocking,
no *Sync call, no deleting another ClientQUIC from inside it). A
callback set after the close, which happens when the peer goes before a
ListenerQUIC's connectCallback runs, fires as a pool task instead, never
from inside OnClosed, where the caller is still using the object.
Its state is a shared CloseNotice rather than part of the Impl, since
the handler fires it after giving up the handle, when ~ClientQUIC on
another thread no longer waits for the worker and may free the Impl.
The rules are std::stop_callback's: ~ClientQUIC marks the owner gone
first, after which the callback never starts; one already running on
another thread is waited for, so the owner may free what it uses as
soon as the destructor returns; one on the destructor's own thread, a
callback deleting its connection, is not. The handler takes the handle
and the notice under one lock, so the destructor cannot slip between.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 02:18:50 +01:00
|
|
|
|
|
|
|
|
// Told once when the connection has shut down for any reason other
|
|
|
|
|
// than this object being destroyed: the peer closed it, the idle
|
|
|
|
|
// timeout or a transport error ended it, or Stop() was called. This
|
|
|
|
|
// is how the owner of a connection a ListenerQUIC accepted learns
|
|
|
|
|
// that it is time to delete it.
|
|
|
|
|
//
|
|
|
|
|
// - It runs on the connection's msquic worker thread, once the
|
|
|
|
|
// connection's handle is closed; if the connection had already
|
|
|
|
|
// closed when OnClosed was called, it runs as a pool task
|
|
|
|
|
// instead, never inside OnClosed itself.
|
|
|
|
|
// - Deleting this ClientQUIC from inside it is safe, and the usual
|
|
|
|
|
// thing to do. Do nothing else that blocks: deleting a
|
|
|
|
|
// *different* ClientQUIC, or any *Sync call, waits on msquic
|
|
|
|
|
// workers, this one perhaps among them. It must not throw; an
|
|
|
|
|
// exception leaving it ends the program.
|
|
|
|
|
// - Once ~ClientQUIC has begun it never starts, and a run already
|
|
|
|
|
// under way on another thread finishes before ~ClientQUIC goes
|
|
|
|
|
// on, as with std::stop_callback (one on this thread, deleting
|
|
|
|
|
// its own connection, is not waited for). So do not destroy the
|
|
|
|
|
// ClientQUIC while holding a lock the callback takes.
|
|
|
|
|
// - Calling OnClosed again replaces the callback, and an empty one
|
|
|
|
|
// unregisters it; once it has fired, later calls do nothing.
|
|
|
|
|
void OnClosed(std::function<void()> callback);
|
2026-05-19 02:53:50 +02:00
|
|
|
#endif
|
2026-05-06 04:06:17 +02:00
|
|
|
|
|
|
|
|
// Cleanly shut down the connection.
|
|
|
|
|
void Stop();
|
|
|
|
|
|
2026-05-19 02:53:50 +02:00
|
|
|
#ifndef CRAFTER_NETWORK_BROWSER
|
2026-05-06 04:06:17 +02:00
|
|
|
// Underlying handle for advanced use (parameter queries, etc.).
|
|
|
|
|
HQUIC GetHandle() const;
|
2026-05-19 02:53:50 +02:00
|
|
|
#endif
|
2026-05-06 04:06:17 +02:00
|
|
|
|
|
|
|
|
private:
|
|
|
|
|
struct Impl;
|
|
|
|
|
std::unique_ptr<Impl> impl;
|
|
|
|
|
friend class ListenerQUIC;
|
|
|
|
|
friend class QUICStream;
|
|
|
|
|
};
|
|
|
|
|
}
|