feat(quic): QUICStream::OnRecieve and Abort, callbacks that hold no thread

Every way to wait for a stream's bytes held a thread: RecieveSync and
RecieveUntilCloseSync block their caller with no bound, and
RecieveAsync and RecieveUntilCloseAsync are those run as pool tasks, so
they block a worker of the pool the host shares, for good on a stream a
client keeps open. A server embedded in an app, answering requests from
the app's main loop and streaming logs to followers, can afford neither.

OnRecieve(onData(span), onEnd(clean)) delivers the bytes as they arrive
and the end once: clean for the peer's FIN (PEER_SEND_SHUTDOWN, which
msquic raises only once every byte before it has been taken), not clean
for a reset by either side or a connection that went first. Bytes that
arrived before the call are delivered by it, before it returns, and so
is the end; from then on the callbacks run on the msquic worker inside
the event that made them due, with msquic's own buffers handed over
when nothing is queued, so nothing is copied and no thread waits.

The callbacks of one stream, SendAsync's onSent included, never run at
once. Whichever thread finds nobody delivering becomes the delivering
thread and runs what is ready, picking up what others queue meanwhile;
the others queue and return, the worker never waiting. A mutex held
across the callbacks was the alternative: it would make the worker wait
whenever the registering thread was delivering, and deadlock a callback
that called into msquic and waited for that worker. A failed SendAsync
now reports after the callback that made it, or from a pool task.

Abort(errorCode) resets both directions with an application error code
(ABORT_SEND | ABORT_RECEIVE | IMMEDIATE, so SHUTDOWN_COMPLETE, and
onEnd(false), follow without waiting for the peer's acknowledgement).
From inside onData it is how a server enforces a byte cap: no onData
follows, what was queued is dropped, onEnd(false) comes once.

Destroying the wrapper, or Stop(), unregisters the callbacks the way
std::stop_callback does: once it returns neither runs again, one
running on another thread has finished, and one on the destructor's own
thread, a callback destroying its own wrapper, is not waited for. After
the end or the unregistering the callbacks are released, so a callback
holding its own stream does not keep it forever. The callbacks use the
shared Impl only, so moving the wrapper under them is safe.

Breaking: none. Existing receives are unchanged, and must not be mixed
with OnRecieve on one stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Jorijn van der Graaf 2026-11-01 02:32:48 +01:00
commit c5b4e18eb8
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
2 changed files with 316 additions and 28 deletions

View file

@ -150,9 +150,11 @@ namespace Crafter {
// thread, inside this stream's own callbacks included, and as often
// as you like: the sends queue in msquic in call order, none waits for
// another, and BytesInFlight says how far behind they are.
// - onSent runs on the connection's msquic worker thread. A send
// that fails before msquic takes it (a closed stream, a moved-from
// wrapper) reports from a pool task instead, never from inside
// - onSent runs on the connection's msquic worker thread, never at
// the same time as another callback of this stream (see
// OnRecieve). A send that fails before msquic takes it (a closed
// stream, a moved-from wrapper) reports from a pool task, or
// after the callback that made it returns, never from inside
// SendAsync, so the caller may hold a lock onSent takes.
// - In onSent: SendAsync and the other non-blocking calls are fine;
// nothing that blocks, no *Sync call (it waits for this very
@ -191,6 +193,47 @@ namespace Crafter {
// to a follower can drop one whose backlog passes a cap. Any thread.
std::uint64_t BytesInFlight() const;
std::size_t SendsInFlight() const;
// Receive by callback, holding no thread while nothing arrives.
// onData gets each run of bytes as it arrives, as a span valid only
// for the call (copy what you keep); onEnd fires once, after the last
// onData: clean=true when the peer finished its side (FIN), false
// when either side aborted the stream (Abort(), the peer's reset) or
// the connection went first.
// - What arrived before the call is delivered first, on the calling
// thread before OnRecieve returns, and so is onEnd if the stream
// had already ended; from then on the callbacks run on the
// connection's msquic worker thread. No two callbacks of one
// stream (onData, onEnd, SendAsync's onSent) ever run at once.
// - Inside them: SendAsync, Abort, BytesInFlight, SendsInFlight,
// and destroying or moving this wrapper are fine. Nothing that
// blocks: no *Sync call on any stream (it waits for this very
// worker), no ~ClientQUIC (it waits for its streams to drain), no
// waiting on a thread that may be waiting on this one. A callback
// that throws ends the program.
// - Destroying the wrapper, or Stop(), unregisters both: once that
// returns neither runs again, a callback running on another
// thread has finished (as with std::stop_callback; one on this
// thread, destroying its own wrapper, is not waited for), and an
// onEnd not delivered by then never is. So do not destroy it
// while holding a lock its callbacks take. Sends still complete.
// - Once per stream: a second call throws QUICException. Do not mix
// with the Recieve* calls, which then get nothing. On a stream
// that cannot receive, or a closed or moved-from wrapper, onEnd
// (false) fires at once.
void OnRecieve(std::function<void(std::span<const char> data)> onData,
std::function<void(bool clean)> onEnd);
// Reset both directions at once with a QUIC application error code
// (62 bits; larger values are clamped): the peer's receive side sees
// a reset and its send side a stop-sending. Returns at once. Sends
// not yet delivered report false, no onData follows (not even the
// rest of a run being delivered when called from inside one), and
// onEnd(false) follows unless the end was delivered already or the
// wrapper is destroyed first. Callable from any thread and from
// inside this stream's callbacks, which is how a server enforces a
// byte cap. A no-op on a closed or moved-from wrapper.
void Abort(std::uint64_t errorCode = 0);
#endif
// Shut the stream down in both directions: our send side gracefully
@ -200,6 +243,8 @@ namespace Crafter {
// releases its transport handle. Returns immediately; completion is
// asynchronous. Called by the destructor, so an explicit call is only
// needed to shut a stream down ahead of its wrapper going away.
// Natively it also unregisters OnRecieve's callbacks (see there);
// sends already made still go out, ahead of the FIN.
void Stop();
private: