Catcrafts/Crafter.Network
Watch
2
0
Fork
You've already forked Crafter.Network
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(http1): ClientHTTP1::Fetch follows redirects by the RFC, on a URL type

Browse source 
One request to an http(s) URL with redirects followed by RFC 9110 §15.4:
301, 302, 303, 307 and 308 for every method; a 303 makes the request a
GET without its body and content headers, a 301 or 302 does so for a POST
alone, a 307 or 308 keeps the method and the body. The Location is
resolved against the sending URL by RFC 3986, a hop between http and
https is allowed, Authorization, Proxy-Authorization and Cookie are
dropped on a change of origin, and each hop is a client of its own, since
one ClientHTTP1 is one origin. A redirect without a usable Location, to
another scheme, back to a URL already visited or past maxRedirects comes
back as the response, with `stop` saying why.

The new :URL partition is the URL type behind it, exported: Parse,
Resolve (RFC 3986 §5.2; its §5.4 examples are the test), ToString, and
the pieces a request is built from.

This replaces the FetchHTTP1 free function, which followed redirects the
way Python's urllib did (its repeat limit, its quoting of the Location, a
POST turned back on a 307) for a port that no longer needs the fidelity.

Breaking: FetchHTTP1, FetchOptionsHTTP1 and FetchResultHTTP1 are gone;
use ClientHTTP1::Fetch, FetchOptions and FetchResult, whose `url` is a
URL and whose `limited` became `stop`.
This commit is contained in:
Jorijn van der Graaf 2026-10-31 20:39:43 +01:00
commit 7ee938c98e
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
10 changed files with 596 additions and 372 deletions
Download patch file Download diff file Expand all files Collapse all files

116
interfaces/Crafter.Network-ClientHTTP1.cppm
View file

@ -6,9 +6,53 @@ import std;
import :HTTP;
import :HTTP1;
import :TLS;
import :URL;
#ifndef CRAFTER_NETWORK_BROWSER
namespace Crafter {
// What ClientHTTP1::Fetch needs beyond the URL.
export struct FetchOptions {
std::string method = "GET";
// Lowercase names, as on HTTPRequest. Host comes from the URL.
std::unordered_map<std::string, std::string> headers;
std::string body;
// As on ClientHTTP1, for every request of the chain.
std::chrono::milliseconds connectTimeout{15000};
std::chrono::milliseconds handshakeTimeout{15000};
std::chrono::milliseconds timeout{30000};
HTTP1::MessageLimits limits;
// For every https:// request of the chain.
TLSClientCredentials credentials;
// How many redirects are followed at most; the one after comes back
// unfollowed.
std::uint32_t maxRedirects = 10;
};
// Why a Fetch ended on a redirect it did not follow.
export enum class FetchStop {
// It did not: the chain ended on a final response.
None,
// The redirect carried no Location, or one that is no URL.
NoLocation,
// The Location names a scheme other than http or https.
OtherScheme,
// The Location is a URL the chain has already been sent to.
Loop,
// maxRedirects redirects had been followed already.
Limit,
};
export struct FetchResult {
// The last response of the chain: the final response, or the
// redirect that was not followed, with `stop` saying why.
HTTPResponse response;
// The URL that answered with `response`.
URL url;
// Redirects followed to get there.
std::uint32_t redirects = 0;
FetchStop stop = FetchStop::None;
};
// HTTP/1.1 client over TCP, with or without TLS, for peers that cannot
// speak HTTP/3. The request/response types are the ones the HTTP/3 client
// uses, so swapping ClientHTTP for ClientHTTP1 is a one-line change at the
@ -78,6 +122,27 @@ namespace Crafter {
// Drop the pooled connection; the next Send() dials again.
void Disconnect();
// One request to an http:// or https:// URL, following redirects by
// RFC 9110 §15.4: 301, 302, 303, 307 and 308, for every method. A
// 303 turns the request into a GET (a HEAD stays a HEAD) without its
// body and content headers; 301 and 302 do the same for a POST
// alone; 307 and 308 keep the method and the body. The Location is
// resolved against the URL that sent it (URL::Resolve), and a
// redirect may move between http:// and https://, `credentials`
// serving every https:// hop. Authorization, Proxy-Authorization and
// Cookie are not carried to another origin. Each hop is a client of
// its own, since one ClientHTTP1 is one origin: the redirect's body
// is read and its connection closed before the next request goes
// out. A redirect without a usable Location, to another scheme, to
// a URL already visited, or past maxRedirects comes back as the
// response with `stop` set. Proxies are not consulted.
//
// Throws std::invalid_argument for a URL that is not http(s) or that
// URL::Parse refuses, and whatever a request throws otherwise
// (SocketErrorException, SocketTimeoutException, TLSException,
// HTTP1::HTTP1ProtocolError, std::runtime_error).
static FetchResult Fetch(std::string_view url, const FetchOptions& options = {});
// Limits applied to responses. Set before the first Send().
HTTP1::MessageLimits limits;
// How long to wait for the next piece of a response before giving
@ -97,56 +162,5 @@ namespace Crafter {
struct Impl;
std::unique_ptr<Impl> impl;
};
// What FetchHTTP1 needs beyond the URL.
export struct FetchOptionsHTTP1 {
std::string method = "GET";
// Lowercase names, as on HTTPRequest. Host comes from the URL.
std::unordered_map<std::string, std::string> headers;
std::string body;
// As on ClientHTTP1, for every request of the chain.
std::chrono::milliseconds connectTimeout{15000};
std::chrono::milliseconds handshakeTimeout{15000};
std::chrono::milliseconds timeout{30000};
HTTP1::MessageLimits limits;
// For every https:// request of the chain.
TLSClientCredentials credentials;
// Python urllib's limits: a chain stops at a redirect to a URL it has
// already been sent to `maxRepeats` times, or once it has been sent
// to `maxRedirects` different URLs.
std::uint32_t maxRedirects = 10;
std::uint32_t maxRepeats = 4;
};
export struct FetchResultHTTP1 {
// The last response of the chain. A redirect only when it had nothing
// to follow — no Location, a method urllib does not redirect, a
// scheme other than http(s) — or when the limits stopped the chain.
HTTPResponse response;
// The URL that answered with `response`.
std::string url;
// Redirects followed to get there.
std::uint32_t redirects = 0;
// The limits stopped the chain; `response` is the redirect that would
// have gone past them.
bool limited = false;
};
// One request to an http:// or https:// URL, following redirects the way
// Python's urllib does, so a port of urllib code keeps its behaviour:
// 301, 302, 303, 307 and 308 are followed for GET and HEAD, the first
// three also for POST; the redirected request is a GET (a HEAD stays a
// HEAD) without the body and its content headers. A relative Location is
// resolved against the URL that sent it, as urljoin resolves it; a
// redirect may switch between http:// and https://. Each redirect's body
// is read before the next request goes out. Proxies are not consulted.
//
// The URL is `scheme://[user@]host[:port][/path][?query][#fragment]`,
// with an IPv6 host in brackets; the fragment is not sent and the user
// part is dropped. Throws std::invalid_argument for a URL that is not
// http(s) or has no host or a bad port, and whatever a request throws
// (SocketErrorException, SocketTimeoutException, TLSException,
// std::runtime_error) otherwise.
export FetchResultHTTP1 FetchHTTP1(std::string_view url, const FetchOptionsHTTP1& options = {});
}
#endif