Catcrafts/Crafter.Network
Watch
2
0
Fork
You've already forked Crafter.Network
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(http1): ClientHTTP1::Fetch follows redirects by the RFC, on a URL type

Browse source 
One request to an http(s) URL with redirects followed by RFC 9110 §15.4:
301, 302, 303, 307 and 308 for every method; a 303 makes the request a
GET without its body and content headers, a 301 or 302 does so for a POST
alone, a 307 or 308 keeps the method and the body. The Location is
resolved against the sending URL by RFC 3986, a hop between http and
https is allowed, Authorization, Proxy-Authorization and Cookie are
dropped on a change of origin, and each hop is a client of its own, since
one ClientHTTP1 is one origin. A redirect without a usable Location, to
another scheme, back to a URL already visited or past maxRedirects comes
back as the response, with `stop` saying why.

The new :URL partition is the URL type behind it, exported: Parse,
Resolve (RFC 3986 §5.2; its §5.4 examples are the test), ToString, and
the pieces a request is built from.

This replaces the FetchHTTP1 free function, which followed redirects the
way Python's urllib did (its repeat limit, its quoting of the Location, a
POST turned back on a 307) for a port that no longer needs the fidelity.

Breaking: FetchHTTP1, FetchOptionsHTTP1 and FetchResultHTTP1 are gone;
use ClientHTTP1::Fetch, FetchOptions and FetchResult, whose `url` is a
URL and whose `limited` became `stop`.
This commit is contained in:
Jorijn van der Graaf 2026-10-31 20:39:43 +01:00
commit 7ee938c98e
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
10 changed files with 596 additions and 372 deletions
Download patch file Download diff file Expand all files Collapse all files

106
tests/ShouldFollowRedirectsHTTP1/main.cpp
View file

@ -1,20 +1,22 @@
//SPDX-License-Identifier: LGPL-3.0-only
//SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
// FetchHTTP1 follows redirects the way Python's urllib does, so code ported
// from urllib keeps its behaviour:
// - a relative Location is resolved against the URL that sent it (urljoin,
// `..` included), and a redirect may move from http:// to https://;
// - 301/302/303/307/308 are followed for a GET; a POST follows 301/302/303
// as a GET without its body, and comes back with a 307 or 308 unfollowed;
// - a redirect without a Location comes back as the response;
// - the chain stops at the 5th redirect to one URL (4 repeats) or after 10
// different URLs, returning that redirect with `limited` set;
// - a Location with a space is percent-encoded, as urllib quotes it;
// ClientHTTP1::Fetch follows redirects by RFC 9110 §15.4:
// - a relative Location is resolved against the URL that sent it (RFC
// 3986, `..` included), and a redirect may move from http:// to https://;
// - 301, 302, 303, 307 and 308 are followed for every method: 303 turns
// the request into a GET without its body, 301 and 302 do so for a POST
// alone, 307 and 308 keep the method and the body;
// - a redirect without a Location, or to another scheme, comes back as the
// response, with `stop` saying why;
// - a chain stops at a URL it has already been sent to, or after
// maxRedirects, returning that redirect;
// - Authorization and Cookie are not carried to another origin;
// - a URL that is not http(s), has no host or has a bad port is refused.
//
// Two listeners: plaintext on 8130, TLS on 8131 with a self-signed certificate
// the client is handed as its trust anchor.
// the client is handed as its trust anchor. The cross-origin case goes from
// 127.0.0.1 to localhost, the same listener under another name.
#include <cstdlib>
@ -43,12 +45,20 @@ std::unordered_map<std::string, std::function<HTTPResponse(const HTTPRequest&)>>
routes["/to-https"] = [](const HTTPRequest&) { return Redirect("307", "https://localhost:8131/secure"); };
routes["/loop"] = [](const HTTPRequest&) { return Redirect("302", "/loop"); };
routes["/nowhere"] = [](const HTTPRequest&) { return CreateResponseHTTP("302", std::string("no location")); };
routes["/elsewhere"] = [](const HTTPRequest&) { return Redirect("302", "ftp://127.0.0.1/file"); };
routes["/form"] = [](const HTTPRequest&) { return Redirect("303", "/method"); };
routes["/found"] = [](const HTTPRequest&) { return Redirect("302", "/method"); };
routes["/keep"] = [](const HTTPRequest&) { return Redirect("307", "/method"); };
routes["/method"] = [](const HTTPRequest& request) {
return CreateResponseHTTP("200", request.method + " body=" + request.body);
return CreateResponseHTTP("200", request.method + " body=" + request.body + " type=" + (request.headers.contains("content-type") ? request.headers.at("content-type") : "none"));
};
routes["/auth-away"] = [](const HTTPRequest&) { return Redirect("302", "http://localhost:8130/show-auth"); };
routes["/auth-home"] = [](const HTTPRequest&) { return Redirect("302", "/show-auth"); };
routes["/show-auth"] = [](const HTTPRequest& request) {
return CreateResponseHTTP("200", std::string("authorization=") + (request.headers.contains("authorization") ? "yes" : "no")
+ " cookie=" + (request.headers.contains("cookie") ? "yes" : "no")
+ " accept=" + (request.headers.contains("accept") ? "yes" : "no"));
};
routes["/spaced"] = [](const HTTPRequest&) { return Redirect("302", "/with space"); };
return routes;
}
@ -63,7 +73,7 @@ HTTPResponse Fallback(const HTTPRequest& request) {
bool Refused(std::string_view url) {
try {
FetchHTTP1(url);
ClientHTTP1::Fetch(url);
} catch (const std::invalid_argument&) {
return true;
} catch (const std::exception&) {
@ -83,61 +93,81 @@ int main() {
ListenerAsyncHTTP1 secure(8131, secureRoutes, TLSServerCredentials { .selfSigned = true });
{
const FetchResultHTTP1 result = FetchHTTP1("http://127.0.0.1:8130/start");
const FetchResult result = ClientHTTP1::Fetch("http://127.0.0.1:8130/start");
Check(result.response.status == "200" && result.response.body == "/final?x=1",
std::format("a relative chain arrives at /final?x=1 (got {} '{}')", result.response.status, result.response.body));
Check(result.url == "http://127.0.0.1:8130/final?x=1", std::format("the URL that answered is reported ({})", result.url));
Check(result.redirects == 2 && !result.limited, std::format("two redirects, not limited (got {})", result.redirects));
Check(result.url.ToString() == "http://127.0.0.1:8130/final?x=1", std::format("the URL that answered is reported ({})", result.url.ToString()));
Check(result.redirects == 2 && result.stop == FetchStop::None, std::format("two redirects, the chain ended on a final response (got {})", result.redirects));
}
{
const FetchResultHTTP1 result = FetchHTTP1("http://127.0.0.1:8130/a/b/start");
const FetchResult result = ClientHTTP1::Fetch("http://127.0.0.1:8130/a/b/start");
Check(result.response.body == "/a/c/d", std::format("../c/d from /a/b/start resolves to /a/c/d (got '{}')", result.response.body));
}
{
FetchOptionsHTTP1 options;
FetchOptions options;
options.credentials.caPem = GetSelfSignedCertificatePem().certificate;
const FetchResultHTTP1 result = FetchHTTP1("http://127.0.0.1:8130/to-https", options);
const FetchResult result = ClientHTTP1::Fetch("http://127.0.0.1:8130/to-https", options);
Check(result.response.status == "200" && result.response.body == "secure over https",
std::format("a redirect from http:// to https:// is followed (got {} '{}')", result.response.status, result.response.body));
Check(result.url == "https://localhost:8131/secure", std::format("and lands on the https:// URL ({})", result.url));
Check(result.url.ToString() == "https://localhost:8131/secure" && result.url.Secure(), std::format("and lands on the https:// URL ({})", result.url.ToString()));
}
{
const FetchResultHTTP1 result = FetchHTTP1("http://127.0.0.1:8130/loop");
Check(result.limited && result.response.status == "302" && result.redirects == 4,
std::format("a redirect loop stops after 4 repeats (limited {}, {} redirects)", result.limited, result.redirects));
const FetchResult result = ClientHTTP1::Fetch("http://127.0.0.1:8130/loop");
Check(result.stop == FetchStop::Loop && result.response.status == "302" && result.redirects == 0,
std::format("a redirect back to a URL already visited stops the chain at once (stop {}, {} redirects)", static_cast<int>(result.stop), result.redirects));
}
{
const FetchResultHTTP1 result = FetchHTTP1("http://127.0.0.1:8130/hop/0");
Check(result.limited && result.redirects == 10 && result.url == "http://127.0.0.1:8130/hop/10",
std::format("an endless chain stops after 10 different URLs ({} redirects, at {})", result.redirects, result.url));
const FetchResult result = ClientHTTP1::Fetch("http://127.0.0.1:8130/hop/0");
Check(result.stop == FetchStop::Limit && result.redirects == 10 && result.url.ToString() == "http://127.0.0.1:8130/hop/10",
std::format("an endless chain stops after maxRedirects, 10 ({} redirects, at {})", result.redirects, result.url.ToString()));
FetchOptions two;
two.maxRedirects = 2;
const FetchResult shorter = ClientHTTP1::Fetch("http://127.0.0.1:8130/hop/0", two);
Check(shorter.stop == FetchStop::Limit && shorter.redirects == 2, "maxRedirects is the option's");
}
{
const FetchResultHTTP1 result = FetchHTTP1("http://127.0.0.1:8130/nowhere");
Check(result.response.status == "302" && result.redirects == 0 && !result.limited,
const FetchResult result = ClientHTTP1::Fetch("http://127.0.0.1:8130/nowhere");
Check(result.stop == FetchStop::NoLocation && result.response.status == "302" && result.redirects == 0,
"a redirect without a Location comes back as the response");
const FetchResult other = ClientHTTP1::Fetch("http://127.0.0.1:8130/elsewhere");
Check(other.stop == FetchStop::OtherScheme && other.response.status == "302", "a redirect to another scheme comes back as the response");
}
{
FetchOptionsHTTP1 post;
FetchOptions post;
post.method = "POST";
post.body = "payload";
post.headers["content-type"] = "text/plain";
const FetchResultHTTP1 seeOther = FetchHTTP1("http://127.0.0.1:8130/form", post);
Check(seeOther.response.body == "GET body=",
std::format("a POST answered with 303 is followed as a GET without its body (got '{}')", seeOther.response.body));
const FetchResultHTTP1 kept = FetchHTTP1("http://127.0.0.1:8130/keep", post);
Check(kept.response.status == "307" && kept.redirects == 0, "a POST answered with 307 comes back unfollowed");
const FetchResult seeOther = ClientHTTP1::Fetch("http://127.0.0.1:8130/form", post);
Check(seeOther.response.body == "GET body= type=none",
std::format("a POST answered with 303 is followed as a GET without its body or content headers (got '{}')", seeOther.response.body));
const FetchResult found = ClientHTTP1::Fetch("http://127.0.0.1:8130/found", post);
Check(found.response.body == "GET body= type=none", std::format("a POST answered with 302 is followed as a GET (got '{}')", found.response.body));
const FetchResult kept = ClientHTTP1::Fetch("http://127.0.0.1:8130/keep", post);
Check(kept.response.body == "POST body=payload type=text/plain" && kept.redirects == 1,
std::format("a POST answered with 307 is followed with its method, body and content headers (got '{}')", kept.response.body));
FetchOptions put = post;
put.method = "PUT";
const FetchResult putFound = ClientHTTP1::Fetch("http://127.0.0.1:8130/found", put);
Check(putFound.response.body == "PUT body=payload type=text/plain", std::format("a PUT answered with 302 keeps its method and body (got '{}')", putFound.response.body));
}
{
const FetchResultHTTP1 result = FetchHTTP1("http://127.0.0.1:8130/spaced");
Check(result.response.body == "/with%20space",
std::format("a Location with a space is percent-encoded before it is followed (got '{}')", result.response.body));
FetchOptions options;
options.headers["authorization"] = "Bearer secret";
options.headers["cookie"] = "a=b";
options.headers["accept"] = "text/plain";
const FetchResult away = ClientHTTP1::Fetch("http://127.0.0.1:8130/auth-away", options);
Check(away.response.body == "authorization=no cookie=no accept=yes",
std::format("Authorization and Cookie are dropped on a redirect to another origin, other headers kept (got '{}')", away.response.body));
const FetchResult home = ClientHTTP1::Fetch("http://127.0.0.1:8130/auth-home", options);
Check(home.response.body == "authorization=yes cookie=yes accept=yes",
std::format("and kept on a redirect within the origin (got '{}')", home.response.body));
}
Check(Refused("ftp://127.0.0.1/file"), "an ftp:// URL is refused");