This commit is contained in:
parent
2fa6e70af1
commit
6841623e23
17 changed files with 2306 additions and 148 deletions
|
|
@ -27,12 +27,13 @@ jobs:
|
||||||
# run inside this archlinux container — the runner execs them with
|
# run inside this archlinux container — the runner execs them with
|
||||||
# node. This shell step needs no node, so installing it here (before
|
# node. This shell step needs no node, so installing it here (before
|
||||||
# Checkout) is enough.
|
# Checkout) is enough.
|
||||||
# ffmpeg is for ffprobe, which tools/fetch-media.sh uses to read the
|
# ffmpeg/ffprobe are both used by tools/fetch-media.sh. ffprobe reads
|
||||||
# pixel dimensions of each mirrored file. Those become the width/height
|
# the pixel dimensions of each mirrored file, which become the
|
||||||
# attributes that stop the posts page reflowing as 5 MB recordings
|
# width/height attributes that stop the posts page reflowing as 5 MB
|
||||||
# arrive, and tools/e2e.sh asserts they are present — so without this
|
# recordings arrive. ffmpeg transcodes each still image into the AVIF
|
||||||
# package the deploy fails at the e2e gate rather than shipping a
|
# and PNG renditions the page serves it between. tools/e2e.sh asserts
|
||||||
# janky page.
|
# both are present — so without this package the deploy fails at the
|
||||||
|
# e2e gate rather than shipping a janky page.
|
||||||
pacman -Syu --noconfirm --needed \
|
pacman -Syu --noconfirm --needed \
|
||||||
nodejs \
|
nodejs \
|
||||||
clang lld libc++ \
|
clang lld libc++ \
|
||||||
|
|
@ -84,18 +85,19 @@ jobs:
|
||||||
run: tools/fetch-rates.sh
|
run: tools/fetch-rates.sh
|
||||||
|
|
||||||
- name: Fetch fediverse posts
|
- name: Fetch fediverse posts
|
||||||
# Build-time, not run-time: the site embeds the owner's own posts and
|
# Build-time, not run-time: the site hosts the owner's own posts in full
|
||||||
# links out for discussion, so there is no sync service and no runtime
|
# and links out for the discussion, so there is no sync service and no
|
||||||
# dependency on the instance being up. The script leaves the committed
|
# runtime dependency on the instance being up. The script leaves the
|
||||||
# content/posts.json untouched and exits 0 on any failure, so a
|
# committed content/posts.json untouched and exits 0 on any failure, so a
|
||||||
# fediverse outage cannot fail a deploy.
|
# fediverse outage cannot fail a deploy.
|
||||||
run: tools/fetch-posts.sh
|
run: tools/fetch-posts.sh
|
||||||
|
|
||||||
- name: Mirror post media
|
- name: Mirror post media
|
||||||
# Downloads the images and screen recordings the posts carry and rewrites
|
# Downloads the images and screen recordings the posts carry — both the
|
||||||
# content/posts.json to point at our own copies, so nothing the browser
|
# headline file and everything embedded inside the body — and rewrites
|
||||||
# loads is third-party — which is what keeps the privacy notice's
|
# content/posts.json (including the body text itself) to point at our own
|
||||||
# "everything comes from catcrafts.net" true.
|
# copies, so nothing the browser loads is third-party. That is what keeps
|
||||||
|
# the privacy notice's "everything comes from catcrafts.net" true.
|
||||||
#
|
#
|
||||||
# Content-addressed: the name is the hash of the bytes. Note what that
|
# Content-addressed: the name is the hash of the bytes. Note what that
|
||||||
# does NOT mean — a third-party file is re-fetched on every build, because
|
# does NOT mean — a third-party file is re-fetched on every build, because
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -128,6 +128,62 @@ build. Neither fails the build on a network error — a fediverse outage leaves
|
||||||
previous `posts.json` in place, and a single failed download leaves that one entry
|
previous `posts.json` in place, and a single failed download leaves that one entry
|
||||||
pointing at its original URL rather than losing the post.
|
pointing at its original URL rather than losing the post.
|
||||||
|
|
||||||
|
### The full body, and its inline media
|
||||||
|
|
||||||
|
Each post is hosted whole at `/posts/<slug>`, with the card on `/posts` linking
|
||||||
|
to it — the writing is what the site is about, and a page this site can name as
|
||||||
|
canonical is the only version a search engine can be pointed at. The comments
|
||||||
|
are *not* mirrored: every post page links out to the thread, which is where the
|
||||||
|
discussion belongs.
|
||||||
|
|
||||||
|
The body stays Markdown in `posts.json` and is rendered by
|
||||||
|
`Catcrafts.Shared:Markdown` at page-render time, never converted to HTML by the
|
||||||
|
shell. That is deliberate: the renderer is inside the escaping guarantee, and
|
||||||
|
text fetched from someone else's server must not be able to become markup
|
||||||
|
anywhere else. Raw HTML in a body is always shown as text.
|
||||||
|
|
||||||
|
`fetch-media.sh` mirrors what the body embeds as well as the headline file, and
|
||||||
|
rewrites the URLs **inside the Markdown**, so a post page loads nothing
|
||||||
|
third-party either. It also writes a `body_media` list per post — dimensions,
|
||||||
|
poster frame and format renditions for each inline file, which Markdown syntax
|
||||||
|
has nowhere to carry. Slugs come from the title; a duplicate title takes the
|
||||||
|
post's numeric id as a suffix, so an old post's URL is never renumbered by a new
|
||||||
|
one. Re-running the script is a no-op: local paths are adopted from the mount
|
||||||
|
rather than re-fetched.
|
||||||
|
|
||||||
|
### The image format ladder
|
||||||
|
|
||||||
|
Every mirrored still image is transcoded to two siblings named after its content
|
||||||
|
hash, and `Catcrafts.Shared:Media` serves all three as one `<picture>` — so the
|
||||||
|
browser fetches **exactly one**:
|
||||||
|
|
||||||
|
| tier | file | size vs. WebP | who gets it |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `<source type="image/avif">` | `<hash>.avif` | **76%** | almost everyone |
|
||||||
|
| `<source type="image/webp">` | `<hash>.webp` (the mirrored original) | 100% | Safari 14–16 |
|
||||||
|
| `<img src>` | `<hash>.png` | **875%** | neither of the above |
|
||||||
|
|
||||||
|
The middle tier is why the PNG being ~9× the WebP does not matter: it is free
|
||||||
|
(the mirror already downloaded that file) and it is what the small number of
|
||||||
|
non-AVIF browsers actually land on. The PNG is the floor nothing can refuse.
|
||||||
|
|
||||||
|
AVIF is encoded at `crf 26, cpu-used 6, yuv444p` — measured at SSIM 0.997
|
||||||
|
against the source and still smaller than it. Full chroma is deliberate: these
|
||||||
|
are screenshots of text, and re-subsampling chroma that pict-rs already
|
||||||
|
subsampled once fringes coloured text visibly, for about 3% more bytes.
|
||||||
|
|
||||||
|
Encoding is skipped when the sibling is already on the mount, so only genuinely
|
||||||
|
new images cost encoder time (~0.5 s each). Animated sources are left alone
|
||||||
|
entirely — one moving GIF beats three copies of its first frame. Video posters
|
||||||
|
are skipped too: `poster` takes exactly one URL, so a `<video>` cannot negotiate
|
||||||
|
a format the way `<picture>` can and the renditions would be unreachable.
|
||||||
|
|
||||||
|
Both encodes pin `-c:v` and then **verify the codec that actually came out**.
|
||||||
|
That check earned its place immediately: `-f image2 out.png` without an explicit
|
||||||
|
codec makes ffmpeg fall back to the muxer default, which is MJPEG — it silently
|
||||||
|
produced a full set of lossy JPEGs under `.png` names, served to browsers as
|
||||||
|
`image/png`. A rendition that fails the check is discarded and its tier dropped.
|
||||||
|
|
||||||
### Publish the media first, then post it
|
### Publish the media first, then post it
|
||||||
|
|
||||||
**The recommended flow is to put a recording on catcrafts.net before writing the
|
**The recommended flow is to put a recording on catcrafts.net before writing the
|
||||||
|
|
|
||||||
|
|
@ -177,8 +177,7 @@ namespace Catcrafts {
|
||||||
// restarting at cc-link-0 and colliding with the content links —
|
// restarting at cc-link-0 and colliding with the content links —
|
||||||
// getElementById would then return whichever appeared first in the
|
// getElementById would then return whichever appeared first in the
|
||||||
// document and half the links would navigate to the wrong place.
|
// document and half the links would navigate to the wrong place.
|
||||||
const RouteKind navKind =
|
const RouteKind navKind = NavKindFor(route.kind);
|
||||||
route.kind == RouteKind::LegacyBlog ? RouteKind::Posts : route.kind;
|
|
||||||
Dom::HtmlElementPtr header("cc-header");
|
Dom::HtmlElementPtr header("cc-header");
|
||||||
if (header.ptr != 0) {
|
if (header.ptr != 0) {
|
||||||
header.SetInnerHTML(TagLinks(Views::RenderNav(navKind).View(), linkTargets));
|
header.SetInnerHTML(TagLinks(Views::RenderNav(navKind).View(), linkTargets));
|
||||||
|
|
|
||||||
|
|
@ -45,12 +45,14 @@ static Configuration* SharedLibrary(std::span<const std::string_view> args) {
|
||||||
ApplyStandardArgs(*shared, args); // inherits --target / --debug from the parent
|
ApplyStandardArgs(*shared, args); // inherits --target / --debug from the parent
|
||||||
shared->type = ConfigurationType::LibraryStatic;
|
shared->type = ConfigurationType::LibraryStatic;
|
||||||
|
|
||||||
std::array<fs::path, 9> ifaces = {
|
std::array<fs::path, 11> ifaces = {
|
||||||
"shared/interfaces/Catcrafts.Shared",
|
"shared/interfaces/Catcrafts.Shared",
|
||||||
"shared/interfaces/Catcrafts.Shared-Html",
|
"shared/interfaces/Catcrafts.Shared-Html",
|
||||||
"shared/interfaces/Catcrafts.Shared-Form",
|
"shared/interfaces/Catcrafts.Shared-Form",
|
||||||
"shared/interfaces/Catcrafts.Shared-Json",
|
"shared/interfaces/Catcrafts.Shared-Json",
|
||||||
"shared/interfaces/Catcrafts.Shared-Model",
|
"shared/interfaces/Catcrafts.Shared-Model",
|
||||||
|
"shared/interfaces/Catcrafts.Shared-Media",
|
||||||
|
"shared/interfaces/Catcrafts.Shared-Markdown",
|
||||||
"shared/interfaces/Catcrafts.Shared-Content",
|
"shared/interfaces/Catcrafts.Shared-Content",
|
||||||
"shared/interfaces/Catcrafts.Shared-Money",
|
"shared/interfaces/Catcrafts.Shared-Money",
|
||||||
"shared/interfaces/Catcrafts.Shared-Route",
|
"shared/interfaces/Catcrafts.Shared-Route",
|
||||||
|
|
|
||||||
|
|
@ -287,8 +287,7 @@ HTTPResponse RenderPage(std::string_view target) {
|
||||||
if (const Demo* d = gContent.FindDemo(route.slug)) wantsWasm = d->needsWasm;
|
if (const Demo* d = gContent.FindDemo(route.slug)) wantsWasm = d->needsWasm;
|
||||||
}
|
}
|
||||||
res.body = Views::RenderDocument(page,
|
res.body = Views::RenderDocument(page,
|
||||||
Views::RenderNav(route.kind == RouteKind::LegacyBlog
|
Views::RenderNav(NavKindFor(route.kind)),
|
||||||
? RouteKind::Posts : route.kind),
|
|
||||||
Views::RenderFooter(),
|
Views::RenderFooter(),
|
||||||
wantsWasm ? gBootScripts : std::string_view{},
|
wantsWasm ? gBootScripts : std::string_view{},
|
||||||
gCssHref);
|
gCssHref);
|
||||||
|
|
@ -310,6 +309,15 @@ HTTPResponse ServeSitemap() {
|
||||||
out += Html::Escape(pr.slug).Str();
|
out += Html::Escape(pr.slug).Str();
|
||||||
out += "</loc></url>\n";
|
out += "</loc></url>\n";
|
||||||
}
|
}
|
||||||
|
// Post pages, from the same HasPage() test the "read more" links use — a
|
||||||
|
// sitemap that advertised a post without a body would be pointing crawlers
|
||||||
|
// at the 404 the dispatcher correctly returns for it.
|
||||||
|
for (const Post& po : gContent.posts) {
|
||||||
|
if (!po.HasPage()) continue;
|
||||||
|
out += " <url><loc>https://catcrafts.net/posts/";
|
||||||
|
out += Html::Escape(po.slug).Str();
|
||||||
|
out += "</loc></url>\n";
|
||||||
|
}
|
||||||
out += "</urlset>\n";
|
out += "</urlset>\n";
|
||||||
ApplyPageHeaders(res, "application/xml; charset=utf-8", true, false);
|
ApplyPageHeaders(res, "application/xml; charset=utf-8", true, false);
|
||||||
res.body = std::move(out);
|
res.body = std::move(out);
|
||||||
|
|
|
||||||
|
|
@ -109,6 +109,394 @@ void RunSelfTest() {
|
||||||
CheckEq(Escape("a") + Escape("<"), "a<", "operator+: escapes preserved");
|
CheckEq(Escape("a") + Escape("<"), "a<", "operator+: escapes preserved");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The format ladder. One <picture>/<video> builder serves both the cards and
|
||||||
|
// the post bodies, so these assertions cover every image and video the site
|
||||||
|
// emits — and the ordering ones matter: a browser takes the FIRST source it
|
||||||
|
// understands, so a mis-ordered ladder silently serves the wrong tier to
|
||||||
|
// everyone rather than failing visibly.
|
||||||
|
void RunMediaSelfTest() {
|
||||||
|
auto img = [](std::string src, std::string avif, std::string png,
|
||||||
|
std::int64_t w = 0, std::int64_t h = 0) {
|
||||||
|
PostMedia m;
|
||||||
|
m.src = std::move(src);
|
||||||
|
m.kind = "image";
|
||||||
|
m.avif = std::move(avif);
|
||||||
|
m.fallback = std::move(png);
|
||||||
|
m.width = w;
|
||||||
|
m.height = h;
|
||||||
|
return m;
|
||||||
|
};
|
||||||
|
|
||||||
|
// The full ladder: AVIF, then the mirrored original, then the PNG the <img>
|
||||||
|
// itself points at. Exactly one of the three is ever fetched.
|
||||||
|
CheckEq(Media::Tag(img("/media/x.webp", "/media/x.avif", "/media/x.png", 800, 600)),
|
||||||
|
R"(<picture><source srcset="/media/x.avif" type="image/avif">)"
|
||||||
|
R"(<source srcset="/media/x.webp" type="image/webp">)"
|
||||||
|
R"(<img class="post-media__item" loading="lazy" decoding="async" alt="")"
|
||||||
|
R"( src="/media/x.png" width="800" height="600"></picture>)",
|
||||||
|
"media: image ladder is avif, original, png");
|
||||||
|
|
||||||
|
// Alt text reaches the <img>, not the <picture> — a screen reader reads the
|
||||||
|
// img, and an alt on the wrapper is invisible to it.
|
||||||
|
Check(Media::Tag(img("/media/x.webp", "/media/x.avif", "/media/x.png"), "a cat")
|
||||||
|
.View().find(R"(alt="a cat" src="/media/x.png")") != std::string_view::npos,
|
||||||
|
"media: alt lands on the img");
|
||||||
|
|
||||||
|
// Degradation, one tier at a time. Each of these is a real state: no
|
||||||
|
// encoder on the build host, a source that was already PNG, a body image
|
||||||
|
// whose download failed so there is nothing but the original URL.
|
||||||
|
CheckEq(Media::Tag(img("/media/x.webp", "", "/media/x.png")),
|
||||||
|
R"(<picture><source srcset="/media/x.webp" type="image/webp">)"
|
||||||
|
R"(<img class="post-media__item" loading="lazy" decoding="async" alt="")"
|
||||||
|
R"( src="/media/x.png"></picture>)",
|
||||||
|
"media: no avif still offers the original above the png");
|
||||||
|
CheckEq(Media::Tag(img("/media/x.webp", "/media/x.avif", "")),
|
||||||
|
R"(<picture><source srcset="/media/x.avif" type="image/avif">)"
|
||||||
|
R"(<img class="post-media__item" loading="lazy" decoding="async" alt="")"
|
||||||
|
R"( src="/media/x.webp"></picture>)",
|
||||||
|
"media: no png leaves the original as the base");
|
||||||
|
CheckEq(Media::Tag(img("/media/x.webp", "", "")),
|
||||||
|
R"(<img class="post-media__item" loading="lazy" decoding="async" alt="")"
|
||||||
|
R"( src="/media/x.webp">)",
|
||||||
|
"media: no renditions is a bare img, as before any of this existed");
|
||||||
|
// A source that is already PNG is its own fallback, and must not be
|
||||||
|
// offered twice — once as a <source> and once as the <img>.
|
||||||
|
CheckEq(Media::Tag(img("/media/x.png", "/media/x.avif", "/media/x.png")),
|
||||||
|
R"(<picture><source srcset="/media/x.avif" type="image/avif">)"
|
||||||
|
R"(<img class="post-media__item" loading="lazy" decoding="async" alt="")"
|
||||||
|
R"( src="/media/x.png"></picture>)",
|
||||||
|
"media: a png source is not also listed as a source");
|
||||||
|
// Likewise a source that is already AVIF.
|
||||||
|
Check(Media::Tag(img("/media/x.avif", "/media/x.avif", "/media/x.png"))
|
||||||
|
.View().find("image/avif\"><source") == std::string_view::npos,
|
||||||
|
"media: an avif source is not listed twice");
|
||||||
|
|
||||||
|
// A URL is still a URL: the scheme allowlist applies to srcset exactly as
|
||||||
|
// it does to src, or the ladder becomes a way around it.
|
||||||
|
Check(Media::Tag(img("/media/x.webp", "javascript:alert(1)", "/media/x.png"))
|
||||||
|
.View().find(R"(srcset="#")") != std::string_view::npos,
|
||||||
|
"media: a hostile srcset is neutralised");
|
||||||
|
|
||||||
|
// Video is unchanged by any of this and must stay so.
|
||||||
|
{
|
||||||
|
PostMedia v;
|
||||||
|
v.src = "/media/v.mp4";
|
||||||
|
v.kind = "video";
|
||||||
|
v.poster = "/media/v.webp";
|
||||||
|
v.fallback = "/media/v.h264.mp4";
|
||||||
|
const auto out = Media::Tag(v);
|
||||||
|
Check(out.View().starts_with("<video class=\"post-media__item\" controls preload=\"metadata\""),
|
||||||
|
"media: video is still a video", out.View());
|
||||||
|
Check(out.View().find("codecs=av01") != std::string_view::npos
|
||||||
|
&& out.View().find(R"(<source src="/media/v.h264.mp4" type="video/mp4">)")
|
||||||
|
!= std::string_view::npos,
|
||||||
|
"media: AV1 then H.264, in that order");
|
||||||
|
Check(out.View().find("<picture>") == std::string_view::npos,
|
||||||
|
"media: a video is not wrapped in a picture");
|
||||||
|
}
|
||||||
|
|
||||||
|
// A path with no record renders from the path alone — the mirror failed,
|
||||||
|
// and showing the picture beats dropping the paragraph's subject.
|
||||||
|
{
|
||||||
|
const std::array<PostMedia, 1> known{
|
||||||
|
img("/media/x.webp", "/media/x.avif", "/media/x.png") };
|
||||||
|
Check(Media::Find(known, "/media/x.webp") != nullptr, "media: found by src");
|
||||||
|
Check(Media::Find(known, "/media/nope.webp") == nullptr, "media: unknown src");
|
||||||
|
const PostMedia guessed = Media::Describe(known, "https://i.example/a.webp");
|
||||||
|
Check(guessed.kind == "image" && guessed.avif.empty(),
|
||||||
|
"media: an unmirrored image is described from its path");
|
||||||
|
Check(Media::Describe(known, "https://i.example/a.mp4").kind == "video",
|
||||||
|
"media: an unmirrored video is recognised as one");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The Markdown renderer, which is the newest place untrusted text becomes
|
||||||
|
// markup — post bodies are fetched from someone else's server, so every one of
|
||||||
|
// these assertions is ultimately about the same thing: nothing in a body can
|
||||||
|
// escape into the document. The structural cases are here too, because a parser
|
||||||
|
// that silently drops a construct loses content invisibly.
|
||||||
|
void RunMarkdownSelfTest() {
|
||||||
|
auto md = [](std::string_view text,
|
||||||
|
std::span<const PostMedia> media = {}) {
|
||||||
|
return Markdown::Render(text, media);
|
||||||
|
};
|
||||||
|
|
||||||
|
// ── the guarantee ─────────────────────────────────────────────────
|
||||||
|
CheckEq(md("<script>alert(1)</script>"),
|
||||||
|
"<p><script>alert(1)</script></p>", "md: html is text, never markup");
|
||||||
|
CheckEq(md(")"),
|
||||||
|
R"(<div class="post-media"><img class="post-media__item" loading="lazy" )"
|
||||||
|
R"(decoding="async" alt="x" src="#"></div>)",
|
||||||
|
"md: javascript: image source neutralised");
|
||||||
|
CheckEq(md("[x](javascript:alert(1))"),
|
||||||
|
R"(<p><a href="#" rel="noopener">x</a></p>)",
|
||||||
|
"md: javascript: link neutralised");
|
||||||
|
CheckEq(md("a \" b & c"), "<p>a " b & c</p>", "md: quotes and ampersands escaped");
|
||||||
|
// A code span is verbatim text, and verbatim is exactly where an escaper
|
||||||
|
// is most often forgotten.
|
||||||
|
CheckEq(md("`<b>`"), "<p><code><b></code></p>", "md: code span escaped");
|
||||||
|
|
||||||
|
// ── blocks ────────────────────────────────────────────────────────
|
||||||
|
CheckEq(md(""), "", "md: empty body renders nothing");
|
||||||
|
CheckEq(md("plain text"), "<p>plain text</p>", "md: paragraph");
|
||||||
|
// Demotion by one: the page h1 is the post title, so a body's own top-level
|
||||||
|
// heading is a section within it.
|
||||||
|
CheckEq(md("# Heading"), "<h2>Heading</h2>", "md: h1 demoted to h2");
|
||||||
|
CheckEq(md("### Heading"), "<h4>Heading</h4>", "md: h3 demoted to h4");
|
||||||
|
CheckEq(md("#nothashtag"), "<p>#nothashtag</p>", "md: # without a space is not a heading");
|
||||||
|
CheckEq(md("> quoted"),
|
||||||
|
R"(<blockquote class="post-body__quote"><p>quoted</p></blockquote>)",
|
||||||
|
"md: blockquote");
|
||||||
|
// The quoted lines are re-parsed, so a multi-paragraph quote keeps its
|
||||||
|
// paragraphs instead of collapsing into one run-on line.
|
||||||
|
CheckEq(md("> one\n>\n> two"),
|
||||||
|
R"(<blockquote class="post-body__quote"><p>one</p><p>two</p></blockquote>)",
|
||||||
|
"md: blockquote keeps its paragraphs");
|
||||||
|
CheckEq(md("- a\n- b"),
|
||||||
|
R"(<ul class="post-body__list"><li>a</li><li>b</li></ul>)", "md: unordered list");
|
||||||
|
CheckEq(md("1. a\n2. b"),
|
||||||
|
R"(<ol class="post-body__list"><li>a</li><li>b</li></ol>)", "md: ordered list");
|
||||||
|
// A list resumed after an interrupting paragraph continues its numbering.
|
||||||
|
// Without the start attribute the mini-guide in one of these posts renders
|
||||||
|
// as steps 1-4 followed by steps 1, 2, 3.
|
||||||
|
CheckEq(md("5. e"),
|
||||||
|
R"(<ol class="post-body__list" start="5"><li>e</li></ol>)",
|
||||||
|
"md: ordered list keeps the number it announced");
|
||||||
|
// Blank lines between items are spacing, not seven one-item lists.
|
||||||
|
CheckEq(md("1. a\n\n2. b"),
|
||||||
|
R"(<ol class="post-body__list"><li>a</li><li>b</li></ol>)",
|
||||||
|
"md: blank line inside a list does not split it");
|
||||||
|
CheckEq(md("---"), "<hr>", "md: thematic break");
|
||||||
|
CheckEq(md("- - -"), "<hr>", "md: spaced rule is not a one-item list");
|
||||||
|
// Whitespace in pasted terminal output is the content.
|
||||||
|
CheckEq(md("```\n a\tb\n```"),
|
||||||
|
"<pre class=\"post-body__code\"><code> a\tb\n</code></pre>",
|
||||||
|
"md: fenced code is verbatim");
|
||||||
|
// An unterminated fence must not swallow the document into nothing.
|
||||||
|
Check(md("```\nx").View().find("<code>x") != std::string_view::npos,
|
||||||
|
"md: unterminated fence still renders its content");
|
||||||
|
|
||||||
|
// ── inline ────────────────────────────────────────────────────────
|
||||||
|
CheckEq(md("**bold**"), "<p><strong>bold</strong></p>", "md: strong");
|
||||||
|
CheckEq(md("*em*"), "<p><em>em</em></p>", "md: emphasis");
|
||||||
|
CheckEq(md("2 * 3 * 4"), "<p>2 * 3 * 4</p>", "md: spaced asterisks stay literal");
|
||||||
|
// Underscores are deliberately inert: these posts paste kernel symbol
|
||||||
|
// names into prose, and italicising half of one is worse than not
|
||||||
|
// italicising a word that used the underscore form.
|
||||||
|
CheckEq(md("kworker/u16:8-qc_ufs_qos_swq"),
|
||||||
|
"<p>kworker/u16:8-qc_ufs_qos_swq</p>", "md: underscores are not emphasis");
|
||||||
|
CheckEq(md("\\*literal\\*"), "<p>*literal*</p>", "md: backslash escape");
|
||||||
|
CheckEq(md("[label](https://x.example/y)"),
|
||||||
|
R"(<p><a href="https://x.example/y" rel="noopener">label</a></p>)", "md: link");
|
||||||
|
// Bare addresses are pasted constantly in these posts; leaving them inert
|
||||||
|
// would strip most of the outbound value out of the page.
|
||||||
|
CheckEq(md("see https://x.example/y"),
|
||||||
|
R"(<p>see <a href="https://x.example/y">https://x.example/y</a></p>)",
|
||||||
|
"md: bare URL autolinked");
|
||||||
|
|
||||||
|
// ── embedded media ────────────────────────────────────────────────
|
||||||
|
// A paragraph that is nothing but images becomes the same media block the
|
||||||
|
// cards use, rather than a <p> of pictures.
|
||||||
|
CheckEq(md(""),
|
||||||
|
R"(<div class="post-media"><img class="post-media__item" loading="lazy" )"
|
||||||
|
R"(decoding="async" alt="a" src="/media/x.webp"></div>)",
|
||||||
|
"md: image-only paragraph is a media block");
|
||||||
|
Check(md("text ").View().starts_with("<p>text <img"),
|
||||||
|
"md: an image inside a sentence stays inline");
|
||||||
|
|
||||||
|
// Dimensions come from the sidecar list, because Markdown syntax has
|
||||||
|
// nowhere to carry them — and without them the prose below every
|
||||||
|
// screenshot jumps as the file arrives.
|
||||||
|
{
|
||||||
|
std::vector<PostMedia> media;
|
||||||
|
PostMedia img;
|
||||||
|
img.src = "/media/x.webp";
|
||||||
|
img.kind = "image";
|
||||||
|
img.avif = "/media/x.avif";
|
||||||
|
img.fallback = "/media/x.png";
|
||||||
|
img.width = 800;
|
||||||
|
img.height = 600;
|
||||||
|
media.push_back(img);
|
||||||
|
|
||||||
|
PostMedia vid;
|
||||||
|
vid.src = "/media/v.mp4";
|
||||||
|
vid.kind = "video";
|
||||||
|
vid.poster = "/media/v.poster.webp";
|
||||||
|
vid.fallback = "/media/v.h264.mp4";
|
||||||
|
vid.width = 1080;
|
||||||
|
vid.height = 1920;
|
||||||
|
media.push_back(vid);
|
||||||
|
|
||||||
|
const auto out = md("", media);
|
||||||
|
Check(out.View().find(R"(width="800" height="600")") != std::string_view::npos,
|
||||||
|
"md: inline image carries its dimensions", out.View());
|
||||||
|
// Routed through :Media, so a body image gets the same format ladder a
|
||||||
|
// card image does rather than a second, plainer implementation.
|
||||||
|
Check(out.View().find(R"(<source srcset="/media/x.avif" type="image/avif">)")
|
||||||
|
!= std::string_view::npos
|
||||||
|
&& out.View().find(R"(src="/media/x.png")") != std::string_view::npos,
|
||||||
|
"md: inline image gets the avif/png ladder", out.View());
|
||||||
|
|
||||||
|
// An inline video gets the same treatment a headline one does,
|
||||||
|
// fallback source and all.
|
||||||
|
const auto vout = md("", media);
|
||||||
|
Check(vout.View().find(R"(poster="/media/v.poster.webp")") != std::string_view::npos
|
||||||
|
&& vout.View().find("codecs=av01") != std::string_view::npos
|
||||||
|
&& vout.View().find(R"(<source src="/media/v.h264.mp4")") != std::string_view::npos,
|
||||||
|
"md: inline video gets poster and H.264 fallback", vout.View());
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── termination ───────────────────────────────────────────────────
|
||||||
|
// Unbalanced delimiters are the classic way to hang a hand-written
|
||||||
|
// parser, and a body is input from someone else's server.
|
||||||
|
Check(!md("**unclosed").View().empty(), "md: unclosed strong terminates");
|
||||||
|
Check(!md("[unclosed](").View().empty(), "md: unclosed link terminates");
|
||||||
|
Check(!md(".View().empty(), "md: unclosed image terminates");
|
||||||
|
Check(!md("`unclosed").View().empty(), "md: unclosed code span terminates");
|
||||||
|
Check(!md("> > > > > > > > deep").View().empty(), "md: over-deep nesting terminates");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Post pages: the routing, the loader's guard on what becomes a URL, and the
|
||||||
|
// schema.org joins that keep every post attributed to the one Organization and
|
||||||
|
// the one Person the rest of the site describes.
|
||||||
|
void RunPostSelfTest() {
|
||||||
|
// ── routing ───────────────────────────────────────────────────────
|
||||||
|
Check(ParseRoute("/posts").kind == RouteKind::Posts, "route: /posts is the list");
|
||||||
|
Check(ParseRoute("/posts/hello-world").kind == RouteKind::Post, "route: /posts/<slug>");
|
||||||
|
Check(ParseRoute("/posts/hello-world").slug == "hello-world", "route: post slug captured");
|
||||||
|
Check(ParseRoute("/posts/hello-world/").kind == RouteKind::Post,
|
||||||
|
"route: trailing slash normalised");
|
||||||
|
Check(ParseRoute("/posts/Hello").kind == RouteKind::NotFound,
|
||||||
|
"route: uppercase slug is not a post URL");
|
||||||
|
Check(ParseRoute("/posts/../etc").kind == RouteKind::NotFound,
|
||||||
|
"route: traversal never reaches a lookup");
|
||||||
|
Check(NavKindFor(RouteKind::Post) == RouteKind::Posts,
|
||||||
|
"route: a post page highlights the Posts nav entry");
|
||||||
|
Check(NavKindFor(RouteKind::LegacyBlog) == RouteKind::Posts,
|
||||||
|
"route: the retired /blog URL highlights it too");
|
||||||
|
Check(NavKindFor(RouteKind::Shop) == RouteKind::Shop, "route: a nav route is its own entry");
|
||||||
|
|
||||||
|
// ── the loader ────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
const auto posts = LoadPosts(R"([
|
||||||
|
{"title":"Good","slug":"good-post","permalink":"https://i.example/post/1",
|
||||||
|
"body":"Hello.","published":"2026-01-01T00:00:00Z",
|
||||||
|
"body_media":[{"src":"/media/a.webp","kind":"image","w":10,"h":20}]},
|
||||||
|
{"title":"Bad slug","slug":"NOT A SLUG","permalink":"https://i.example/post/2",
|
||||||
|
"body":"Hello."},
|
||||||
|
{"title":"No body","slug":"no-body","permalink":"https://i.example/post/3"}
|
||||||
|
])");
|
||||||
|
Check(posts.size() == 3, "posts: all three load");
|
||||||
|
if (posts.size() == 3) {
|
||||||
|
Check(posts[0].HasPage() && posts[0].slug == "good-post", "posts: valid slug kept");
|
||||||
|
Check(posts[0].body == "Hello.", "posts: body loaded");
|
||||||
|
Check(posts[0].bodyMedia.size() == 1 && posts[0].bodyMedia[0].width == 10,
|
||||||
|
"posts: body media loaded with dimensions");
|
||||||
|
// A slug that could never match a route would render a "read more"
|
||||||
|
// link to a 404 this site points at itself.
|
||||||
|
Check(posts[1].slug.empty() && !posts[1].HasPage(),
|
||||||
|
"posts: malformed slug is dropped, costing the page");
|
||||||
|
// A title and a link out is not a page worth minting a URL for.
|
||||||
|
Check(!posts[2].HasPage(), "posts: no body means no page");
|
||||||
|
}
|
||||||
|
|
||||||
|
Views::SiteContent content;
|
||||||
|
content.posts = posts;
|
||||||
|
Check(content.FindPost("good-post") != nullptr, "posts: found by slug");
|
||||||
|
Check(content.FindPost("no-body") == nullptr, "posts: a pageless post is not findable");
|
||||||
|
Check(content.FindPost("nope") == nullptr, "posts: unknown slug is not found");
|
||||||
|
// Which means the route 404s rather than rendering an empty article.
|
||||||
|
Check(Views::RenderRoute(ParseRoute("/posts/no-body"), content).status == 404,
|
||||||
|
"posts: a pageless slug is a real 404");
|
||||||
|
Check(Views::RenderRoute(ParseRoute("/posts/good-post"), content).status == 200,
|
||||||
|
"posts: a real post renders");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── the page ──────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
Post p;
|
||||||
|
p.title = "Working GPS!";
|
||||||
|
p.slug = "working-gps";
|
||||||
|
p.permalink = "https://lemmy.example/post/42";
|
||||||
|
p.community = "linuxphones@lemmy.example";
|
||||||
|
p.published = "2026-06-26T23:16:25Z";
|
||||||
|
p.excerpt = "A short summary.";
|
||||||
|
p.body = "# How\n\nIt works.";
|
||||||
|
PostMedia m;
|
||||||
|
m.src = "/media/shot.webp";
|
||||||
|
m.kind = "image";
|
||||||
|
p.media.push_back(m);
|
||||||
|
|
||||||
|
const auto page = Views::RenderPost(p);
|
||||||
|
Check(page.meta.canonical == "/posts/working-gps",
|
||||||
|
"post page: canonical is this site, not the instance");
|
||||||
|
Check(page.meta.ogType == "article", "post page: og:type is article");
|
||||||
|
Check(page.meta.ogImage == "/media/shot.webp", "post page: og:image from the post media");
|
||||||
|
Check(page.meta.description == p.excerpt, "post page: description is the excerpt");
|
||||||
|
Check(page.main.View().find("<h2>How</h2>") != std::string_view::npos,
|
||||||
|
"post page: the body is rendered, not escaped away");
|
||||||
|
// The whole reason the body is hosted: the thread is still one click
|
||||||
|
// away, and the reader is told where the discussion is.
|
||||||
|
Check(page.main.View().find(p.permalink) != std::string_view::npos,
|
||||||
|
"post page: still links the thread");
|
||||||
|
|
||||||
|
// The identity graph, same joins every other page makes. A typo'd @id
|
||||||
|
// still renders and still validates — it just quietly splits this post
|
||||||
|
// away from the entity the rest of the site describes.
|
||||||
|
auto ld = Json::Parse(page.meta.jsonLd);
|
||||||
|
Check(ld && ld->IsObject() && ld->Str("@type") == "BlogPosting",
|
||||||
|
"post schema: parses as a BlogPosting");
|
||||||
|
if (ld && ld->IsObject()) {
|
||||||
|
Check(ld->Str("url") == "https://catcrafts.net/posts/working-gps",
|
||||||
|
"post schema: url is the on-site page");
|
||||||
|
Check(ld->Str("discussionUrl") == p.permalink,
|
||||||
|
"post schema: the thread is the discussion, not the content");
|
||||||
|
Check(ld->Str("datePublished") == p.published, "post schema: publication date");
|
||||||
|
const Json::Value* author = ld->Find("author");
|
||||||
|
const Json::Value* publisher = ld->Find("publisher");
|
||||||
|
Check(author && author->Str("@id") == "https://catcrafts.net/about#person",
|
||||||
|
"post schema: authored by the Person node on /about");
|
||||||
|
Check(publisher && publisher->Str("@id") == "https://catcrafts.net/#organization",
|
||||||
|
"post schema: published by the Organization node");
|
||||||
|
}
|
||||||
|
|
||||||
|
// A post with a page is advertised as one from the list and from home;
|
||||||
|
// one without keeps pointing at the thread, because there is nothing
|
||||||
|
// here to send the reader to.
|
||||||
|
const std::array<Post, 1> one{ p };
|
||||||
|
const auto list = Views::RenderPosts(one);
|
||||||
|
Check(list.main.View().find(R"(href="/posts/working-gps")") != std::string_view::npos,
|
||||||
|
"posts list: links the on-site page");
|
||||||
|
Check(list.main.View().find("Read the full post") != std::string_view::npos,
|
||||||
|
"posts list: offers the full post");
|
||||||
|
// Inside the excerpt paragraph, trailing the text — not a row of its
|
||||||
|
// own below the media, where it was the same offer made a screen
|
||||||
|
// further down.
|
||||||
|
Check(list.main.View().find(
|
||||||
|
R"(A short summary. <a class="link-more" href="/posts/working-gps">)"
|
||||||
|
R"(Read the full post</a></p>)") != std::string_view::npos,
|
||||||
|
"posts list: read-more trails the excerpt", list.main.View());
|
||||||
|
|
||||||
|
// With no excerpt there is no sentence to continue, so it falls back to
|
||||||
|
// a row rather than vanishing with the paragraph that would have held it.
|
||||||
|
Post unexcerpted = p;
|
||||||
|
unexcerpted.excerpt.clear();
|
||||||
|
const std::array<Post, 1> bare{ unexcerpted };
|
||||||
|
const auto listBare = Views::RenderPosts(bare);
|
||||||
|
Check(listBare.main.View().find("Read the full post") != std::string_view::npos,
|
||||||
|
"posts list: an excerptless post still offers the full post");
|
||||||
|
|
||||||
|
Post bodyless = p;
|
||||||
|
bodyless.body.clear();
|
||||||
|
const std::array<Post, 1> none{ bodyless };
|
||||||
|
const auto listNone = Views::RenderPosts(none);
|
||||||
|
Check(listNone.main.View().find("Read the full post") == std::string_view::npos,
|
||||||
|
"posts list: no read-more without a page to read");
|
||||||
|
Check(listNone.main.View().find(p.permalink) != std::string_view::npos,
|
||||||
|
"posts list: a pageless post still links its thread");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
void RunJsonSelfTest() {
|
void RunJsonSelfTest() {
|
||||||
using namespace Catcrafts::Json;
|
using namespace Catcrafts::Json;
|
||||||
|
|
||||||
|
|
@ -810,6 +1198,9 @@ int main(int argc, char** argv) {
|
||||||
RunJsonSelfTest();
|
RunJsonSelfTest();
|
||||||
RunFormSelfTest();
|
RunFormSelfTest();
|
||||||
RunMoneySelfTest();
|
RunMoneySelfTest();
|
||||||
|
RunMediaSelfTest();
|
||||||
|
RunMarkdownSelfTest();
|
||||||
|
RunPostSelfTest();
|
||||||
if (failures == 0) {
|
if (failures == 0) {
|
||||||
std::println("Catcrafts.Shared self-test: all assertions passed");
|
std::println("Catcrafts.Shared self-test: all assertions passed");
|
||||||
return 0;
|
return 0;
|
||||||
|
|
@ -829,7 +1220,7 @@ int main(int argc, char** argv) {
|
||||||
const Views::RenderedPage page = Views::RenderRoute(route, content);
|
const Views::RenderedPage page = Views::RenderRoute(route, content);
|
||||||
std::print("{}", Views::RenderDocument(
|
std::print("{}", Views::RenderDocument(
|
||||||
page,
|
page,
|
||||||
Views::RenderNav(route.kind == RouteKind::LegacyBlog ? RouteKind::Posts : route.kind),
|
Views::RenderNav(NavKindFor(route.kind)),
|
||||||
Views::RenderFooter(),
|
Views::RenderFooter(),
|
||||||
/*bootScripts=*/"", // no wasm on a plain server render
|
/*bootScripts=*/"", // no wasm on a plain server render
|
||||||
/*cssHref=*/"/styles.css"));
|
/*cssHref=*/"/styles.css"));
|
||||||
|
|
@ -859,6 +1250,14 @@ int main(int argc, char** argv) {
|
||||||
std::print(" <url><loc>https://catcrafts.net/shop/{}</loc></url>\n",
|
std::print(" <url><loc>https://catcrafts.net/shop/{}</loc></url>\n",
|
||||||
Html::Escape(pr.slug).Str());
|
Html::Escape(pr.slug).Str());
|
||||||
}
|
}
|
||||||
|
// Same rule as the served sitemap: only posts that actually have a
|
||||||
|
// page. Both must agree, because this is the copy baked into the wasm
|
||||||
|
// bundle and that one is what a crawler fetches.
|
||||||
|
for (const Post& po : content.posts) {
|
||||||
|
if (!po.HasPage()) continue;
|
||||||
|
std::print(" <url><loc>https://catcrafts.net/posts/{}</loc></url>\n",
|
||||||
|
Html::Escape(po.slug).Str());
|
||||||
|
}
|
||||||
std::print("</urlset>\n");
|
std::print("</urlset>\n");
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
@ -877,7 +1276,7 @@ int main(int argc, char** argv) {
|
||||||
"/order/not-a-token",
|
"/order/not-a-token",
|
||||||
"/legal/privacy", "/legal/imprint",
|
"/legal/privacy", "/legal/imprint",
|
||||||
"/legal/terms", "/legal/nope",
|
"/legal/terms", "/legal/nope",
|
||||||
"/projects", "/posts", "/demos",
|
"/projects", "/posts", "/posts/nope", "/demos",
|
||||||
"/demos/raytracer", "/demos/nope", "/demo",
|
"/demos/raytracer", "/demos/nope", "/demo",
|
||||||
"/projects/", "/blog", "/blog/hello-world", "/nope" }) {
|
"/projects/", "/blog", "/blog/hello-world", "/nope" }) {
|
||||||
const Route r = ParseRoute(p);
|
const Route r = ParseRoute(p);
|
||||||
|
|
|
||||||
588
shared/interfaces/Catcrafts.Shared-Markdown.cppm
Normal file
588
shared/interfaces/Catcrafts.Shared-Markdown.cppm
Normal file
|
|
@ -0,0 +1,588 @@
|
||||||
|
/*
|
||||||
|
catcrafts.net
|
||||||
|
Copyright (C) 2026 Catcrafts
|
||||||
|
|
||||||
|
The source code of this website is made available for viewing purposes only.
|
||||||
|
No permission is granted to copy, modify, distribute, or create derivative works.
|
||||||
|
*/
|
||||||
|
|
||||||
|
// A deliberately small Markdown renderer, for fediverse post bodies only.
|
||||||
|
//
|
||||||
|
// The site used to have no markdown pipeline at all, on the grounds that the
|
||||||
|
// posts page only ever showed a 280-character preview and the body stayed on
|
||||||
|
// the instance. Hosting the full body changes that calculation: a post IS
|
||||||
|
// prose with headings, quotes, code blocks and screenshots, and rendering it
|
||||||
|
// as one flat paragraph of literal `**asterisks**` would be worse than not
|
||||||
|
// hosting it. So: a parser, but only as much of one as these bodies use.
|
||||||
|
//
|
||||||
|
// WHAT IT SUPPORTS — everything observed in the real bodies, and nothing else:
|
||||||
|
//
|
||||||
|
// blocks ATX headings, fenced code, blockquotes (nested), ordered and
|
||||||
|
// unordered lists, thematic breaks, paragraphs
|
||||||
|
// inline links, images, code spans, ** strong **, * emphasis *, and bare
|
||||||
|
// URLs via Html::Autolink
|
||||||
|
//
|
||||||
|
// WHAT IT DELIBERATELY DOES NOT SUPPORT:
|
||||||
|
//
|
||||||
|
// * Raw HTML. Never. A post body is text fetched from someone else's server,
|
||||||
|
// so the ONE thing this renderer must guarantee is that no byte of it can
|
||||||
|
// become markup. Every character of body text leaves here through
|
||||||
|
// Html::Escape or Html::Autolink, and the only SafeHtml built from a raw
|
||||||
|
// string is the fixed structural markup written in this file. That is why
|
||||||
|
// `<` in a body renders as a less-than sign rather than opening a tag.
|
||||||
|
// * Underscore emphasis. `_` is common inside identifiers that appear in
|
||||||
|
// these posts unquoted (kworker/u16:8-qc_ufs_qos_swq), and mangling half a
|
||||||
|
// symbol name into italics is a worse failure than not italicising a word
|
||||||
|
// that used the underscore form. Asterisks are unambiguous here.
|
||||||
|
// * Setext headings, reference links, tables, footnotes, HTML entities.
|
||||||
|
// None appear; adding them speculatively is parser surface with no reader.
|
||||||
|
// * Trailing-double-space hard breaks. An invisible two-character difference
|
||||||
|
// is not something a reader can see in the source or a writer can rely on
|
||||||
|
// having typed; the lines of a paragraph join with a space, and a break
|
||||||
|
// that was meant is written as a blank line.
|
||||||
|
//
|
||||||
|
// Anything unrecognised degrades to text rather than being dropped, so a
|
||||||
|
// construct this parser does not know shows up as visibly odd prose instead of
|
||||||
|
// silently vanishing from the page.
|
||||||
|
|
||||||
|
export module Catcrafts.Shared:Markdown;
|
||||||
|
import std;
|
||||||
|
import :Html;
|
||||||
|
import :Media;
|
||||||
|
import :Model;
|
||||||
|
|
||||||
|
namespace Catcrafts::Markdown {
|
||||||
|
|
||||||
|
using Html::SafeHtml;
|
||||||
|
using Html::Escape;
|
||||||
|
using Html::Autolink;
|
||||||
|
using Html::Attr;
|
||||||
|
using Html::Format;
|
||||||
|
using Html::Join;
|
||||||
|
using Html::Raw;
|
||||||
|
using Html::Url;
|
||||||
|
|
||||||
|
// Blockquotes recurse, and a body is untrusted input, so the recursion needs a
|
||||||
|
// bound that does not depend on the input being sane. Four is past anything
|
||||||
|
// these posts do (a quote inside a list item inside a quote) and far short of
|
||||||
|
// anything that could trouble the stack.
|
||||||
|
constexpr int kMaxDepth = 4;
|
||||||
|
|
||||||
|
// ── small string helpers ──────────────────────────────────────────────
|
||||||
|
|
||||||
|
bool IsSpace(char c) { return c == ' ' || c == '\t'; }
|
||||||
|
|
||||||
|
std::string_view TrimRight(std::string_view s) {
|
||||||
|
while (!s.empty() && (IsSpace(s.back()) || s.back() == '\r')) s.remove_suffix(1);
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::string_view TrimLeft(std::string_view s) {
|
||||||
|
while (!s.empty() && IsSpace(s.front())) s.remove_prefix(1);
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::string_view Trim(std::string_view s) { return TrimLeft(TrimRight(s)); }
|
||||||
|
|
||||||
|
bool Blank(std::string_view line) { return Trim(line).empty(); }
|
||||||
|
|
||||||
|
// Up to three leading spaces are indentation a block marker is still allowed
|
||||||
|
// to carry; four or more would be a code block in real Markdown, which these
|
||||||
|
// bodies never use (they fence instead).
|
||||||
|
std::string_view Undent(std::string_view line) {
|
||||||
|
std::size_t n = 0;
|
||||||
|
while (n < line.size() && n < 3 && line[n] == ' ') ++n;
|
||||||
|
return line.substr(n);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── media ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// One embedded file, in the same shape (and CSS) the posts page uses for a
|
||||||
|
// post's headline media — an inline screenshot and a post's headline recording
|
||||||
|
// are the same kind of thing to a reader, so they should not look like two
|
||||||
|
// different components. :Media is what guarantees that: both go through it.
|
||||||
|
//
|
||||||
|
// A src with no mirrored record (its download failed, so the body still points
|
||||||
|
// at the original URL) still renders, just without dimensions or format tiers.
|
||||||
|
SafeHtml MediaTag(std::string_view src, std::string_view alt,
|
||||||
|
std::span<const PostMedia> media) {
|
||||||
|
return Media::Tag(Media::Describe(media, src), alt);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── inline ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
SafeHtml RenderInline(std::string_view text, std::span<const PostMedia> media, int depth);
|
||||||
|
|
||||||
|
// The span between `open` and its matching close delimiter, honouring nesting,
|
||||||
|
// or npos when it never closes. Used for [link text] and (link target), both of
|
||||||
|
// which can legitimately contain their own brackets — the Fairphone manual link
|
||||||
|
// in one of these posts has a parenthesised sentence as its text.
|
||||||
|
std::size_t MatchingDelimiter(std::string_view s, std::size_t from, char open, char close) {
|
||||||
|
int depth = 0;
|
||||||
|
for (std::size_t i = from; i < s.size(); ++i) {
|
||||||
|
if (s[i] == '\\') { ++i; continue; }
|
||||||
|
if (s[i] == open) { ++depth; continue; }
|
||||||
|
if (s[i] == close) {
|
||||||
|
if (depth == 0) return i;
|
||||||
|
--depth;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return std::string_view::npos;
|
||||||
|
}
|
||||||
|
|
||||||
|
// A run of ordinary prose, escaped and with bare URLs linked.
|
||||||
|
//
|
||||||
|
// Autolink rather than Escape because these bodies paste addresses constantly —
|
||||||
|
// mailing-list archives, merge requests, the shop — as bare text with no
|
||||||
|
// Markdown link syntax around them. Rendering those inert would strip most of
|
||||||
|
// the outbound value out of the post. Autolink escapes everything itself and
|
||||||
|
// puts each address through Html::Url, so this adds no new trusted path.
|
||||||
|
SafeHtml PlainRun(std::string_view text) {
|
||||||
|
return text.empty() ? SafeHtml{} : Autolink(text);
|
||||||
|
}
|
||||||
|
|
||||||
|
// True when the delimiter at `i` opens (rather than closes) an emphasis run:
|
||||||
|
// it must be followed by something that is not whitespace. Combined with the
|
||||||
|
// closing test below this is what keeps a lone asterisk — a multiplication
|
||||||
|
// sign, a footnote marker, a shell glob — from swallowing the rest of a
|
||||||
|
// paragraph into italics.
|
||||||
|
bool OpensEmphasis(std::string_view s, std::size_t after) {
|
||||||
|
return after < s.size() && !IsSpace(s[after]) && s[after] != '\n';
|
||||||
|
}
|
||||||
|
|
||||||
|
// The closing delimiter for an emphasis run opened at `from`, or npos. The
|
||||||
|
// character before it must not be whitespace, so "a * b * c" stays literal.
|
||||||
|
std::size_t FindEmphasisClose(std::string_view s, std::size_t from, std::string_view delim) {
|
||||||
|
std::size_t i = from;
|
||||||
|
while (i < s.size()) {
|
||||||
|
const std::size_t at = s.find(delim, i);
|
||||||
|
if (at == std::string_view::npos) return std::string_view::npos;
|
||||||
|
if (at > from && !IsSpace(s[at - 1])) return at;
|
||||||
|
i = at + delim.size();
|
||||||
|
}
|
||||||
|
return std::string_view::npos;
|
||||||
|
}
|
||||||
|
|
||||||
|
SafeHtml RenderInline(std::string_view text, std::span<const PostMedia> media, int depth) {
|
||||||
|
std::vector<SafeHtml> out;
|
||||||
|
std::size_t run = 0; // start of the pending plain-text run
|
||||||
|
|
||||||
|
auto flush = [&](std::size_t upto) {
|
||||||
|
if (upto > run) out.push_back(PlainRun(text.substr(run, upto - run)));
|
||||||
|
};
|
||||||
|
|
||||||
|
std::size_t i = 0;
|
||||||
|
while (i < text.size()) {
|
||||||
|
const char c = text[i];
|
||||||
|
|
||||||
|
// A backslash escape hides the next character from this parser. The
|
||||||
|
// pair is emitted as the second character alone, which is what lets a
|
||||||
|
// post write a literal asterisk.
|
||||||
|
if (c == '\\' && i + 1 < text.size()) {
|
||||||
|
flush(i);
|
||||||
|
out.push_back(Escape(text.substr(i + 1, 1)));
|
||||||
|
i += 2;
|
||||||
|
run = i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// `code` — highest precedence, so an asterisk inside a code span is a
|
||||||
|
// literal asterisk and not an emphasis delimiter.
|
||||||
|
if (c == '`') {
|
||||||
|
const std::size_t close = text.find('`', i + 1);
|
||||||
|
if (close != std::string_view::npos) {
|
||||||
|
flush(i);
|
||||||
|
out.push_back(Format(R"(<code>{}</code>)",
|
||||||
|
Escape(text.substr(i + 1, close - i - 1))));
|
||||||
|
i = close + 1;
|
||||||
|
run = i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
//  — an image. Checked before the link case, since the
|
||||||
|
// bracket that follows would otherwise parse as one.
|
||||||
|
if (c == '!' && i + 1 < text.size() && text[i + 1] == '[' && depth < kMaxDepth) {
|
||||||
|
const std::size_t altEnd = MatchingDelimiter(text, i + 2, '[', ']');
|
||||||
|
if (altEnd != std::string_view::npos && altEnd + 1 < text.size()
|
||||||
|
&& text[altEnd + 1] == '(') {
|
||||||
|
const std::size_t srcEnd = MatchingDelimiter(text, altEnd + 2, '(', ')');
|
||||||
|
if (srcEnd != std::string_view::npos) {
|
||||||
|
flush(i);
|
||||||
|
out.push_back(MediaTag(Trim(text.substr(altEnd + 2, srcEnd - altEnd - 2)),
|
||||||
|
text.substr(i + 2, altEnd - i - 2),
|
||||||
|
media));
|
||||||
|
i = srcEnd + 1;
|
||||||
|
run = i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// [text](href)
|
||||||
|
if (c == '[' && depth < kMaxDepth) {
|
||||||
|
const std::size_t textEnd = MatchingDelimiter(text, i + 1, '[', ']');
|
||||||
|
if (textEnd != std::string_view::npos && textEnd + 1 < text.size()
|
||||||
|
&& text[textEnd + 1] == '(') {
|
||||||
|
const std::size_t hrefEnd = MatchingDelimiter(text, textEnd + 2, '(', ')');
|
||||||
|
if (hrefEnd != std::string_view::npos) {
|
||||||
|
const std::string_view href =
|
||||||
|
Trim(text.substr(textEnd + 2, hrefEnd - textEnd - 2));
|
||||||
|
flush(i);
|
||||||
|
// The label is rendered rather than escaped flat, because
|
||||||
|
// these posts bold inside link text. Depth-guarded, so a
|
||||||
|
// link whose label contains a link cannot recurse forever.
|
||||||
|
out.push_back(Format(
|
||||||
|
R"(<a{} rel="noopener">{}</a>)",
|
||||||
|
Url("href", href),
|
||||||
|
RenderInline(text.substr(i + 1, textEnd - i - 1), media, depth + 1)));
|
||||||
|
i = hrefEnd + 1;
|
||||||
|
run = i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// **strong** before *emphasis*: the longer delimiter has to win, or
|
||||||
|
// every bold run parses as an empty italic followed by loose text.
|
||||||
|
if (c == '*') {
|
||||||
|
const bool doubled = i + 1 < text.size() && text[i + 1] == '*';
|
||||||
|
const std::string_view delim = doubled ? "**" : "*";
|
||||||
|
const std::size_t inner = i + delim.size();
|
||||||
|
if (depth < kMaxDepth && OpensEmphasis(text, inner)) {
|
||||||
|
const std::size_t close = FindEmphasisClose(text, inner, delim);
|
||||||
|
if (close != std::string_view::npos) {
|
||||||
|
flush(i);
|
||||||
|
const SafeHtml body =
|
||||||
|
RenderInline(text.substr(inner, close - inner), media, depth + 1);
|
||||||
|
out.push_back(doubled ? Format(R"(<strong>{}</strong>)", body)
|
||||||
|
: Format(R"(<em>{}</em>)", body));
|
||||||
|
i = close + delim.size();
|
||||||
|
run = i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
++i;
|
||||||
|
}
|
||||||
|
|
||||||
|
flush(text.size());
|
||||||
|
return Join(out);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── blocks ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// A line's leading run of '#', when it is an ATX heading marker.
|
||||||
|
// Returns 0 when the line is not a heading.
|
||||||
|
int HeadingLevel(std::string_view line) {
|
||||||
|
std::size_t n = 0;
|
||||||
|
while (n < line.size() && line[n] == '#') ++n;
|
||||||
|
if (n == 0 || n > 6) return 0;
|
||||||
|
// "#tag" is not a heading; a marker has to be followed by space or be the
|
||||||
|
// whole line.
|
||||||
|
if (n < line.size() && !IsSpace(line[n])) return 0;
|
||||||
|
return static_cast<int>(n);
|
||||||
|
}
|
||||||
|
|
||||||
|
bool IsFence(std::string_view line) { return TrimLeft(line).starts_with("```"); }
|
||||||
|
|
||||||
|
// `---`, `***`, `___`: three or more of one character, nothing else but spaces.
|
||||||
|
bool IsThematicBreak(std::string_view line) {
|
||||||
|
const std::string_view s = Trim(line);
|
||||||
|
if (s.size() < 3) return false;
|
||||||
|
const char c = s.front();
|
||||||
|
if (c != '-' && c != '*' && c != '_') return false;
|
||||||
|
int count = 0;
|
||||||
|
for (const char ch : s) {
|
||||||
|
if (ch == c) { ++count; continue; }
|
||||||
|
if (!IsSpace(ch)) return false;
|
||||||
|
}
|
||||||
|
return count >= 3;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct ListMarker {
|
||||||
|
bool ok = false;
|
||||||
|
bool ordered = false;
|
||||||
|
std::int64_t start = 1; // the number an ordered item announced
|
||||||
|
std::size_t contentAt = 0; // offset of the item's text within the line
|
||||||
|
};
|
||||||
|
|
||||||
|
ListMarker ParseListMarker(std::string_view line) {
|
||||||
|
ListMarker m;
|
||||||
|
const std::string_view s = Undent(line);
|
||||||
|
const std::size_t indent = line.size() - s.size();
|
||||||
|
if (s.empty()) return m;
|
||||||
|
|
||||||
|
if ((s[0] == '-' || s[0] == '*' || s[0] == '+') && s.size() > 1 && IsSpace(s[1])) {
|
||||||
|
// A thematic break is also a run of dashes; it wins, because "- - -"
|
||||||
|
// is a rule everywhere and a three-item list nowhere.
|
||||||
|
if (IsThematicBreak(line)) return m;
|
||||||
|
m.ok = true;
|
||||||
|
m.contentAt = indent + 2;
|
||||||
|
return m;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::size_t n = 0;
|
||||||
|
while (n < s.size() && s[n] >= '0' && s[n] <= '9') ++n;
|
||||||
|
// Bounded so a line starting with a long number is not mistaken for a list.
|
||||||
|
if (n == 0 || n > 9) return m;
|
||||||
|
if (n + 1 >= s.size() || (s[n] != '.' && s[n] != ')') || !IsSpace(s[n + 1])) return m;
|
||||||
|
|
||||||
|
std::int64_t value = 0;
|
||||||
|
std::from_chars(s.data(), s.data() + n, value);
|
||||||
|
m.ok = true;
|
||||||
|
m.ordered = true;
|
||||||
|
m.start = value;
|
||||||
|
m.contentAt = indent + n + 2;
|
||||||
|
return m;
|
||||||
|
}
|
||||||
|
|
||||||
|
bool StartsBlock(std::string_view line) {
|
||||||
|
return HeadingLevel(Undent(line)) > 0 || IsFence(line) || IsThematicBreak(line)
|
||||||
|
|| Undent(line).starts_with('>') || ParseListMarker(line).ok;
|
||||||
|
}
|
||||||
|
|
||||||
|
SafeHtml RenderBlocks(std::span<const std::string_view> lines,
|
||||||
|
std::span<const PostMedia> media, int depth);
|
||||||
|
|
||||||
|
// A paragraph's lines, joined and rendered.
|
||||||
|
//
|
||||||
|
// A paragraph whose entire content is embedded files becomes the same
|
||||||
|
// .post-media block the cards use rather than a <p> of images: that is what
|
||||||
|
// gives a run of screenshots the two-up grid instead of a column of full-width
|
||||||
|
// pictures with paragraph spacing between them.
|
||||||
|
SafeHtml RenderParagraph(std::span<const std::string_view> lines,
|
||||||
|
std::span<const PostMedia> media, int depth) {
|
||||||
|
// Lines join with a space: a body wraps its prose at whatever width the
|
||||||
|
// author's editor used, and those wraps are not meaningful.
|
||||||
|
std::string joined;
|
||||||
|
for (std::size_t i = 0; i < lines.size(); ++i) {
|
||||||
|
joined += Trim(lines[i]);
|
||||||
|
if (i + 1 < lines.size()) joined += ' ';
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only-images test: strip every  span and see whether anything but
|
||||||
|
// whitespace is left.
|
||||||
|
bool onlyMedia = false;
|
||||||
|
{
|
||||||
|
std::string rest;
|
||||||
|
std::size_t i = 0;
|
||||||
|
std::size_t found = 0;
|
||||||
|
const std::string_view s = joined;
|
||||||
|
while (i < s.size()) {
|
||||||
|
if (s[i] == '!' && i + 1 < s.size() && s[i + 1] == '[') {
|
||||||
|
const std::size_t altEnd = MatchingDelimiter(s, i + 2, '[', ']');
|
||||||
|
if (altEnd != std::string_view::npos && altEnd + 1 < s.size()
|
||||||
|
&& s[altEnd + 1] == '(') {
|
||||||
|
const std::size_t srcEnd = MatchingDelimiter(s, altEnd + 2, '(', ')');
|
||||||
|
if (srcEnd != std::string_view::npos) {
|
||||||
|
++found;
|
||||||
|
i = srcEnd + 1;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rest += s[i];
|
||||||
|
++i;
|
||||||
|
}
|
||||||
|
onlyMedia = found > 0 && Trim(rest).empty();
|
||||||
|
}
|
||||||
|
|
||||||
|
const SafeHtml inner = RenderInline(joined, media, depth);
|
||||||
|
if (inner.Empty()) return SafeHtml{};
|
||||||
|
if (onlyMedia) return Format(R"(<div class="post-media">{}</div>)", inner);
|
||||||
|
return Format(R"(<p>{}</p>)", inner);
|
||||||
|
}
|
||||||
|
|
||||||
|
SafeHtml RenderBlocks(std::span<const std::string_view> lines,
|
||||||
|
std::span<const PostMedia> media, int depth) {
|
||||||
|
std::vector<SafeHtml> out;
|
||||||
|
std::size_t i = 0;
|
||||||
|
|
||||||
|
while (i < lines.size()) {
|
||||||
|
if (Blank(lines[i])) { ++i; continue; }
|
||||||
|
|
||||||
|
const std::string_view line = lines[i];
|
||||||
|
const std::string_view body = Undent(line);
|
||||||
|
|
||||||
|
// ── fenced code ───────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// Taken verbatim and escaped: the battery-measurement tables and the
|
||||||
|
// android top output in these posts are the one place where every
|
||||||
|
// space matters and nothing inside should be interpreted at all.
|
||||||
|
if (IsFence(line)) {
|
||||||
|
std::size_t j = i + 1;
|
||||||
|
std::string code;
|
||||||
|
while (j < lines.size() && !IsFence(lines[j])) {
|
||||||
|
code += TrimRight(lines[j]);
|
||||||
|
code += '\n';
|
||||||
|
++j;
|
||||||
|
}
|
||||||
|
out.push_back(Format(R"(<pre class="post-body__code"><code>{}</code></pre>)",
|
||||||
|
Escape(code)));
|
||||||
|
// Past the closing fence, or to the end when it never closed —
|
||||||
|
// an unterminated fence renders as code rather than swallowing
|
||||||
|
// the rest of the post into a parse failure.
|
||||||
|
i = (j < lines.size()) ? j + 1 : j;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── heading ───────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// Demoted by one: the page's <h1> is the post title, so a body's own
|
||||||
|
// top-level heading is a section inside it. Without the shift every
|
||||||
|
// post would carry two h1s and the document outline would be wrong on
|
||||||
|
// exactly the pages this whole change exists to make indexable.
|
||||||
|
if (const int level = HeadingLevel(body); level > 0) {
|
||||||
|
std::string_view textPart = body.substr(static_cast<std::size_t>(level));
|
||||||
|
// A closing run of #s is decoration, not content.
|
||||||
|
textPart = TrimRight(textPart);
|
||||||
|
while (!textPart.empty() && textPart.back() == '#') textPart.remove_suffix(1);
|
||||||
|
const int tag = std::min(level + 1, 6);
|
||||||
|
out.push_back(Format("<h{}>{}</h{}>",
|
||||||
|
Html::Num(tag),
|
||||||
|
RenderInline(Trim(textPart), media, depth),
|
||||||
|
Html::Num(tag)));
|
||||||
|
++i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (IsThematicBreak(line)) {
|
||||||
|
out.push_back(Raw("<hr>"));
|
||||||
|
++i;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── blockquote ────────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// The quoted lines are re-parsed as blocks, so a quote keeps its own
|
||||||
|
// paragraphs and headings — which matters here, because the longest
|
||||||
|
// quotes in these posts are multi-paragraph company copy being taken
|
||||||
|
// apart line by line.
|
||||||
|
if (body.starts_with('>')) {
|
||||||
|
std::vector<std::string_view> quoted;
|
||||||
|
std::size_t j = i;
|
||||||
|
while (j < lines.size() && Undent(lines[j]).starts_with('>')) {
|
||||||
|
std::string_view q = Undent(lines[j]).substr(1);
|
||||||
|
if (!q.empty() && q.front() == ' ') q.remove_prefix(1);
|
||||||
|
quoted.push_back(q);
|
||||||
|
++j;
|
||||||
|
}
|
||||||
|
// At the depth limit the quote is still shown, just flattened —
|
||||||
|
// dropping it would lose content, which is the worse failure.
|
||||||
|
out.push_back(Format(
|
||||||
|
R"(<blockquote class="post-body__quote">{}</blockquote>)",
|
||||||
|
depth < kMaxDepth ? RenderBlocks(quoted, media, depth + 1)
|
||||||
|
: RenderParagraph(quoted, media, depth)));
|
||||||
|
i = j;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── list ──────────────────────────────────────────────────────
|
||||||
|
if (const ListMarker first = ParseListMarker(line); first.ok) {
|
||||||
|
std::vector<SafeHtml> items;
|
||||||
|
std::vector<std::string_view> current;
|
||||||
|
std::size_t j = i;
|
||||||
|
|
||||||
|
auto flushItem = [&] {
|
||||||
|
if (current.empty()) return;
|
||||||
|
items.push_back(Format(R"(<li>{}</li>)",
|
||||||
|
RenderInline([&] {
|
||||||
|
std::string joined;
|
||||||
|
for (std::size_t k = 0; k < current.size(); ++k) {
|
||||||
|
joined += Trim(current[k]);
|
||||||
|
if (k + 1 < current.size()) joined += ' ';
|
||||||
|
}
|
||||||
|
return joined;
|
||||||
|
}(), media, depth)));
|
||||||
|
current.clear();
|
||||||
|
};
|
||||||
|
|
||||||
|
while (j < lines.size()) {
|
||||||
|
if (Blank(lines[j])) {
|
||||||
|
// A blank line ends the list UNLESS the next non-blank line
|
||||||
|
// is another item of the same kind. These posts space their
|
||||||
|
// numbered steps apart, and treating that as seven separate
|
||||||
|
// one-item lists would restart the numbering at every gap.
|
||||||
|
std::size_t peek = j;
|
||||||
|
while (peek < lines.size() && Blank(lines[peek])) ++peek;
|
||||||
|
const ListMarker next =
|
||||||
|
peek < lines.size() ? ParseListMarker(lines[peek]) : ListMarker{};
|
||||||
|
if (!next.ok || next.ordered != first.ordered) break;
|
||||||
|
j = peek;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (const ListMarker m = ParseListMarker(lines[j]); m.ok) {
|
||||||
|
if (m.ordered != first.ordered) break;
|
||||||
|
flushItem();
|
||||||
|
current.push_back(lines[j].substr(
|
||||||
|
std::min(m.contentAt, lines[j].size())));
|
||||||
|
++j;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// A non-marker line that would start some other block ends the
|
||||||
|
// list; anything else is this item's text continuing onto the
|
||||||
|
// next line.
|
||||||
|
if (StartsBlock(lines[j])) break;
|
||||||
|
current.push_back(lines[j]);
|
||||||
|
++j;
|
||||||
|
}
|
||||||
|
flushItem();
|
||||||
|
|
||||||
|
if (first.ordered) {
|
||||||
|
// start= only when it is not 1, so the common case stays clean
|
||||||
|
// markup — and so a list resumed after an interrupting
|
||||||
|
// paragraph continues its numbering instead of starting over.
|
||||||
|
out.push_back(Format(R"(<ol class="post-body__list"{}>{}</ol>)",
|
||||||
|
first.start == 1 ? SafeHtml{}
|
||||||
|
: Attr("start", std::to_string(first.start)),
|
||||||
|
Join(items)));
|
||||||
|
} else {
|
||||||
|
out.push_back(Format(R"(<ul class="post-body__list">{}</ul>)", Join(items)));
|
||||||
|
}
|
||||||
|
i = j;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── paragraph ─────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
std::size_t j = i;
|
||||||
|
while (j < lines.size() && !Blank(lines[j])) {
|
||||||
|
// A block marker on a later line interrupts the paragraph
|
||||||
|
// rather than being absorbed into it as text.
|
||||||
|
if (j > i && StartsBlock(lines[j])) break;
|
||||||
|
++j;
|
||||||
|
}
|
||||||
|
out.push_back(RenderParagraph(lines.subspan(i, j - i), media, depth));
|
||||||
|
i = j;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return Join(out);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── entry point ───────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// Render a post body to the inner markup of the article element.
|
||||||
|
//
|
||||||
|
// `media` is the post's mirrored inline files, used to attach dimensions (and
|
||||||
|
// a poster, and the H.264 fallback) to whatever the body embeds. Passing an
|
||||||
|
// empty span is fine: the markup then simply carries no dimensions, exactly
|
||||||
|
// like a file whose mirror failed.
|
||||||
|
export SafeHtml Render(std::string_view text, std::span<const PostMedia> media = {}) {
|
||||||
|
std::vector<std::string_view> lines;
|
||||||
|
std::size_t start = 0;
|
||||||
|
while (start <= text.size()) {
|
||||||
|
const std::size_t nl = text.find('\n', start);
|
||||||
|
if (nl == std::string_view::npos) {
|
||||||
|
lines.push_back(text.substr(start));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
lines.push_back(text.substr(start, nl - start));
|
||||||
|
start = nl + 1;
|
||||||
|
}
|
||||||
|
return RenderBlocks(lines, media, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace Catcrafts::Markdown
|
||||||
192
shared/interfaces/Catcrafts.Shared-Media.cppm
Normal file
192
shared/interfaces/Catcrafts.Shared-Media.cppm
Normal file
|
|
@ -0,0 +1,192 @@
|
||||||
|
/*
|
||||||
|
catcrafts.net
|
||||||
|
Copyright (C) 2026 Catcrafts
|
||||||
|
|
||||||
|
The source code of this website is made available for viewing purposes only.
|
||||||
|
No permission is granted to copy, modify, distribute, or create derivative works.
|
||||||
|
*/
|
||||||
|
|
||||||
|
// The markup for one mirrored file — image or video.
|
||||||
|
//
|
||||||
|
// This exists as its own partition because two callers need identical output:
|
||||||
|
// :Views renders a post's headline media on the card and at the top of the post
|
||||||
|
// page, and :Markdown renders whatever the body embeds. They were separate
|
||||||
|
// copies of the same element-building code, which is how the AV1 fallback logic
|
||||||
|
// came to be written twice; a format tier added to one and forgotten in the
|
||||||
|
// other shows up as the same picture served two different ways on two pages.
|
||||||
|
//
|
||||||
|
// FORMAT LADDER. Every file the mirror stores is served in the best form the
|
||||||
|
// browser will take, with a form every browser will take underneath it:
|
||||||
|
//
|
||||||
|
// video AV1 in MP4, falling back to H.264 in MP4. Two <source> elements;
|
||||||
|
// the codecs parameter on the first is what lets a browser without
|
||||||
|
// AV1 skip it rather than fail on a file it cannot decode.
|
||||||
|
// image AVIF, falling back to the mirrored original (usually WebP), falling
|
||||||
|
// back to PNG. A <picture>, so the browser fetches exactly one of
|
||||||
|
// them — the tiers cost nothing to the visitors who do not need them.
|
||||||
|
//
|
||||||
|
// The middle image tier is free: it is the file the mirror already downloaded,
|
||||||
|
// so it adds no encode and no disk. It matters because it is what stands
|
||||||
|
// between "no AVIF" and the PNG, and PNG is lossless — for a photograph that is
|
||||||
|
// an order of magnitude larger than the WebP beside it. With the middle tier
|
||||||
|
// the PNG is reached only by a browser that supports neither AVIF nor WebP.
|
||||||
|
//
|
||||||
|
// Any tier can be absent (the transcode was unavailable, or the file was never
|
||||||
|
// mirrored at all) and the markup degrades a step at a time, down to a bare
|
||||||
|
// <img src> — which is what the site emitted before any of this existed.
|
||||||
|
|
||||||
|
export module Catcrafts.Shared:Media;
|
||||||
|
import std;
|
||||||
|
import :Html;
|
||||||
|
import :Model;
|
||||||
|
|
||||||
|
namespace Catcrafts::Media {
|
||||||
|
|
||||||
|
using Html::SafeHtml;
|
||||||
|
using Html::Escape;
|
||||||
|
using Html::Attr;
|
||||||
|
using Html::Format;
|
||||||
|
using Html::Join;
|
||||||
|
using Html::Url;
|
||||||
|
|
||||||
|
bool EndsWithNoCase(std::string_view s, std::string_view suffix) {
|
||||||
|
if (s.size() < suffix.size()) return false;
|
||||||
|
const std::size_t off = s.size() - suffix.size();
|
||||||
|
for (std::size_t i = 0; i < suffix.size(); ++i) {
|
||||||
|
char a = s[off + i];
|
||||||
|
if (a >= 'A' && a <= 'Z') a = static_cast<char>(a - 'A' + 'a');
|
||||||
|
if (a != suffix[i]) return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// True when the path names something the <video> path should handle. Used only
|
||||||
|
// to guess for a file that was never mirrored, so there is no record to ask.
|
||||||
|
export bool LooksLikeVideo(std::string_view src) {
|
||||||
|
return EndsWithNoCase(src, ".mp4") || EndsWithNoCase(src, ".webm")
|
||||||
|
|| EndsWithNoCase(src, ".mov");
|
||||||
|
}
|
||||||
|
|
||||||
|
// The type= a <source> should advertise. Empty for anything not recognised, in
|
||||||
|
// which case the tier is dropped rather than guessed: a wrong type is worse
|
||||||
|
// than a missing source, because the browser believes it.
|
||||||
|
std::string_view MimeFor(std::string_view src) {
|
||||||
|
if (EndsWithNoCase(src, ".avif")) return "image/avif";
|
||||||
|
if (EndsWithNoCase(src, ".webp")) return "image/webp";
|
||||||
|
if (EndsWithNoCase(src, ".png")) return "image/png";
|
||||||
|
if (EndsWithNoCase(src, ".jpg") || EndsWithNoCase(src, ".jpeg")) return "image/jpeg";
|
||||||
|
if (EndsWithNoCase(src, ".gif")) return "image/gif";
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
|
||||||
|
// Width and height, or nothing when the mirror could not probe them. Both or
|
||||||
|
// neither: a lone dimension is worse than none, because the browser derives the
|
||||||
|
// other from it and gets the aspect ratio wrong.
|
||||||
|
SafeHtml Dimensions(const PostMedia& m) {
|
||||||
|
if (m.width <= 0 || m.height <= 0) return SafeHtml{};
|
||||||
|
return Format("{}{}", Attr("width", std::to_string(m.width)),
|
||||||
|
Attr("height", std::to_string(m.height)));
|
||||||
|
}
|
||||||
|
|
||||||
|
SafeHtml ImageTag(const PostMedia& m, std::string_view alt) {
|
||||||
|
// What every browser reads, and the only element that is always present.
|
||||||
|
// The PNG when there is one, because that is the tier nothing can refuse.
|
||||||
|
const std::string_view imgSrc = m.fallback.empty() ? m.src : m.fallback;
|
||||||
|
|
||||||
|
std::vector<SafeHtml> sources;
|
||||||
|
if (!m.avif.empty()) {
|
||||||
|
sources.push_back(Format(R"(<source{} type="image/avif">)", Url("srcset", m.avif)));
|
||||||
|
}
|
||||||
|
// The mirrored original, sitting between the AVIF and the PNG. Skipped when
|
||||||
|
// it IS what the <img> points at or what the AVIF tier already offered
|
||||||
|
// (nothing left to say), or when its type cannot be named.
|
||||||
|
if (m.src != imgSrc && m.src != m.avif) {
|
||||||
|
if (const std::string_view mime = MimeFor(m.src); !mime.empty()) {
|
||||||
|
sources.push_back(Format(R"(<source{}{}>)",
|
||||||
|
Url("srcset", m.src), Attr("type", mime)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// alt is whatever the author wrote, which for a post's headline media is
|
||||||
|
// nothing: those are screenshots whose meaning is already in the title and
|
||||||
|
// the excerpt, and inventing descriptive alt text here would be making up
|
||||||
|
// what the picture shows. Emitted explicitly even when empty — alt="" is
|
||||||
|
// skipped cleanly by a screen reader, a missing alt makes it read the file
|
||||||
|
// name out loud.
|
||||||
|
const SafeHtml img = Format(
|
||||||
|
R"(<img class="post-media__item" loading="lazy" decoding="async" alt="{}"{}{}>)",
|
||||||
|
Escape(alt), Url("src", imgSrc), Dimensions(m));
|
||||||
|
|
||||||
|
if (sources.empty()) return img;
|
||||||
|
return Format("<picture>{}{}</picture>", Join(sources), img);
|
||||||
|
}
|
||||||
|
|
||||||
|
SafeHtml VideoTag(const PostMedia& m) {
|
||||||
|
// preload="metadata", not "auto": a page with several 5 MB recordings must
|
||||||
|
// not pull them all on load.
|
||||||
|
const SafeHtml poster = m.poster.empty() ? SafeHtml{} : Url("poster", m.poster);
|
||||||
|
const SafeHtml dims = Dimensions(m);
|
||||||
|
|
||||||
|
if (m.fallback.empty() || m.fallback == m.src) {
|
||||||
|
return Format(
|
||||||
|
R"(<video class="post-media__item" controls preload="metadata" playsinline{}{}{}></video>)",
|
||||||
|
Url("src", m.src), poster, dims);
|
||||||
|
}
|
||||||
|
// An AV1 video with its H.264 rendition. Both are .mp4, so the container
|
||||||
|
// alone cannot tell them apart: the codecs parameter on the first <source>
|
||||||
|
// is what lets a browser without AV1 (Safari before 17, Apple hardware
|
||||||
|
// without the decoder) skip it and take the H.264 instead of failing on a
|
||||||
|
// file it cannot decode. The string is advisory and used only for
|
||||||
|
// selection — once a source is picked the browser reads the actual stream —
|
||||||
|
// so the canonical profile-0 8-bit form is right for anything
|
||||||
|
// publish-media.sh emits (yuv420p is pinned there).
|
||||||
|
return Format(
|
||||||
|
R"(<video class="post-media__item" controls preload="metadata" playsinline{}{}>)"
|
||||||
|
R"(<source{} type="video/mp4; codecs=av01.0.08M.08">)"
|
||||||
|
R"(<source{} type="video/mp4">)"
|
||||||
|
R"(</video>)",
|
||||||
|
poster, dims, Url("src", m.src), Url("src", m.fallback));
|
||||||
|
}
|
||||||
|
|
||||||
|
// The element for one mirrored file.
|
||||||
|
export SafeHtml Tag(const PostMedia& m, std::string_view alt = {}) {
|
||||||
|
return m.kind == "video" ? VideoTag(m) : ImageTag(m, alt);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The mirrored record for `src`, or nullptr when this path was never mirrored.
|
||||||
|
//
|
||||||
|
// Matching is on the exact string because tools/fetch-media.sh rewrites the
|
||||||
|
// body text and the body_media list from the same mapping in the same pass —
|
||||||
|
// they cannot disagree about a path without the mirror step itself being wrong.
|
||||||
|
export const PostMedia* Find(std::span<const PostMedia> media, std::string_view src) {
|
||||||
|
for (const PostMedia& m : media) {
|
||||||
|
if (m.src == src) return &m;
|
||||||
|
}
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
|
||||||
|
// The record for `src`, or one invented from the path alone.
|
||||||
|
//
|
||||||
|
// The invented case is a file whose download failed, so the body still points
|
||||||
|
// at its original URL: there are no dimensions and no renditions to offer, and
|
||||||
|
// the markup degrades to the bare element. Rendering something beats dropping
|
||||||
|
// the picture the paragraph is talking about.
|
||||||
|
export PostMedia Describe(std::span<const PostMedia> media, std::string_view src) {
|
||||||
|
if (const PostMedia* hit = Find(media, src)) return *hit;
|
||||||
|
PostMedia m;
|
||||||
|
m.src = std::string(src);
|
||||||
|
m.kind = LooksLikeVideo(src) ? "video" : "image";
|
||||||
|
return m;
|
||||||
|
}
|
||||||
|
|
||||||
|
// A post's media as one block. More than one file gets a two-up grid via CSS,
|
||||||
|
// so a post with four screenshots does not become a mile of scrolling.
|
||||||
|
export SafeHtml Block(std::span<const PostMedia> media) {
|
||||||
|
if (media.empty()) return SafeHtml{};
|
||||||
|
std::vector<SafeHtml> items;
|
||||||
|
items.reserve(media.size());
|
||||||
|
for (const PostMedia& m : media) items.push_back(Tag(m));
|
||||||
|
return Format(R"(<div class="post-media">{}</div>)", Join(items));
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace Catcrafts::Media
|
||||||
|
|
@ -30,6 +30,11 @@ No permission is granted to copy, modify, distribute, or create derivative works
|
||||||
export module Catcrafts.Shared:Model;
|
export module Catcrafts.Shared:Model;
|
||||||
import std;
|
import std;
|
||||||
import :Json;
|
import :Json;
|
||||||
|
// For IsValidSlug. A post slug becomes a URL, so it is checked here on the way
|
||||||
|
// in rather than trusted because CI wrote it: an unparseable slug would render
|
||||||
|
// a "read more" link to a route that can never match, which is a 404 the site
|
||||||
|
// links to itself. Dropping it instead degrades to a post with no page.
|
||||||
|
import :Route;
|
||||||
|
|
||||||
namespace Catcrafts {
|
namespace Catcrafts {
|
||||||
|
|
||||||
|
|
@ -50,34 +55,69 @@ export struct PostMedia {
|
||||||
// play — and these posts are their video, so the black box is the page.
|
// play — and these posts are their video, so the black box is the page.
|
||||||
// Empty for images, and empty when the instance generated no thumbnail.
|
// Empty for images, and empty when the instance generated no thumbnail.
|
||||||
std::string poster;
|
std::string poster;
|
||||||
// H.264 rendition of an AV1 video, when tools/publish-media.sh uploaded one
|
// The rendition every browser can take, and the bottom of the format
|
||||||
// beside it. Non-empty means the renderer emits a <source> pair instead of
|
// ladder :Media builds. Empty means there is only `src`.
|
||||||
// a bare src, so a browser without AV1 (Safari before 17, Apple hardware
|
//
|
||||||
// without the decoder) gets a file it can play. Empty for images and for
|
// video H.264 in MP4, uploaded beside the AV1 by
|
||||||
// videos that need no fallback.
|
// tools/publish-media.sh. Without it a browser lacking AV1
|
||||||
|
// (Safari before 17, Apple hardware without the decoder) has an
|
||||||
|
// element it cannot play.
|
||||||
|
// image PNG, transcoded by tools/fetch-media.sh. It is what the <img>
|
||||||
|
// itself points at, so a browser that understands no <source>
|
||||||
|
// type at all still gets the picture.
|
||||||
std::string fallback;
|
std::string fallback;
|
||||||
|
// AVIF rendition of an image, transcoded by tools/fetch-media.sh. Offered
|
||||||
|
// above `src` — it is consistently smaller than the WebP the instances
|
||||||
|
// serve, at 4:4:4 chroma so coloured text in a screenshot does not fringe.
|
||||||
|
// Empty for videos, and for an image the transcode could not produce.
|
||||||
|
std::string avif;
|
||||||
std::int64_t width = 0;
|
std::int64_t width = 0;
|
||||||
std::int64_t height = 0;
|
std::int64_t height = 0;
|
||||||
};
|
};
|
||||||
|
|
||||||
// A post mirrored from the fediverse (Lemmy). Deliberately not the full post:
|
// A post mirrored from the fediverse (Lemmy).
|
||||||
// the body stays on the community's instance, where the comments are. We show
|
//
|
||||||
// enough to be worth clicking and then hand off — no crawling, no comment
|
// The body IS mirrored — the comments are not. That split is the whole policy:
|
||||||
// mirroring, no markdown pipeline.
|
// the writing is the work and belongs on the site that is about the work (and
|
||||||
|
// is the only version a search engine can be pointed at as canonical), while
|
||||||
|
// the discussion belongs to the community that is having it. So every post
|
||||||
|
// page carries the full text and then hands off to the thread. No crawling, no
|
||||||
|
// comment mirroring, no runtime dependency on the instance.
|
||||||
//
|
//
|
||||||
// `permalink` is resolved at fetch time to the COMMUNITY's instance, not the
|
// `permalink` is resolved at fetch time to the COMMUNITY's instance, not the
|
||||||
// author's. Both host a federated copy; the community's is where the discussion
|
// author's. Both host a federated copy; the community's is where the discussion
|
||||||
// actually is, and it is what the site should be pointing readers at.
|
// actually is, and it is what the site should be pointing readers at.
|
||||||
export struct Post {
|
export struct Post {
|
||||||
std::string title;
|
std::string title;
|
||||||
|
// URL-safe name of this post's own page at /posts/<slug>, derived from the
|
||||||
|
// title by tools/fetch-posts.sh. Empty only if the title reduced to nothing
|
||||||
|
// a slug can be made of, which is what HasPage() below tests for.
|
||||||
|
std::string slug;
|
||||||
std::string permalink; // canonical ap_id on the instance; where "discuss" goes
|
std::string permalink; // canonical ap_id on the instance; where "discuss" goes
|
||||||
std::string linkUrl; // for link posts, the linked target; empty otherwise
|
std::string linkUrl; // for link posts, the linked target; empty otherwise
|
||||||
std::string community; // e.g. "linux@lemmy.ml"
|
std::string community; // e.g. "linux@lemmy.ml"
|
||||||
std::string published; // ISO-8601, as emitted by the API
|
std::string published; // ISO-8601, as emitted by the API
|
||||||
std::string excerpt; // plain text, truncated by CI — never markdown
|
std::string excerpt; // plain text, truncated by CI — never markdown
|
||||||
|
// The full post, still as Markdown. Rendered by :Markdown at page-render
|
||||||
|
// time rather than converted to HTML at fetch time, because the renderer is
|
||||||
|
// where this codebase's escaping guarantee lives — text that arrives from
|
||||||
|
// someone else's server must not become markup anywhere but there.
|
||||||
|
std::string body;
|
||||||
std::int64_t score = 0;
|
std::int64_t score = 0;
|
||||||
std::int64_t comments = 0;
|
std::int64_t comments = 0;
|
||||||
|
// The post's headline media — `post.url` upstream, the recording or
|
||||||
|
// screenshot the post is *about*. Shown on the card and above the body.
|
||||||
std::vector<PostMedia> media;
|
std::vector<PostMedia> media;
|
||||||
|
// Files embedded inside the body, mirrored by the same pass. The body text
|
||||||
|
// already points at these local paths; this list exists so the renderer can
|
||||||
|
// attach dimensions (and a poster, and the H.264 fallback) to them, which
|
||||||
|
// Markdown syntax has nowhere to carry.
|
||||||
|
std::vector<PostMedia> bodyMedia;
|
||||||
|
|
||||||
|
// Whether this post gets its own page. A post with no body has nothing to
|
||||||
|
// put on one — a page consisting of a title and a link out is thin content
|
||||||
|
// that should not be minted as a URL, indexed, or linked to as "read more".
|
||||||
|
bool HasPage() const { return !slug.empty() && !body.empty(); }
|
||||||
};
|
};
|
||||||
|
|
||||||
// An entry on the projects page. Repo content, not a database — these change
|
// An entry on the projects page. Repo content, not a database — these change
|
||||||
|
|
@ -294,6 +334,32 @@ export struct LegalPage {
|
||||||
|
|
||||||
// ── loaders ───────────────────────────────────────────────────────────
|
// ── loaders ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// The media array shape, shared by a post's headline media and its body media —
|
||||||
|
// they are the same records produced by the same mirror pass, so they are read
|
||||||
|
// by one function rather than two that could drift.
|
||||||
|
std::vector<PostMedia> LoadPostMedia(const Json::Value* array) {
|
||||||
|
std::vector<PostMedia> out;
|
||||||
|
if (!array || !array->IsArray()) return out;
|
||||||
|
for (const Json::Value& mv : array->array) {
|
||||||
|
if (!mv.IsObject()) continue;
|
||||||
|
PostMedia pm;
|
||||||
|
pm.src = std::string(mv.Str("src"));
|
||||||
|
pm.kind = std::string(mv.Str("kind", "image"));
|
||||||
|
pm.poster = std::string(mv.Str("poster"));
|
||||||
|
pm.fallback = std::string(mv.Str("fallback"));
|
||||||
|
pm.avif = std::string(mv.Str("avif"));
|
||||||
|
pm.width = mv.Int("w");
|
||||||
|
pm.height = mv.Int("h");
|
||||||
|
// Only the two kinds the renderer knows how to emit. Anything else
|
||||||
|
// would fall through to an <img> for a file that is not an image, so
|
||||||
|
// treat it as image only when it says so.
|
||||||
|
if (pm.kind != "image" && pm.kind != "video") pm.kind = "image";
|
||||||
|
if (pm.src.empty()) continue;
|
||||||
|
out.push_back(std::move(pm));
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
export std::vector<Post> LoadPosts(std::string_view json) {
|
export std::vector<Post> LoadPosts(std::string_view json) {
|
||||||
std::vector<Post> out;
|
std::vector<Post> out;
|
||||||
auto doc = Json::Parse(json);
|
auto doc = Json::Parse(json);
|
||||||
|
|
@ -303,31 +369,19 @@ export std::vector<Post> LoadPosts(std::string_view json) {
|
||||||
if (!item.IsObject()) continue;
|
if (!item.IsObject()) continue;
|
||||||
Post p;
|
Post p;
|
||||||
p.title = std::string(item.Str("title"));
|
p.title = std::string(item.Str("title"));
|
||||||
|
if (const std::string_view slug = item.Str("slug"); IsValidSlug(slug)) {
|
||||||
|
p.slug = std::string(slug);
|
||||||
|
}
|
||||||
p.permalink = std::string(item.Str("permalink"));
|
p.permalink = std::string(item.Str("permalink"));
|
||||||
p.linkUrl = std::string(item.Str("url"));
|
p.linkUrl = std::string(item.Str("url"));
|
||||||
p.community = std::string(item.Str("community"));
|
p.community = std::string(item.Str("community"));
|
||||||
p.published = std::string(item.Str("published"));
|
p.published = std::string(item.Str("published"));
|
||||||
p.excerpt = std::string(item.Str("excerpt"));
|
p.excerpt = std::string(item.Str("excerpt"));
|
||||||
|
p.body = std::string(item.Str("body"));
|
||||||
p.score = item.Int("score");
|
p.score = item.Int("score");
|
||||||
p.comments = item.Int("comments");
|
p.comments = item.Int("comments");
|
||||||
if (const Json::Value* m = item.Find("media"); m && m->IsArray()) {
|
p.media = LoadPostMedia(item.Find("media"));
|
||||||
for (const Json::Value& mv : m->array) {
|
p.bodyMedia = LoadPostMedia(item.Find("body_media"));
|
||||||
if (!mv.IsObject()) continue;
|
|
||||||
PostMedia pm;
|
|
||||||
pm.src = std::string(mv.Str("src"));
|
|
||||||
pm.kind = std::string(mv.Str("kind", "image"));
|
|
||||||
pm.poster = std::string(mv.Str("poster"));
|
|
||||||
pm.fallback = std::string(mv.Str("fallback"));
|
|
||||||
pm.width = mv.Int("w");
|
|
||||||
pm.height = mv.Int("h");
|
|
||||||
// Only the two kinds the renderer knows how to emit. Anything
|
|
||||||
// else would fall through to an <img> for a file that is not an
|
|
||||||
// image, so treat it as image only when it says so.
|
|
||||||
if (pm.kind != "image" && pm.kind != "video") pm.kind = "image";
|
|
||||||
if (pm.src.empty()) continue;
|
|
||||||
p.media.push_back(std::move(pm));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// A post with no title and nowhere to click is not renderable; drop it
|
// A post with no title and nowhere to click is not renderable; drop it
|
||||||
// rather than emit an empty card.
|
// rather than emit an empty card.
|
||||||
if (p.title.empty() && p.permalink.empty()) continue;
|
if (p.title.empty() && p.permalink.empty()) continue;
|
||||||
|
|
|
||||||
|
|
@ -23,6 +23,7 @@ export enum class RouteKind {
|
||||||
About, // /about — the person behind the company
|
About, // /about — the person behind the company
|
||||||
Projects,
|
Projects,
|
||||||
Posts,
|
Posts,
|
||||||
|
Post, // /posts/<slug> — one post, in full
|
||||||
Demos, // /demos — the list
|
Demos, // /demos — the list
|
||||||
Demo, // /demos/<slug>
|
Demo, // /demos/<slug>
|
||||||
Shop, // /shop — the (currently single-item) product list
|
Shop, // /shop — the (currently single-item) product list
|
||||||
|
|
@ -45,7 +46,8 @@ export struct Route {
|
||||||
std::string query; // raw, including leading '?', or empty
|
std::string query; // raw, including leading '?', or empty
|
||||||
// Non-empty when the request should be canonicalised to a different URL.
|
// Non-empty when the request should be canonicalised to a different URL.
|
||||||
std::string canonicalRedirect;
|
std::string canonicalRedirect;
|
||||||
// The <slug> of /shop/<slug>, empty for every other route.
|
// The <slug> of /shop/<slug>, /demos/<slug>, /legal/<slug> or /posts/<slug>
|
||||||
|
// (and the token of /order/<token>); empty for every other route.
|
||||||
std::string slug;
|
std::string slug;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
@ -134,6 +136,20 @@ export Route ParseRoute(std::string_view path, std::string_view query = {}) {
|
||||||
return r;
|
return r;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// /posts/<slug>. Whether a slug names a post the site actually has is the
|
||||||
|
// dispatcher's question, exactly as it is for /shop/<slug>; this only
|
||||||
|
// decides that the URL is shaped like a post page at all.
|
||||||
|
if (p.starts_with("/posts/")) {
|
||||||
|
const std::string_view slug = p.substr(7);
|
||||||
|
if (IsValidSlug(slug)) {
|
||||||
|
r.kind = RouteKind::Post;
|
||||||
|
r.slug = std::string(slug);
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
r.kind = RouteKind::NotFound;
|
||||||
|
return r;
|
||||||
|
}
|
||||||
|
|
||||||
if (p.starts_with("/demos/")) {
|
if (p.starts_with("/demos/")) {
|
||||||
const std::string_view slug = p.substr(7);
|
const std::string_view slug = p.substr(7);
|
||||||
if (IsValidSlug(slug)) {
|
if (IsValidSlug(slug)) {
|
||||||
|
|
@ -183,6 +199,20 @@ export struct NavItem {
|
||||||
RouteKind kind;
|
RouteKind kind;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// Which nav entry a route highlights. A route that is not itself a nav entry
|
||||||
|
// borrows its section's: an individual post and the retired /blog URL both sit
|
||||||
|
// under Posts. Shared so the three call sites (the server, the CLI renderer and
|
||||||
|
// the wasm router) cannot disagree about where the reader is — they each used
|
||||||
|
// to spell the /blog case out, and the third one to be added would have been
|
||||||
|
// the third chance to forget it.
|
||||||
|
export RouteKind NavKindFor(RouteKind kind) {
|
||||||
|
switch (kind) {
|
||||||
|
case RouteKind::Post:
|
||||||
|
case RouteKind::LegacyBlog: return RouteKind::Posts;
|
||||||
|
default: return kind;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
export std::span<const NavItem> NavItems() {
|
export std::span<const NavItem> NavItems() {
|
||||||
static constexpr std::array<NavItem, 6> items{{
|
static constexpr std::array<NavItem, 6> items{{
|
||||||
{ "Home", "/", RouteKind::Home },
|
{ "Home", "/", RouteKind::Home },
|
||||||
|
|
|
||||||
|
|
@ -23,6 +23,8 @@ import std;
|
||||||
import :Content;
|
import :Content;
|
||||||
import :Html;
|
import :Html;
|
||||||
import :Form;
|
import :Form;
|
||||||
|
import :Markdown;
|
||||||
|
import :Media;
|
||||||
import :Model;
|
import :Model;
|
||||||
import :Money;
|
import :Money;
|
||||||
import :Route;
|
import :Route;
|
||||||
|
|
@ -189,7 +191,11 @@ export RenderedPage RenderHome(std::span<const Project> projects,
|
||||||
const Post& post = posts[i];
|
const Post& post = posts[i];
|
||||||
recent.push_back(Format(
|
recent.push_back(Format(
|
||||||
R"(<li class="recent__item"><a{}>{}</a><span class="recent__date">{}</span></li>)",
|
R"(<li class="recent__item"><a{}>{}</a><span class="recent__date">{}</span></li>)",
|
||||||
Url("href", post.permalink),
|
// The on-site page when the post has one, the thread otherwise.
|
||||||
|
// Same rule as the cards on /posts: a link from the home page is
|
||||||
|
// worth more pointing at a page this site owns than at someone
|
||||||
|
// else's copy of it, and the post page links onward to the thread.
|
||||||
|
Url("href", post.HasPage() ? "/posts/" + post.slug : post.permalink),
|
||||||
Escape(post.title),
|
Escape(post.title),
|
||||||
Escape(DateOnly(post.published))));
|
Escape(DateOnly(post.published))));
|
||||||
}
|
}
|
||||||
|
|
@ -341,65 +347,62 @@ export RenderedPage RenderProjects(std::span<const Project> projects) {
|
||||||
// keeps the privacy notice's "everything comes from catcrafts.net" true and
|
// keeps the privacy notice's "everything comes from catcrafts.net" true and
|
||||||
// stops every visitor's IP reaching whichever instance hosted the file.
|
// stops every visitor's IP reaching whichever instance hosted the file.
|
||||||
//
|
//
|
||||||
// Video is preload="metadata", not "auto": a page with several 5 MB recordings
|
// The element itself — the format ladder, the dimensions, the poster — is
|
||||||
// must not pull them all on load. Dimensions come from the content file so the
|
// :Media's job, shared with the body renderer so a post's headline screenshot
|
||||||
// browser reserves the right box and nothing jumps as files arrive.
|
// and one embedded in its prose cannot be served differently.
|
||||||
SafeHtml RenderPostMedia(std::span<const PostMedia> media) {
|
SafeHtml RenderPostMedia(std::span<const PostMedia> media) {
|
||||||
if (media.empty()) return SafeHtml{};
|
return Media::Block(media);
|
||||||
std::vector<SafeHtml> items;
|
|
||||||
for (const PostMedia& m : media) {
|
|
||||||
SafeHtml dims = (m.width > 0 && m.height > 0)
|
|
||||||
? Format("{}{}", Attr("width", std::to_string(m.width)),
|
|
||||||
Attr("height", std::to_string(m.height)))
|
|
||||||
: SafeHtml{};
|
|
||||||
if (m.kind == "video") {
|
|
||||||
SafeHtml poster = m.poster.empty() ? SafeHtml{} : Url("poster", m.poster);
|
|
||||||
if (m.fallback.empty() || m.fallback == m.src) {
|
|
||||||
items.push_back(Format(
|
|
||||||
R"(<video class="post-media__item" controls preload="metadata" )"
|
|
||||||
R"(playsinline{}{}{}></video>)",
|
|
||||||
Url("src", m.src), poster, dims));
|
|
||||||
} else {
|
|
||||||
// An AV1 video with its H.264 rendition. Both are .mp4, so the
|
|
||||||
// container alone cannot tell them apart: the codecs parameter
|
|
||||||
// on the first <source> is what lets a browser without AV1
|
|
||||||
// (Safari before 17, Apple hardware without the decoder) skip
|
|
||||||
// it and take the H.264 instead of failing on a file it cannot
|
|
||||||
// decode. The string is advisory and used only for selection —
|
|
||||||
// once a source is picked the browser reads the actual stream —
|
|
||||||
// so the canonical profile-0 8-bit form is right for anything
|
|
||||||
// publish-media.sh emits (yuv420p is pinned there).
|
|
||||||
items.push_back(Format(
|
|
||||||
R"(<video class="post-media__item" controls preload="metadata" )"
|
|
||||||
R"(playsinline{}{}>)"
|
|
||||||
R"(<source{} type="video/mp4; codecs=av01.0.08M.08">)"
|
|
||||||
R"(<source{} type="video/mp4">)"
|
|
||||||
R"(</video>)",
|
|
||||||
poster, dims, Url("src", m.src), Url("src", m.fallback)));
|
|
||||||
}
|
}
|
||||||
} else {
|
|
||||||
// alt is empty and aria-hidden is absent on purpose: these are
|
// A link post gets a second, clearly-labelled outbound link. Without the label
|
||||||
// screenshots whose meaning is already in the post title and
|
// the two links are indistinguishable and one of them silently leaves the site.
|
||||||
// excerpt, and inventing descriptive alt text here would be making
|
SafeHtml RenderPostLinkRow(const Post& p) {
|
||||||
// up what the picture shows.
|
if (p.linkUrl.empty()) return SafeHtml{};
|
||||||
items.push_back(Format(
|
return Format(
|
||||||
R"(<img class="post-media__item" loading="lazy" decoding="async" )"
|
R"(<p class="post-card__link"><a{} rel="nofollow noopener">{}</a></p>)",
|
||||||
R"(alt=""{}{}>)",
|
Url("href", p.linkUrl), Escape(p.linkUrl));
|
||||||
Url("src", m.src), dims));
|
|
||||||
}
|
}
|
||||||
}
|
|
||||||
return Format(R"(<div class="post-media">{}</div>)", Join(items));
|
// Points and the thread link, identical on a card and at the foot of a post
|
||||||
|
// page — the reader is being offered the same two things in both places, and
|
||||||
|
// they should not look like two different components.
|
||||||
|
SafeHtml RenderPostStats(const Post& p) {
|
||||||
|
return Format(
|
||||||
|
R"(<footer class="post-card__footer">)"
|
||||||
|
R"(<span class="stat">{} points</span>)"
|
||||||
|
R"(<a class="stat stat--link"{} rel="noopener">Discuss on the fediverse ({} comments) →</a>)"
|
||||||
|
R"(</footer>)",
|
||||||
|
Num(p.score), Url("href", p.permalink), Num(p.comments));
|
||||||
}
|
}
|
||||||
|
|
||||||
export RenderedPage RenderPosts(std::span<const Post> posts) {
|
export RenderedPage RenderPosts(std::span<const Post> posts) {
|
||||||
std::vector<SafeHtml> cards;
|
std::vector<SafeHtml> cards;
|
||||||
for (const Post& p : posts) {
|
for (const Post& p : posts) {
|
||||||
// A link post gets a second, clearly-labelled outbound link. Without
|
// The title goes to this site's copy when there is one. Before post
|
||||||
// the label the two links are indistinguishable and one of them
|
// pages existed it could only go to the thread, which meant the list
|
||||||
// silently leaves the site.
|
// of everything this site is about was a list of links off it.
|
||||||
SafeHtml linkRow = p.linkUrl.empty() ? SafeHtml{} : Format(
|
const std::string titleHref = p.HasPage() ? "/posts/" + p.slug : p.permalink;
|
||||||
R"(<p class="post-card__link"><a{} rel="nofollow noopener">{}</a></p>)",
|
|
||||||
Url("href", p.linkUrl), Escape(p.linkUrl));
|
// No "read more" without a page to read more on — a post whose body
|
||||||
|
// never arrived would otherwise offer a link to its own 404.
|
||||||
|
//
|
||||||
|
// It goes inline at the end of the excerpt, immediately after the
|
||||||
|
// ellipsis the truncation left: that is where the sentence stops and
|
||||||
|
// where the reader is already looking for the rest of it. Below the
|
||||||
|
// media it was the same offer made a screen further down, after the
|
||||||
|
// reader had already decided.
|
||||||
|
const SafeHtml more = !p.HasPage() ? SafeHtml{} : Format(
|
||||||
|
R"( <a class="link-more"{}>Read the full post</a>)",
|
||||||
|
Url("href", "/posts/" + p.slug));
|
||||||
|
|
||||||
|
// With no excerpt there is no sentence to continue, so the link falls
|
||||||
|
// back to a row of its own rather than disappearing along with it.
|
||||||
|
const SafeHtml excerptRow =
|
||||||
|
!p.excerpt.empty()
|
||||||
|
? Format(R"(<p class="post-card__excerpt">{}{}</p>)", Escape(p.excerpt), more)
|
||||||
|
: more.Empty()
|
||||||
|
? SafeHtml{}
|
||||||
|
: Format(R"(<p class="post-card__more">{}</p>)", more);
|
||||||
|
|
||||||
cards.push_back(Format(
|
cards.push_back(Format(
|
||||||
R"(<article class="post-card">)"
|
R"(<article class="post-card">)"
|
||||||
|
|
@ -412,32 +415,28 @@ export RenderedPage RenderPosts(std::span<const Post> posts) {
|
||||||
R"({})"
|
R"({})"
|
||||||
R"({})"
|
R"({})"
|
||||||
R"({})"
|
R"({})"
|
||||||
R"(<footer class="post-card__footer">)"
|
R"({})"
|
||||||
R"(<span class="stat">{} points</span>)"
|
|
||||||
R"(<a class="stat stat--link"{} rel="noopener">Discuss on the fediverse ({} comments) →</a>)"
|
|
||||||
R"(</footer>)"
|
|
||||||
R"(</article>)",
|
R"(</article>)",
|
||||||
Url("href", p.permalink), Escape(p.title),
|
Url("href", titleHref), Escape(p.title),
|
||||||
Attr("datetime", p.published), Escape(DateOnly(p.published)),
|
Attr("datetime", p.published), Escape(DateOnly(p.published)),
|
||||||
Escape(p.community),
|
Escape(p.community),
|
||||||
p.excerpt.empty() ? SafeHtml{}
|
excerptRow,
|
||||||
: Format(R"(<p class="post-card__excerpt">{}</p>)", Escape(p.excerpt)),
|
|
||||||
RenderPostMedia(p.media),
|
RenderPostMedia(p.media),
|
||||||
linkRow,
|
RenderPostLinkRow(p),
|
||||||
Num(p.score),
|
RenderPostStats(p)));
|
||||||
Url("href", p.permalink), Num(p.comments)));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
RenderedPage page;
|
RenderedPage page;
|
||||||
page.meta.title = "Posts — Catcrafts";
|
page.meta.title = "Posts — Catcrafts";
|
||||||
page.meta.description = "Posts from the fediverse; discussion happens there.";
|
page.meta.description = "Posts about mobile Linux, kernel work and open hardware, "
|
||||||
|
"written for the fediverse and archived here in full.";
|
||||||
page.meta.canonical = "/posts";
|
page.meta.canonical = "/posts";
|
||||||
page.main = Format(
|
page.main = Format(
|
||||||
R"(<header class="page-header">)"
|
R"(<header class="page-header">)"
|
||||||
R"(<h1 class="page-header__title">Posts</h1>)"
|
R"(<h1 class="page-header__title">Posts</h1>)"
|
||||||
R"(<p class="page-header__lede">Catcrafts posts on the fediverse rather than keeping a blog here. )"
|
R"(<p class="page-header__lede">Catcrafts posts on the fediverse rather than keeping a blog here. )"
|
||||||
R"(Each of these links to the original thread on whichever instance it lives on. )"
|
R"(Each one is kept here in full, and links to the original thread on whichever )"
|
||||||
R"(Follow one to read it and join the discussion there.</p>)"
|
R"(instance it lives on — that is where the discussion is.</p>)"
|
||||||
R"(</header>)"
|
R"(</header>)"
|
||||||
R"(<div class="post-list">{}</div>)",
|
R"(<div class="post-list">{}</div>)",
|
||||||
cards.empty()
|
cards.empty()
|
||||||
|
|
@ -446,6 +445,88 @@ export RenderedPage RenderPosts(std::span<const Post> posts) {
|
||||||
return page;
|
return page;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── one post, in full ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// The whole point of hosting the body: a page a search engine can index, a
|
||||||
|
// reader can link to, and this site can claim as canonical. The card on /posts
|
||||||
|
// is a summary of this; the thread on the instance is where the comments are.
|
||||||
|
//
|
||||||
|
// The body arrives as Markdown and is rendered HERE rather than converted at
|
||||||
|
// fetch time, because :Markdown is inside the escaping guarantee and a shell
|
||||||
|
// script writing HTML into a content file would not be. See that module for
|
||||||
|
// what it does and does not accept.
|
||||||
|
export RenderedPage RenderPost(const Post& p) {
|
||||||
|
// Whatever the post leads with, for the link-preview card. A video has no
|
||||||
|
// still of its own to offer, so its poster frame stands in. Only our own
|
||||||
|
// mirrored copies qualify: an og:image on someone else's instance is the
|
||||||
|
// same third-party fetch the mirror exists to avoid, just performed by a
|
||||||
|
// crawler instead of a reader.
|
||||||
|
std::string ogImage;
|
||||||
|
for (const PostMedia& m : p.media) {
|
||||||
|
const std::string& candidate = m.kind == "video" ? m.poster : m.src;
|
||||||
|
if (candidate.starts_with("/")) { ogImage = candidate; break; }
|
||||||
|
}
|
||||||
|
|
||||||
|
const std::string canonical = "/posts/" + p.slug;
|
||||||
|
|
||||||
|
RenderedPage page;
|
||||||
|
page.meta.title = p.title + " — Catcrafts";
|
||||||
|
page.meta.description = p.excerpt;
|
||||||
|
page.meta.canonical = canonical;
|
||||||
|
page.meta.ogType = "article";
|
||||||
|
page.meta.ogImage = ogImage;
|
||||||
|
|
||||||
|
// BlogPosting, joined to the same two nodes every other page names: the
|
||||||
|
// Person on /about is the author and the Organization is the publisher.
|
||||||
|
// Without those @ids each post would introduce a fourth unrelated
|
||||||
|
// "Catcrafts" to a search index that already has trouble telling this one
|
||||||
|
// from the name-twins — see RenderHome for the whole argument.
|
||||||
|
//
|
||||||
|
// discussionUrl is the honest way to say what the fediverse link is: the
|
||||||
|
// comments belong to the thread, and this page is not pretending to mirror
|
||||||
|
// them.
|
||||||
|
page.meta.jsonLd = std::format(
|
||||||
|
R"({{"@context":"https://schema.org","@type":"BlogPosting",)"
|
||||||
|
R"("headline":{},"datePublished":{},"url":{},)"
|
||||||
|
R"("mainEntityOfPage":{{"@type":"WebPage","@id":{}}},)"
|
||||||
|
R"("author":{{"@id":"https://catcrafts.net/about#person",)"
|
||||||
|
R"("@type":"Person","name":"Jorijn van der Graaf"}},)"
|
||||||
|
R"("publisher":{{"@id":"https://catcrafts.net/#organization",)"
|
||||||
|
R"("@type":"Organization","name":"Catcrafts"}},)"
|
||||||
|
R"("discussionUrl":{}{}{}}})",
|
||||||
|
JsonStr(p.title), JsonStr(p.published),
|
||||||
|
JsonStr("https://catcrafts.net" + canonical),
|
||||||
|
JsonStr("https://catcrafts.net" + canonical),
|
||||||
|
JsonStr(p.permalink),
|
||||||
|
p.excerpt.empty() ? std::string{} : ",\"description\":" + JsonStr(p.excerpt),
|
||||||
|
ogImage.empty() ? std::string{}
|
||||||
|
: ",\"image\":" + JsonStr("https://catcrafts.net" + ogImage));
|
||||||
|
|
||||||
|
page.main = Format(
|
||||||
|
R"(<article class="post">)"
|
||||||
|
R"(<header class="page-header">)"
|
||||||
|
R"(<h1 class="page-header__title">{}</h1>)"
|
||||||
|
R"(<p class="post-card__meta">)"
|
||||||
|
R"(<time{}>{}</time><span class="post-card__community">{}</span>)"
|
||||||
|
R"(</p>)"
|
||||||
|
R"(</header>)"
|
||||||
|
R"({})"
|
||||||
|
R"({})"
|
||||||
|
R"(<div class="post-body">{}</div>)"
|
||||||
|
R"({})"
|
||||||
|
R"(<p class="post__back"><a class="link-more"{}>All posts</a></p>)"
|
||||||
|
R"(</article>)",
|
||||||
|
Escape(p.title),
|
||||||
|
Attr("datetime", p.published), Escape(DateOnly(p.published)),
|
||||||
|
Escape(p.community),
|
||||||
|
RenderPostMedia(p.media),
|
||||||
|
RenderPostLinkRow(p),
|
||||||
|
Markdown::Render(p.body, p.bodyMedia),
|
||||||
|
RenderPostStats(p),
|
||||||
|
Url("href", "/posts"));
|
||||||
|
return page;
|
||||||
|
}
|
||||||
|
|
||||||
// ── shop ──────────────────────────────────────────────────────────────
|
// ── shop ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
// The product page's price block: the same single headline number as the shop
|
// The product page's price block: the same single headline number as the shop
|
||||||
|
|
@ -1329,6 +1410,16 @@ export struct SiteContent {
|
||||||
}
|
}
|
||||||
return nullptr;
|
return nullptr;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Only posts that have a page. A post with no body has a slug but nothing
|
||||||
|
// to show, so finding it here would mint an indexable URL for a title and
|
||||||
|
// a link — see Post::HasPage.
|
||||||
|
const Post* FindPost(std::string_view slug) const {
|
||||||
|
for (const Post& p : posts) {
|
||||||
|
if (p.HasPage() && p.slug == slug) return &p;
|
||||||
|
}
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
RenderedPage RenderRouteBody(const Route& route, const SiteContent& content);
|
RenderedPage RenderRouteBody(const Route& route, const SiteContent& content);
|
||||||
|
|
@ -1357,6 +1448,13 @@ RenderedPage RenderRouteBody(const Route& route, const SiteContent& content) {
|
||||||
case RouteKind::About: return RenderAbout(Content::AboutPage());
|
case RouteKind::About: return RenderAbout(Content::AboutPage());
|
||||||
case RouteKind::Projects: return RenderProjects(content.projects);
|
case RouteKind::Projects: return RenderProjects(content.projects);
|
||||||
case RouteKind::Posts: return RenderPosts(content.posts);
|
case RouteKind::Posts: return RenderPosts(content.posts);
|
||||||
|
case RouteKind::Post: {
|
||||||
|
// A slug that parsed but names no post is a 404, for the same
|
||||||
|
// reason an unknown product slug is: otherwise every typo and
|
||||||
|
// every retired post becomes an indexable empty page.
|
||||||
|
if (const Post* p = content.FindPost(route.slug)) return RenderPost(*p);
|
||||||
|
break;
|
||||||
|
}
|
||||||
case RouteKind::Demos: return RenderDemos(content.demos);
|
case RouteKind::Demos: return RenderDemos(content.demos);
|
||||||
case RouteKind::Demo: {
|
case RouteKind::Demo: {
|
||||||
if (const Demo* d = content.FindDemo(route.slug)) return RenderDemo(*d);
|
if (const Demo* d = content.FindDemo(route.slug)) return RenderDemo(*d);
|
||||||
|
|
|
||||||
|
|
@ -35,6 +35,8 @@ export import :Html;
|
||||||
export import :Json;
|
export import :Json;
|
||||||
export import :Form;
|
export import :Form;
|
||||||
export import :Model;
|
export import :Model;
|
||||||
|
export import :Media;
|
||||||
|
export import :Markdown;
|
||||||
export import :Content;
|
export import :Content;
|
||||||
export import :Money;
|
export import :Money;
|
||||||
export import :Route;
|
export import :Route;
|
||||||
|
|
|
||||||
|
|
@ -487,6 +487,96 @@ treatment it replaces, which read as the wrong category for the work.
|
||||||
overflow-wrap: anywhere;
|
overflow-wrap: anywhere;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Only reached by a post with no excerpt to trail — normally the read-more
|
||||||
|
link rides at the end of .post-card__excerpt instead. */
|
||||||
|
.post-card__more { margin-top: var(--s-1); }
|
||||||
|
|
||||||
|
/* Trailing an excerpt, the link is part of that paragraph and wraps with it,
|
||||||
|
but it must not break across lines itself: "Read the full / post →" reads
|
||||||
|
as two different things. */
|
||||||
|
.post-card__excerpt .link-more { white-space: nowrap; }
|
||||||
|
|
||||||
|
/* ── one post, in full ──────────────────────────────────────────────
|
||||||
|
The body is prose rendered from Markdown by Catcrafts.Shared:Markdown,
|
||||||
|
so the element set here is exactly what that module can emit and no
|
||||||
|
more. Everything is capped at --measure: a post page is reading
|
||||||
|
material, and full-viewport line lengths are not read, they are
|
||||||
|
skimmed. Media is the exception — it keeps the full column, because a
|
||||||
|
screenshot of a feature-support table is unreadable at text width. */
|
||||||
|
.post-body {
|
||||||
|
display: grid;
|
||||||
|
gap: var(--s0);
|
||||||
|
margin-top: var(--s1);
|
||||||
|
}
|
||||||
|
/* The measure goes on the prose, not on the container, so media can use the
|
||||||
|
full column: a post page is reading material and full-viewport line
|
||||||
|
lengths get skimmed rather than read, but a screenshot of a feature
|
||||||
|
support table is unreadable at text width. */
|
||||||
|
.post-body > :not(.post-media) { max-width: var(--measure); }
|
||||||
|
/* .post-media carries its own top margin for the cards, where it follows an
|
||||||
|
excerpt; inside this grid the gap already provides it. */
|
||||||
|
.post-body > .post-media { margin-top: 0; }
|
||||||
|
/* Grid items default to min-width:auto, which refuses to shrink below their
|
||||||
|
content. Without this the overflow-x on a code block below never engages —
|
||||||
|
the track widens instead and the whole PAGE scrolls sideways, which is the
|
||||||
|
one thing wide content must never cause. */
|
||||||
|
.post-body > *, .post-body__quote > * { min-width: 0; }
|
||||||
|
|
||||||
|
.post-body h2 { font-size: var(--step-2); letter-spacing: -0.01em; }
|
||||||
|
.post-body h3 { font-size: var(--step-1); }
|
||||||
|
.post-body h4, .post-body h5, .post-body h6 { font-size: var(--step-0); }
|
||||||
|
/* Extra air above a heading, none below: a heading belongs to what
|
||||||
|
follows it, and the grid gap alone reads as floating between two
|
||||||
|
equally distant paragraphs. */
|
||||||
|
.post-body :is(h2, h3, h4, h5, h6) { margin-top: var(--s1); }
|
||||||
|
.post-body :is(h2, h3, h4, h5, h6):first-child { margin-top: 0; }
|
||||||
|
|
||||||
|
.post-body a { overflow-wrap: anywhere; }
|
||||||
|
|
||||||
|
.post-body__list { padding-left: var(--s1); display: grid; gap: var(--s-2); }
|
||||||
|
.post-body__list li { padding-left: var(--s-3); }
|
||||||
|
|
||||||
|
.post-body__quote {
|
||||||
|
padding: var(--s-2) var(--s0);
|
||||||
|
border-left: 2px solid var(--border-strong);
|
||||||
|
background: var(--surface-1);
|
||||||
|
border-radius: 0 var(--radius-sm) var(--radius-sm) 0;
|
||||||
|
color: var(--text-muted);
|
||||||
|
display: grid;
|
||||||
|
gap: var(--s-1);
|
||||||
|
}
|
||||||
|
|
||||||
|
.post-body__code {
|
||||||
|
padding: var(--s-1) var(--s0);
|
||||||
|
background: var(--surface-0);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: var(--radius-sm);
|
||||||
|
font-family: var(--font-mono);
|
||||||
|
font-size: var(--step--1);
|
||||||
|
/* Terminal output and measurement tables are tabular: wrapping them
|
||||||
|
destroys the alignment that is the whole reason they were pasted.
|
||||||
|
Scroll the block instead — never the page. */
|
||||||
|
overflow-x: auto;
|
||||||
|
}
|
||||||
|
.post-body :not(pre) > code {
|
||||||
|
font-family: var(--font-mono);
|
||||||
|
font-size: 0.9em;
|
||||||
|
padding: 0.1em 0.3em;
|
||||||
|
background: var(--surface-0);
|
||||||
|
border: 1px solid var(--border);
|
||||||
|
border-radius: var(--radius-sm);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* An image inside a sentence still gets its own line — these are
|
||||||
|
screenshots being pointed at, not icons sitting in the text. The <picture>
|
||||||
|
is what sits in the paragraph now (the reset already makes it a block);
|
||||||
|
the bare .post-media__item covers an image with no format ladder, which
|
||||||
|
is what a mirror failure or a missing encoder leaves behind. */
|
||||||
|
.post-body p > picture,
|
||||||
|
.post-body p > .post-media__item { margin-top: var(--s-2); }
|
||||||
|
|
||||||
|
.post__back { margin-top: var(--s1); }
|
||||||
|
|
||||||
/* Post media — screenshots and screen recordings of the work, mirrored to our
|
/* Post media — screenshots and screen recordings of the work, mirrored to our
|
||||||
own origin by tools/fetch-media.sh. */
|
own origin by tools/fetch-media.sh. */
|
||||||
.post-media {
|
.post-media {
|
||||||
|
|
|
||||||
126
tools/e2e.sh
126
tools/e2e.sh
|
|
@ -140,6 +140,10 @@ status /legal/nope 404
|
||||||
# A slug that cannot be one of ours is rejected before any lookup.
|
# A slug that cannot be one of ours is rejected before any lookup.
|
||||||
status /shop/BAD--slug 404
|
status /shop/BAD--slug 404
|
||||||
status /demos/nope 404
|
status /demos/nope 404
|
||||||
|
# A post slug that parsed but names nothing must be a real 404, or every typo
|
||||||
|
# and every retired post becomes an indexable empty page.
|
||||||
|
status /posts/nope 404
|
||||||
|
status /posts/BAD--slug 404
|
||||||
# The retired blog URLs are still in the wild; they must redirect, not 404.
|
# The retired blog URLs are still in the wild; they must redirect, not 404.
|
||||||
status /blog 301
|
status /blog 301
|
||||||
status /blog/hello-world 301
|
status /blog/hello-world 301
|
||||||
|
|
@ -367,7 +371,7 @@ else
|
||||||
fi
|
fi
|
||||||
# `poster` is in the list because a video poster is fetched on page load exactly
|
# `poster` is in the list because a video poster is fetched on page load exactly
|
||||||
# like an <img> src is, so a third-party poster leaks the same visitor IP.
|
# like an <img> src is, so a third-party poster leaks the same visitor IP.
|
||||||
if curl -s "$BASE/posts" | grep -qE '(src|href|poster)="https?://[^"]*\.(mp4|webm|webp|png|jpe?g|gif)'; then
|
if curl -s "$BASE/posts" | grep -qE '(src|srcset|href|poster)="https?://[^"]*\.(mp4|webm|webp|avif|png|jpe?g|gif)'; then
|
||||||
bad "/posts media origin" "media loaded from a third party"
|
bad "/posts media origin" "media loaded from a third party"
|
||||||
else
|
else
|
||||||
ok "/posts loads no media from a third party"
|
ok "/posts loads no media from a third party"
|
||||||
|
|
@ -417,6 +421,126 @@ else
|
||||||
bad "video preload" "expected preload=\"metadata\""
|
bad "video preload" "expected preload=\"metadata\""
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
echo "== post pages =="
|
||||||
|
# The post page is where the body lives, and the body is the reason the site
|
||||||
|
# has anything for a search engine to index beyond a list of links off it. Its
|
||||||
|
# slug is data, so take one from the page rather than hardcoding a title that
|
||||||
|
# will be wrong the week after it is written.
|
||||||
|
POST_PATH=$(curl -s "$BASE/posts" | grep -oE 'href="/posts/[a-z0-9-]+"' \
|
||||||
|
| head -n1 | sed 's/href="//; s/"$//')
|
||||||
|
if [ -z "$POST_PATH" ]; then
|
||||||
|
bad "post pages" "/posts links no post page; nothing carries a body"
|
||||||
|
else
|
||||||
|
ok "/posts links a post page ($POST_PATH)"
|
||||||
|
status "$POST_PATH" 200
|
||||||
|
body_has /posts 'Read the full post' "/posts offers the full post"
|
||||||
|
# And it trails the excerpt, immediately after the ellipsis the truncation
|
||||||
|
# left, rather than sitting as its own row below the media. The excerpt is
|
||||||
|
# escaped text, so nothing but the link can put a '<' between the two.
|
||||||
|
if curl -s "$BASE/posts" \
|
||||||
|
| grep -qE '<p class="post-card__excerpt">[^<]*<a class="link-more" href="/posts/'; then
|
||||||
|
ok "read-more trails the excerpt"
|
||||||
|
else
|
||||||
|
bad "read-more placement" "not inside the excerpt paragraph"
|
||||||
|
fi
|
||||||
|
body_has "$POST_PATH" '<div class="post-body">' "post page carries the rendered body"
|
||||||
|
# Rendered, not dumped: a body that reached the page as literal Markdown
|
||||||
|
# would show its own asterisks and hashes to the reader and to a crawler.
|
||||||
|
if curl -s "$BASE$POST_PATH" | grep -qE '<(p|h2|h3|h4|ul|ol|blockquote|pre)>'; then
|
||||||
|
ok "post body is real markup, not literal Markdown"
|
||||||
|
else
|
||||||
|
bad "post body markup" "no block elements found in the body"
|
||||||
|
fi
|
||||||
|
# The canonical points here, not at the instance. That is the entire SEO
|
||||||
|
# argument for hosting the body: two copies of the text exist, and this
|
||||||
|
# says which one is the original as far as this site is concerned.
|
||||||
|
body_has "$POST_PATH" 'rel="canonical" href="https://catcrafts.net/posts/' \
|
||||||
|
"post page is its own canonical"
|
||||||
|
body_has "$POST_PATH" '"@type":"BlogPosting"' "post page carries BlogPosting JSON-LD"
|
||||||
|
body_has "$POST_PATH" '"@id":"https://catcrafts.net/#organization"' \
|
||||||
|
"post JSON-LD joins the organization node"
|
||||||
|
body_has "$POST_PATH" '"@id":"https://catcrafts.net/about#person"' \
|
||||||
|
"post JSON-LD joins the founder node"
|
||||||
|
body_has "$POST_PATH" 'property="og:type" content="article"' "post page is an article to og:"
|
||||||
|
# Hosting the body does not mirror the discussion; the thread is still one
|
||||||
|
# click away and is still where the comments are.
|
||||||
|
if curl -s "$BASE$POST_PATH" | grep -qE 'href="https://[a-z0-9.-]+/post/[0-9]+"'; then
|
||||||
|
ok "post page still links its thread"
|
||||||
|
else
|
||||||
|
bad "post thread link" "no https://<instance>/post/<id> link on the post page"
|
||||||
|
fi
|
||||||
|
# The body is prose, not an application. Same rule as /projects.
|
||||||
|
body_lacks "$POST_PATH" '<script>' "post page ships no executable script"
|
||||||
|
body_lacks "$POST_PATH" '<base' "post page has no base tag"
|
||||||
|
# Every inline image and video the body embeds is mirrored, exactly like a
|
||||||
|
# card's media — the privacy notice's "everything comes from catcrafts.net"
|
||||||
|
# covers href as well as src, so a body linking a .webp on someone else's
|
||||||
|
# instance is the same leak as embedding one.
|
||||||
|
for pg in $(curl -s "$BASE/sitemap.xml" \
|
||||||
|
| grep -oE '/posts/[a-z0-9-]+' | head -n 20); do
|
||||||
|
if curl -s "$BASE$pg" \
|
||||||
|
| grep -qE '(src|srcset|href|poster)="https?://[^"]*\.(mp4|webm|webp|avif|png|jpe?g|gif)'; then
|
||||||
|
bad "$pg media origin" "body media loaded from a third party"
|
||||||
|
else
|
||||||
|
ok "$pg loads no media from a third party"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
# The image format ladder: AVIF first, the mirrored original next, and a
|
||||||
|
# PNG on the <img> underneath, so exactly one file is fetched and every
|
||||||
|
# browser can read one of them. Order is the whole point — a browser takes
|
||||||
|
# the first source it understands — so assert the sequence, not just that
|
||||||
|
# the pieces are present.
|
||||||
|
if curl -s "$BASE$POST_PATH" | grep -q '<picture>'; then
|
||||||
|
if curl -s "$BASE$POST_PATH" \
|
||||||
|
| grep -qE '<picture><source srcset="/media/[^"]+\.avif" type="image/avif">'; then
|
||||||
|
ok "inline images lead with an AVIF source"
|
||||||
|
else
|
||||||
|
bad "image ladder" "the first source is not the AVIF"
|
||||||
|
fi
|
||||||
|
if curl -s "$BASE$POST_PATH" \
|
||||||
|
| grep -qE '<img class="post-media__item"[^>]*src="/media/[^"]+\.png"'; then
|
||||||
|
ok "inline images fall back to a PNG the img itself points at"
|
||||||
|
else
|
||||||
|
bad "image fallback" "the <img> base is not a PNG"
|
||||||
|
fi
|
||||||
|
# Every tier has to be a file that exists, or the ladder serves a 404 to
|
||||||
|
# whichever browsers pick that rung — which is precisely the set of
|
||||||
|
# browsers nobody testing this site is using.
|
||||||
|
missing=0
|
||||||
|
for f in $(curl -s "$BASE/sitemap.xml" | grep -oE '/posts/[a-z0-9-]+' | head -n 20 \
|
||||||
|
| while read -r pg; do curl -s "$BASE$pg"; done \
|
||||||
|
| grep -oE '(src|srcset)="/media/[^"]+"' \
|
||||||
|
| sed 's/.*="//; s/"$//' | sort -u); do
|
||||||
|
[ -f "media${f#/media}" ] || { missing=$((missing + 1)); echo " missing: $f" >&2; }
|
||||||
|
done
|
||||||
|
if [ "$missing" -eq 0 ]; then
|
||||||
|
ok "every referenced media file is on the mount"
|
||||||
|
else
|
||||||
|
bad "media files" "$missing referenced file(s) not on the mount"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
skip "image format ladder" "no <picture> on this page — ffmpeg absent at mirror time?"
|
||||||
|
fi
|
||||||
|
# Inline screenshots get dimensions from the sidecar list fetch-media.sh
|
||||||
|
# writes, because Markdown syntax has nowhere to carry them — without it
|
||||||
|
# the prose below every image jumps as the file arrives. Conditional: a
|
||||||
|
# post whose body embeds nothing has nothing to check.
|
||||||
|
if curl -s "$BASE$POST_PATH" | grep -q '<img class="post-media__item"'; then
|
||||||
|
if curl -s "$BASE$POST_PATH" \
|
||||||
|
| grep -qE '<img class="post-media__item"[^>]*width="[0-9]+" height="[0-9]+"'; then
|
||||||
|
ok "inline body images carry width/height"
|
||||||
|
else
|
||||||
|
bad "inline image dimensions" "an embedded body image has no dimensions"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
# The sitemap has to advertise the pages, or hosting the bodies buys nothing.
|
||||||
|
if curl -s "$BASE/sitemap.xml" | grep -qE '<loc>https://catcrafts.net/posts/[a-z0-9-]+</loc>'; then
|
||||||
|
ok "sitemap lists the post pages"
|
||||||
|
else
|
||||||
|
bad "sitemap post pages" "no /posts/<slug> entry"
|
||||||
|
fi
|
||||||
|
|
||||||
echo "== headers =="
|
echo "== headers =="
|
||||||
header_has / 'x-content-type-options: *nosniff' "nosniff on pages"
|
header_has / 'x-content-type-options: *nosniff' "nosniff on pages"
|
||||||
header_has / 'cache-control: *public' "pages are cacheable"
|
header_has / 'cache-control: *public' "pages are cacheable"
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,21 @@
|
||||||
#
|
#
|
||||||
# Run AFTER tools/fetch-posts.sh, which records the original URLs.
|
# Run AFTER tools/fetch-posts.sh, which records the original URLs.
|
||||||
#
|
#
|
||||||
|
# TWO KINDS OF MEDIA, one pipeline:
|
||||||
|
#
|
||||||
|
# * a post's headline file — `.media`, the recording or screenshot the post is
|
||||||
|
# about;
|
||||||
|
# * everything embedded inside the body — the screenshots a post argues with,
|
||||||
|
# which are just as much content and, until the body was hosted here, were
|
||||||
|
# never fetched at all.
|
||||||
|
#
|
||||||
|
# Both are content-addressed into the same directory, so a file used as one
|
||||||
|
# post's headline and quoted inside another's body is stored once. Body files
|
||||||
|
# are rewritten IN THE MARKDOWN TEXT (the body is still Markdown at this point)
|
||||||
|
# and additionally recorded in `.body_media`, which is where the renderer reads
|
||||||
|
# the dimensions, poster and H.264 fallback that Markdown syntax has nowhere to
|
||||||
|
# carry.
|
||||||
|
#
|
||||||
# WHY MIRROR rather than embed from the source:
|
# WHY MIRROR rather than embed from the source:
|
||||||
#
|
#
|
||||||
# * Privacy. The privacy notice states that everything the browser loads comes
|
# * Privacy. The privacy notice states that everything the browser loads comes
|
||||||
|
|
@ -32,6 +47,21 @@ MEDIA_DIR="${1:-media}"
|
||||||
POSTS="content/posts.json"
|
POSTS="content/posts.json"
|
||||||
MAX_BYTES=$((64 * 1024 * 1024))
|
MAX_BYTES=$((64 * 1024 * 1024))
|
||||||
|
|
||||||
|
# What counts as a media reference inside a post body: an absolute URL or a path
|
||||||
|
# we have already rewritten, ending in a media extension.
|
||||||
|
#
|
||||||
|
# Local paths are in the pattern deliberately. Leaving them out looked right —
|
||||||
|
# nothing needs downloading twice — but it is what made a second run destructive
|
||||||
|
# rather than idempotent: the already-rewritten body references were not
|
||||||
|
# enumerated, so they never re-entered the mirror map, so the body_media sidecar
|
||||||
|
# came back with only the handful of entries that happened to still be absolute.
|
||||||
|
# Matching them means they are adopted from the mount and everything is rebuilt
|
||||||
|
# exactly as it was.
|
||||||
|
#
|
||||||
|
# One definition, passed to every jq that needs it, because three copies of a
|
||||||
|
# regex is three chances for one of them to drift.
|
||||||
|
MEDIA_REF_RE='(?:https?://|/media/)[^\s)\]"<>]+\.(?:mp4|webm|mov|webp|png|jpe?g|gif|avif)'
|
||||||
|
|
||||||
# Media we host ourselves, published by tools/publish-media.sh before the post
|
# Media we host ourselves, published by tools/publish-media.sh before the post
|
||||||
# that carries it exists. Such a URL is ALREADY the one the page should use, so
|
# that carries it exists. Such a URL is ALREADY the one the page should use, so
|
||||||
# there is nothing to fetch: the bytes are on the media mount, and downloading
|
# there is nothing to fetch: the bytes are on the media mount, and downloading
|
||||||
|
|
@ -54,6 +84,12 @@ mkdir -p "$MEDIA_DIR"
|
||||||
HAVE_FFPROBE=0
|
HAVE_FFPROBE=0
|
||||||
command -v ffprobe >/dev/null 2>&1 && HAVE_FFPROBE=1
|
command -v ffprobe >/dev/null 2>&1 && HAVE_FFPROBE=1
|
||||||
|
|
||||||
|
# ffmpeg does the still-image transcodes below. Optional in exactly the same
|
||||||
|
# way ffprobe is: without it every image is served as the single file the mirror
|
||||||
|
# downloaded, which is what this site did before the format ladder existed.
|
||||||
|
HAVE_FFMPEG=0
|
||||||
|
command -v ffmpeg >/dev/null 2>&1 && HAVE_FFMPEG=1
|
||||||
|
|
||||||
# Sets $w and $h for the file named in $1, or leaves both 0.
|
# Sets $w and $h for the file named in $1, or leaves both 0.
|
||||||
#
|
#
|
||||||
# One query per dimension. Asking for both at once and splitting the CSV looked
|
# One query per dimension. Asking for both at once and splitting the CSV looked
|
||||||
|
|
@ -95,24 +131,161 @@ probe_dims() {
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Derive the two renditions a mirrored image is served between: AVIF above it
|
||||||
|
# and PNG below. Sets $avif_name / $png_name to the sibling file names, or
|
||||||
|
# leaves one empty when that rendition could not be produced — :Media then drops
|
||||||
|
# the tier rather than pointing at a file that is not on the mount.
|
||||||
|
#
|
||||||
|
# Siblings are named after the source file, which is itself the hash of its
|
||||||
|
# bytes, so a rendition already present is never re-encoded and a changed source
|
||||||
|
# gets new names. Only genuinely new images cost encoder time; a rebuild costs
|
||||||
|
# none, which is what keeps this off the critical path of every deploy.
|
||||||
|
#
|
||||||
|
# WHY BOTH TIERS. AVIF is smaller than the WebP the instances serve (~15% on
|
||||||
|
# these screenshots, far more on photographs) and is what almost every visitor
|
||||||
|
# actually receives. PNG is lossless and universally understood, which is what
|
||||||
|
# makes it a fallback worth having — but it is also several times the size of
|
||||||
|
# the WebP beside it, so the <picture> offers the mirrored original in between
|
||||||
|
# and the PNG is reached only by a browser that understands neither of the
|
||||||
|
# other two.
|
||||||
|
#
|
||||||
|
# The settings, measured against these files rather than guessed:
|
||||||
|
# crf 26, cpu-used 6 SSIM 0.997 against the source and still smaller than
|
||||||
|
# it, at roughly half a second per image.
|
||||||
|
# yuv444p these are screenshots of text. Re-subsampling chroma
|
||||||
|
# that pict-rs already subsampled once fringes coloured
|
||||||
|
# text visibly, and full chroma costs about 3% here.
|
||||||
|
transcode_image() {
|
||||||
|
avif_name=""
|
||||||
|
png_name=""
|
||||||
|
_file="$1"
|
||||||
|
_name="$2"
|
||||||
|
_base="${_name%.*}"
|
||||||
|
|
||||||
|
# Already that format: serve the mirrored file as the tier rather than
|
||||||
|
# re-encoding it into a second copy of itself.
|
||||||
|
case "$_name" in *.avif) avif_name="$_name" ;; esac
|
||||||
|
case "$_name" in *.png) png_name="$_name" ;; esac
|
||||||
|
[ "$HAVE_FFMPEG" = 1 ] || return 0
|
||||||
|
|
||||||
|
# An animated source is not a still, and -frames:v 1 would silently freeze
|
||||||
|
# it. Leave it entirely alone: one moving GIF is worth more than three
|
||||||
|
# copies of its first frame. nb_frames is N/A for WebP, so the frames have
|
||||||
|
# to actually be counted — ~75 ms on a 3 MP image, once per new file.
|
||||||
|
_frames=$(ffprobe -v error -select_streams v:0 -count_frames \
|
||||||
|
-show_entries stream=nb_read_frames \
|
||||||
|
-of default=nw=1:nk=1 "$_file" 2>/dev/null | head -n1)
|
||||||
|
case "$_frames" in
|
||||||
|
''|*[!0-9]*|1) ;; # unknown or a single frame: a still
|
||||||
|
*) echo "fetch-media: $_name is animated, serving it as one file" >&2
|
||||||
|
return 0 ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# Alpha has to survive the transcode: an image with a transparent corner
|
||||||
|
# encoded into a format with no alpha plane gains an opaque black one.
|
||||||
|
_pixfmt=$(ffprobe -v error -select_streams v:0 -show_entries stream=pix_fmt \
|
||||||
|
-of default=nw=1:nk=1 "$_file" 2>/dev/null | head -n1)
|
||||||
|
case "$_pixfmt" in
|
||||||
|
yuva*|rgba*|bgra*|argb*|abgr*|gbrap*|ya8|ya16*|pal8) _avif_pix=yuva444p ;;
|
||||||
|
*) _avif_pix=yuv444p ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# Encoded to a .part, checked, and only then renamed — so an interrupted or
|
||||||
|
# wrong-format encode cannot leave a file the next run adopts as finished.
|
||||||
|
if [ -z "$avif_name" ]; then
|
||||||
|
_cand="$_base.avif"
|
||||||
|
if [ -f "$MEDIA_DIR/$_cand" ]; then
|
||||||
|
avif_name="$_cand"
|
||||||
|
elif encode_rendition "$_file" "$_cand" av1 \
|
||||||
|
-c:v libaom-av1 -still-picture 1 -crf 26 -cpu-used 6 \
|
||||||
|
-pix_fmt "$_avif_pix" -f avif; then
|
||||||
|
avif_name="$_cand"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z "$png_name" ]; then
|
||||||
|
_cand="$_base.png"
|
||||||
|
if [ -f "$MEDIA_DIR/$_cand" ]; then
|
||||||
|
png_name="$_cand"
|
||||||
|
elif encode_rendition "$_file" "$_cand" png -c:v png -f image2; then
|
||||||
|
png_name="$_cand"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# encode_rendition SRC OUTNAME EXPECTED_CODEC ffmpeg-args...
|
||||||
|
#
|
||||||
|
# Runs the encode into a .part, verifies the result really is the codec asked
|
||||||
|
# for, and only then publishes it. Returns non-zero (leaving nothing behind) if
|
||||||
|
# either step fails, which drops that tier rather than shipping a broken one.
|
||||||
|
#
|
||||||
|
# The verification is not paranoia. ffmpeg picks an encoder from the MUXER when
|
||||||
|
# one is not named, and the image2 muxer defaults to MJPEG — so `-f image2
|
||||||
|
# out.png` silently produced a run of lossy JPEGs sitting under .png names, which
|
||||||
|
# the page then advertised to browsers as image/png. The codec is pinned by the
|
||||||
|
# callers above; this is the check that the pin held.
|
||||||
|
encode_rendition() {
|
||||||
|
_src="$1"; _out="$2"; _want="$3"
|
||||||
|
shift 3
|
||||||
|
if ! ffmpeg -y -v error -i "$_src" -frames:v 1 "$@" \
|
||||||
|
"$MEDIA_DIR/$_out.part" 2>/dev/null; then
|
||||||
|
rm -f "$MEDIA_DIR/$_out.part"
|
||||||
|
echo "fetch-media: could not encode $_out, serving without that tier" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
_got=$(ffprobe -v error -select_streams v:0 -show_entries stream=codec_name \
|
||||||
|
-of default=nw=1:nk=1 "$MEDIA_DIR/$_out.part" 2>/dev/null | head -n1)
|
||||||
|
if [ "$_got" != "$_want" ]; then
|
||||||
|
rm -f "$MEDIA_DIR/$_out.part"
|
||||||
|
echo "fetch-media: $_out came out as '$_got', expected '$_want' — discarding it" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
mv "$MEDIA_DIR/$_out.part" "$MEDIA_DIR/$_out"
|
||||||
|
chmod 0644 "$MEDIA_DIR/$_out"
|
||||||
|
encoded=$((encoded + 1))
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
MAP="$(mktemp)"
|
MAP="$(mktemp)"
|
||||||
POSTERMAP="$(mktemp)"
|
POSTERMAP="$(mktemp)"
|
||||||
FALLBACKMAP="$(mktemp)"
|
FALLBACKMAP="$(mktemp)"
|
||||||
trap 'rm -f "$MAP" "$POSTERMAP" "$FALLBACKMAP"' EXIT
|
AVIFMAP="$(mktemp)"
|
||||||
|
POSTERONLY="$(mktemp)"
|
||||||
|
trap 'rm -f "$MAP" "$POSTERMAP" "$FALLBACKMAP" "$AVIFMAP" "$POSTERONLY"' EXIT
|
||||||
printf '[]' > "$MAP"
|
printf '[]' > "$MAP"
|
||||||
printf '[]' > "$POSTERMAP"
|
printf '[]' > "$POSTERMAP"
|
||||||
printf '[]' > "$FALLBACKMAP"
|
printf '[]' > "$FALLBACKMAP"
|
||||||
|
printf '[]' > "$AVIFMAP"
|
||||||
|
|
||||||
|
# URLs that are ONLY ever a video's poster frame. They are skipped by the
|
||||||
|
# transcode above, because `poster` takes exactly one URL: a <video> cannot
|
||||||
|
# negotiate a format the way <picture> can, so the renditions would be files
|
||||||
|
# nothing is able to ask for. A file that is a poster somewhere and an ordinary
|
||||||
|
# image somewhere else is not in this list and is transcoded normally.
|
||||||
|
jq -r --arg re "$MEDIA_REF_RE" \
|
||||||
|
'([.[].media[]? | .poster // empty] | map(select(. != "")) | unique) as $posters
|
||||||
|
| ([.[].media[]? | .src] + [.[] | .body // "" | scan($re)] | unique) as $srcs
|
||||||
|
| ($posters - $srcs) | .[]' "$POSTS" > "$POSTERONLY" 2>/dev/null || true
|
||||||
|
|
||||||
downloaded=0
|
downloaded=0
|
||||||
reused=0
|
reused=0
|
||||||
adopted=0
|
adopted=0
|
||||||
failed=0
|
failed=0
|
||||||
|
encoded=0
|
||||||
|
|
||||||
# Every distinct media URL across all posts, so a file shared by two posts is
|
# Every distinct media URL across all posts, so a file shared by two posts is
|
||||||
# fetched once. Video posters are in here too: a poster left pointing at the
|
# fetched once. Video posters are in here too: a poster left pointing at the
|
||||||
# source instance would leak a visitor IP on page load exactly like an embedded
|
# source instance would leak a visitor IP on page load exactly like an embedded
|
||||||
# image would, and it is the frame shown before anyone presses play.
|
# image would, and it is the frame shown before anyone presses play.
|
||||||
#
|
#
|
||||||
|
# Body URLs are found by pattern rather than by parsing Markdown: anything that
|
||||||
|
# looks like an absolute URL ending in a media extension is mirrored, whether it
|
||||||
|
# was written as an embed, as a link, or bare. That is deliberately wider than
|
||||||
|
# "images the body displays" — the origin rule covers href as well as src, and a
|
||||||
|
# link whose target is a .webp on someone else's instance is still a third-party
|
||||||
|
# address on our page. Already-rewritten paths start with /media/ and so do not
|
||||||
|
# match, which is what makes re-running this a no-op.
|
||||||
|
#
|
||||||
# Fed by a here-document rather than a pipe so the counters below survive — in
|
# Fed by a here-document rather than a pipe so the counters below survive — in
|
||||||
# `jq | while`, the loop runs in a subshell and every increment is discarded.
|
# `jq | while`, the loop runs in a subshell and every increment is discarded.
|
||||||
while IFS= read -r src; do
|
while IFS= read -r src; do
|
||||||
|
|
@ -152,6 +325,26 @@ while IFS= read -r src; do
|
||||||
fi
|
fi
|
||||||
adopted=$((adopted + 1))
|
adopted=$((adopted + 1))
|
||||||
;;
|
;;
|
||||||
|
/media/*)
|
||||||
|
# Already rewritten by an earlier run of this script. Adopt the file on
|
||||||
|
# the mount rather than trying to fetch our own path as though it were a
|
||||||
|
# URL — which is what makes running this twice a no-op instead of a way
|
||||||
|
# to lose every rewrite it made the first time. The script is meant to
|
||||||
|
# follow fetch-posts.sh, but "meant to" is not a guarantee, and the
|
||||||
|
# failure was silent: the body_media list simply came back empty.
|
||||||
|
name=${src#/media/}
|
||||||
|
case "$name" in
|
||||||
|
''|*/*|*..*)
|
||||||
|
echo "fetch-media: refusing suspicious local path: $src" >&2
|
||||||
|
failed=$((failed + 1)); continue ;;
|
||||||
|
esac
|
||||||
|
dest="$MEDIA_DIR/$name"
|
||||||
|
if [ ! -f "$dest" ]; then
|
||||||
|
echo "fetch-media: $name not on the media mount, leaving it alone: $src" >&2
|
||||||
|
failed=$((failed + 1)); continue
|
||||||
|
fi
|
||||||
|
adopted=$((adopted + 1))
|
||||||
|
;;
|
||||||
*)
|
*)
|
||||||
ext=$(printf '%s' "$src" | sed -E 's/.*\.([A-Za-z0-9]+)$/\1/' | tr 'A-Z' 'a-z')
|
ext=$(printf '%s' "$src" | sed -E 's/.*\.([A-Za-z0-9]+)$/\1/' | tr 'A-Z' 'a-z')
|
||||||
case "$ext" in
|
case "$ext" in
|
||||||
|
|
@ -226,6 +419,26 @@ while IFS= read -r src; do
|
||||||
&& mv "$FALLBACKMAP.new" "$FALLBACKMAP"
|
&& mv "$FALLBACKMAP.new" "$FALLBACKMAP"
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
|
*.webp|*.png|*.jpg|*.jpeg|*.gif|*.avif)
|
||||||
|
# The AVIF and PNG tiers this image is served between. Both maps are
|
||||||
|
# keyed by the LOCAL path, like the video ones above, so the rewrite
|
||||||
|
# below can look them up from the src it has just written.
|
||||||
|
if grep -qxF "$src" "$POSTERONLY" 2>/dev/null; then
|
||||||
|
: # poster-only; see POSTERONLY above
|
||||||
|
else
|
||||||
|
transcode_image "$dest" "$name"
|
||||||
|
if [ -n "$avif_name" ]; then
|
||||||
|
jq --arg k "/media/$name" --arg v "/media/$avif_name" \
|
||||||
|
'. + [{key: $k, value: $v}]' "$AVIFMAP" > "$AVIFMAP.new" \
|
||||||
|
&& mv "$AVIFMAP.new" "$AVIFMAP"
|
||||||
|
fi
|
||||||
|
if [ -n "$png_name" ]; then
|
||||||
|
jq --arg k "/media/$name" --arg v "/media/$png_name" \
|
||||||
|
'. + [{key: $k, value: $v}]' "$FALLBACKMAP" > "$FALLBACKMAP.new" \
|
||||||
|
&& mv "$FALLBACKMAP.new" "$FALLBACKMAP"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
jq --arg src "$src" --arg path "/media/$name" \
|
jq --arg src "$src" --arg path "/media/$name" \
|
||||||
|
|
@ -233,20 +446,25 @@ while IFS= read -r src; do
|
||||||
'. + [{src: $src, path: $path, w: $w, h: $h}]' "$MAP" > "$MAP.new" \
|
'. + [{src: $src, path: $path, w: $w, h: $h}]' "$MAP" > "$MAP.new" \
|
||||||
&& mv "$MAP.new" "$MAP"
|
&& mv "$MAP.new" "$MAP"
|
||||||
done <<EOF
|
done <<EOF
|
||||||
$(jq -r '[.[].media[]? | .src, (.poster // empty)] | map(select(. != "")) | unique[]' "$POSTS")
|
$(jq -r --arg re "$MEDIA_REF_RE" \
|
||||||
|
'[ (.[].media[]? | .src, (.poster // empty)),
|
||||||
|
(.[] | .body // "" | scan($re)) ]
|
||||||
|
| map(select(. != "")) | unique[]' "$POSTS")
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
echo "fetch-media: $downloaded new, $reused already present, $adopted self-hosted, $failed failed"
|
echo "fetch-media: $downloaded new, $reused already present, $adopted self-hosted, $failed failed"
|
||||||
|
echo "fetch-media: $encoded image rendition(s) encoded this run"
|
||||||
|
|
||||||
# Rewrite each media entry to the local path. An entry with no mapping (download
|
# Rewrite each media entry to the local path. An entry with no mapping (download
|
||||||
# failed) keeps its original src, so the page still shows something rather than
|
# failed) keeps its original src, so the page still shows something rather than
|
||||||
# silently dropping the post's whole point.
|
# silently dropping the post's whole point.
|
||||||
TMP_POSTS="$(mktemp)"
|
TMP_POSTS="$(mktemp)"
|
||||||
if jq --slurpfile map "$MAP" --slurpfile posters "$POSTERMAP" \
|
if jq --slurpfile map "$MAP" --slurpfile posters "$POSTERMAP" \
|
||||||
--slurpfile fallbacks "$FALLBACKMAP" '
|
--slurpfile fallbacks "$FALLBACKMAP" --slurpfile avifs "$AVIFMAP" '
|
||||||
($map[0] | map({key: .src, value: .}) | from_entries) as $m
|
($map[0] | map({key: .src, value: .}) | from_entries) as $m
|
||||||
| ($posters[0] | from_entries) as $pm
|
| ($posters[0] | from_entries) as $pm
|
||||||
| ($fallbacks[0] | from_entries) as $fm
|
| ($fallbacks[0] | from_entries) as $fm
|
||||||
|
| ($avifs[0] | from_entries) as $am
|
||||||
| map(.media = ((.media // []) | map(
|
| map(.media = ((.media // []) | map(
|
||||||
. as $item
|
. as $item
|
||||||
| ($m[$item.src] // null) as $hit
|
| ($m[$item.src] // null) as $hit
|
||||||
|
|
@ -271,7 +489,42 @@ if jq --slurpfile map "$MAP" --slurpfile posters "$POSTERMAP" \
|
||||||
# sibling to find.
|
# sibling to find.
|
||||||
| if (($fm[.src] // "") != "")
|
| if (($fm[.src] // "") != "")
|
||||||
then . + { fallback: $fm[.src] }
|
then . + { fallback: $fm[.src] }
|
||||||
|
else . end
|
||||||
|
# The AVIF tier, keyed by the LOCAL src like the two maps above. Only
|
||||||
|
# ever set for images, and only when the encode actually produced one.
|
||||||
|
| if (($am[.src] // "") != "")
|
||||||
|
then . + { avif: $am[.src] }
|
||||||
else . end)))
|
else . end)))
|
||||||
|
# ── the body ──────────────────────────────────────────────────────
|
||||||
|
#
|
||||||
|
# Substitution is literal (split/join, not gsub), because these URLs are
|
||||||
|
# full of regex metacharacters and a mirrored path must land in the text
|
||||||
|
# exactly as written. A URL with no mapping — its download failed — is left
|
||||||
|
# alone, so the post still shows the image rather than losing it; the
|
||||||
|
# accounting at the end of this script reports that as media still pointing
|
||||||
|
# at its source.
|
||||||
|
| ($m | to_entries) as $subs
|
||||||
|
| map(.body = (reduce $subs[] as $s ((.body // "");
|
||||||
|
split($s.key) | join($s.value.path))))
|
||||||
|
# Everything the rewritten body now points at, as records the renderer can
|
||||||
|
# read: Markdown has nowhere to put a width, a poster frame or a second
|
||||||
|
# source, so the sidecar list is how an inline video gets the same treatment
|
||||||
|
# as a headline one. Keyed by the LOCAL path, which is what the body says
|
||||||
|
# by this point.
|
||||||
|
| ($m | map({ key: .path, value: . }) | from_entries) as $byPath
|
||||||
|
| map(.body_media = ([ (.body // "")
|
||||||
|
| scan("/media/[A-Za-z0-9._-]+")
|
||||||
|
| . as $path
|
||||||
|
| select($byPath[$path] != null)
|
||||||
|
| { src: $path,
|
||||||
|
kind: (if ($path | test("\\.(?:mp4|webm|mov)$"))
|
||||||
|
then "video" else "image" end),
|
||||||
|
poster: ($pm[$path] // ""),
|
||||||
|
fallback: ($fm[$path] // ""),
|
||||||
|
avif: ($am[$path] // ""),
|
||||||
|
w: ($byPath[$path].w // 0),
|
||||||
|
h: ($byPath[$path].h // 0) } ]
|
||||||
|
| unique_by(.src)))
|
||||||
' "$POSTS" > "$TMP_POSTS" 2>/dev/null; then
|
' "$POSTS" > "$TMP_POSTS" 2>/dev/null; then
|
||||||
# Same reason as the chmod on each mirrored file: mktemp is 0600 and the
|
# Same reason as the chmod on each mirrored file: mktemp is 0600 and the
|
||||||
# mode survives to production, where other users must read this.
|
# mode survives to production, where other users must read this.
|
||||||
|
|
@ -283,10 +536,18 @@ else
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
total=$(jq '[.[].media[]? | .src, (.poster // empty) | select(. != "")] | length' "$POSTS")
|
# Counted over the bodies too, because a body URL that never mirrored is the
|
||||||
local_count=$(jq '[.[].media[]? | .src, (.poster // empty)
|
# same privacy leak as a card one and must not be reported as success.
|
||||||
| select(startswith("/media/"))] | length' "$POSTS")
|
total=$(jq --arg re "$MEDIA_REF_RE" \
|
||||||
echo "fetch-media: $local_count of $total media entries served locally ($(du -sh "$MEDIA_DIR" | cut -f1) in $MEDIA_DIR)"
|
'[ (.[].media[]? | .src, (.poster // empty)),
|
||||||
|
(.[] | .body // "" | scan($re)),
|
||||||
|
(.[].body_media[]? | .src, (.poster // empty)) ]
|
||||||
|
| map(select(. != "")) | unique | length' "$POSTS")
|
||||||
|
local_count=$(jq '[ (.[].media[]? | .src, (.poster // empty)),
|
||||||
|
(.[].body_media[]? | .src, (.poster // empty)) ]
|
||||||
|
| map(select(startswith("/media/"))) | length' "$POSTS")
|
||||||
|
inline=$(jq '[.[].body_media[]?] | length' "$POSTS")
|
||||||
|
echo "fetch-media: $local_count of $total media entries served locally ($inline of them embedded in post bodies; $(du -sh "$MEDIA_DIR" | cut -f1) in $MEDIA_DIR)"
|
||||||
if [ "$local_count" -ne "$total" ]; then
|
if [ "$local_count" -ne "$total" ]; then
|
||||||
echo "fetch-media: $((total - local_count)) still point at their source — see the failures above" >&2
|
echo "fetch-media: $((total - local_count)) still point at their source — see the failures above" >&2
|
||||||
fi
|
fi
|
||||||
|
|
|
||||||
|
|
@ -28,6 +28,12 @@ CONFIG="${1:-content/posts-sources.json}"
|
||||||
OUT="content/posts.json"
|
OUT="content/posts.json"
|
||||||
LIMIT=50
|
LIMIT=50
|
||||||
EXCERPT_CHARS=280
|
EXCERPT_CHARS=280
|
||||||
|
# A ceiling, not a target: the longest of these bodies is around 6 KB and this
|
||||||
|
# file is fetched into memory before the wasm module starts, so an outlier must
|
||||||
|
# not be able to grow the bundle without bound. A truncated body renders as
|
||||||
|
# truncated prose (the Markdown renderer tolerates an unclosed fence), which is
|
||||||
|
# a better failure than a build that silently ships a megabyte.
|
||||||
|
MAX_BODY_CHARS=32768
|
||||||
|
|
||||||
command -v jq >/dev/null 2>&1 || { echo "fetch-posts: jq not found, keeping existing $OUT" >&2; exit 0; }
|
command -v jq >/dev/null 2>&1 || { echo "fetch-posts: jq not found, keeping existing $OUT" >&2; exit 0; }
|
||||||
[ -f "$CONFIG" ] || { echo "fetch-posts: $CONFIG not found, keeping existing $OUT" >&2; exit 0; }
|
[ -f "$CONFIG" ] || { echo "fetch-posts: $CONFIG not found, keeping existing $OUT" >&2; exit 0; }
|
||||||
|
|
@ -84,13 +90,39 @@ fi
|
||||||
# Both are ORIGINAL urls here; tools/fetch-media.sh mirrors them
|
# Both are ORIGINAL urls here; tools/fetch-media.sh mirrors them
|
||||||
# and rewrites to local paths, so nothing the browser loads is
|
# and rewrites to local paths, so nothing the browser loads is
|
||||||
# third-party.
|
# third-party.
|
||||||
# excerpt : body flattened to one line and truncated. Markdown is NOT
|
# excerpt : body flattened to one line and truncated, for the card and for
|
||||||
# rendered — the site has no markdown pipeline by design, so any
|
# the meta description. Markdown markers are stripped rather than
|
||||||
# surviving syntax would show as literal characters. Strip the
|
# rendered — this string lands in places that are plain text by
|
||||||
# common inline markers and let the rest be plain text.
|
# definition (<meta name="description">, og:description), so any
|
||||||
|
# surviving syntax would show as literal characters.
|
||||||
|
# body : the post, whole, still as Markdown. Rendered by
|
||||||
|
# Catcrafts.Shared:Markdown at page-render time rather than
|
||||||
|
# converted here, because that module is inside the escaping
|
||||||
|
# guarantee and a shell script writing HTML into a content file
|
||||||
|
# would not be — text from someone else's server must not be able
|
||||||
|
# to become markup anywhere but there.
|
||||||
|
# slug : the post's own URL at /posts/<slug>, from the title. Collisions
|
||||||
|
# take the post's numeric id as a suffix rather than a positional
|
||||||
|
# one: a later post sharing a title would otherwise renumber an
|
||||||
|
# earlier post's URL out from under everyone who linked it.
|
||||||
# deleted / removed posts are dropped rather than rendered as empty cards.
|
# deleted / removed posts are dropped rather than rendered as empty cards.
|
||||||
if ! jq --argjson n "$EXCERPT_CHARS" \
|
if ! jq --argjson n "$EXCERPT_CHARS" \
|
||||||
|
--argjson maxbody "$MAX_BODY_CHARS" \
|
||||||
--slurpfile cfg "$CONFIG" '
|
--slurpfile cfg "$CONFIG" '
|
||||||
|
# Catcrafts.Shared:Route::IsValidSlug is the contract: lowercase ASCII,
|
||||||
|
# digits and single hyphens, no leading or trailing hyphen, at most 64
|
||||||
|
# characters. A slug that fails it is dropped by the loader, which costs
|
||||||
|
# the post its page — so the shape is produced correctly here rather than
|
||||||
|
# sanitised on the way out.
|
||||||
|
def slugify:
|
||||||
|
ascii_downcase
|
||||||
|
| gsub("[^a-z0-9]+"; "-")
|
||||||
|
| sub("^-+"; "") | sub("-+$"; "");
|
||||||
|
# Truncate on a word boundary where there is one: a slug cut mid-word reads
|
||||||
|
# like a typo, and these titles are long enough to hit the limit.
|
||||||
|
def clamp($n):
|
||||||
|
(if (length > $n) then (.[0:$n] | sub("-[^-]*$"; "")) else . end)
|
||||||
|
| sub("-+$"; "");
|
||||||
($cfg[0].communities | map(ascii_downcase)) as $allow
|
($cfg[0].communities | map(ascii_downcase)) as $allow
|
||||||
| [ .posts[]
|
| [ .posts[]
|
||||||
| select((.post.deleted // false) == false)
|
| select((.post.deleted // false) == false)
|
||||||
|
|
@ -101,6 +133,11 @@ if ! jq --argjson n "$EXCERPT_CHARS" \
|
||||||
| select(($comm | ascii_downcase) as $c | $allow | index($c))
|
| select(($comm | ascii_downcase) as $c | $allow | index($c))
|
||||||
| {
|
| {
|
||||||
title: ($p.post.name // ""),
|
title: ($p.post.name // ""),
|
||||||
|
slug: (($p.post.name // "") | slugify | clamp(64)),
|
||||||
|
# Carried only as far as the de-duplication pass below, which strips
|
||||||
|
# it: it is a tie-breaker, not content.
|
||||||
|
uid: (($p.post.ap_id // "") | sub(".*/"; "") | ascii_downcase
|
||||||
|
| gsub("[^a-z0-9]"; "") | .[0:12]),
|
||||||
permalink: ($p.post.ap_id // ""),
|
permalink: ($p.post.ap_id // ""),
|
||||||
# A link post whose target IS an image or video is a media post, not a
|
# A link post whose target IS an image or video is a media post, not a
|
||||||
# link post: the file is captured in `media` and embedded, so keeping
|
# link post: the file is captured in `media` and embedded, so keeping
|
||||||
|
|
@ -123,6 +160,10 @@ if ! jq --argjson n "$EXCERPT_CHARS" \
|
||||||
| gsub(" +"; " ")
|
| gsub(" +"; " ")
|
||||||
| ltrimstr(" ") | rtrimstr(" ")
|
| ltrimstr(" ") | rtrimstr(" ")
|
||||||
| if (. | length) > $n then (.[0:$n] | sub(" [^ ]*$"; "")) + "…" else . end),
|
| if (. | length) > $n then (.[0:$n] | sub(" [^ ]*$"; "")) + "…" else . end),
|
||||||
|
# Verbatim apart from CR removal (a stray \r would render as a stray
|
||||||
|
# character inside a code block, where nothing is interpreted) and the
|
||||||
|
# size ceiling.
|
||||||
|
body: (($p.post.body // "") | gsub("\r"; "") | .[0:$maxbody]),
|
||||||
media: ([ ($p.post.url // "")
|
media: ([ ($p.post.url // "")
|
||||||
| select(test("\\.(?:mp4|webm|mov|webp|png|jpe?g|gif|avif)$"))
|
| select(test("\\.(?:mp4|webm|mov|webp|png|jpe?g|gif|avif)$"))
|
||||||
| (if test("\\.(mp4|webm|mov)$") then "video" else "image" end) as $kind
|
| (if test("\\.(mp4|webm|mov)$") then "video" else "image" end) as $kind
|
||||||
|
|
@ -139,7 +180,26 @@ if ! jq --argjson n "$EXCERPT_CHARS" \
|
||||||
score: ($p.counts.score // 0),
|
score: ($p.counts.score // 0),
|
||||||
comments: ($p.counts.comments // 0)
|
comments: ($p.counts.comments // 0)
|
||||||
}
|
}
|
||||||
]' "$RAW" > "$TMP" 2>/dev/null; then
|
]
|
||||||
|
# Make every slug unique, and keep it that way across builds.
|
||||||
|
#
|
||||||
|
# Only a duplicate is suffixed, so the ordinary post keeps the clean URL the
|
||||||
|
# title earned. The suffix is the numeric id of the post itself rather than
|
||||||
|
# a counter, because a counter is positional: a new post repeating an older
|
||||||
|
# title arrives at the front of this newest-first list, takes the bare slug,
|
||||||
|
# and silently renumbers the older one — breaking a URL that is already in
|
||||||
|
# search results and in whatever links people have shared.
|
||||||
|
# (Note for editors: this jq program is inside a single-quoted shell string,
|
||||||
|
# so no apostrophes anywhere in it.)
|
||||||
|
| reduce .[] as $e ({ seen: {}, out: [] };
|
||||||
|
(if ($e.slug | length) == 0 then "post" else $e.slug end) as $base
|
||||||
|
| (if (.seen[$base] // false)
|
||||||
|
then (($base | clamp(50)) + "-"
|
||||||
|
+ (if ($e.uid | length) == 0 then "x" else $e.uid end))
|
||||||
|
else $base end) as $slug
|
||||||
|
| { seen: (.seen + { ($base): true, ($slug): true }),
|
||||||
|
out: (.out + [ ($e | del(.uid)) + { slug: $slug } ]) })
|
||||||
|
| .out' "$RAW" > "$TMP" 2>/dev/null; then
|
||||||
echo "fetch-posts: response did not match the expected shape, keeping existing $OUT" >&2
|
echo "fetch-posts: response did not match the expected shape, keeping existing $OUT" >&2
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue