media: H.264 fallback beside every AV1, and the post links the H.264

The AV1-only publish lasted four hours in the wild: the first viewer on
mobile Safari got "bad media error" from the raw file, because a post's
link is fetched raw — Lemmy apps and browsers play that exact URL, with
no <source> negotiation in front of it. The previous commit's note
("--raw ... if that audience matters") had it backwards: the audience
that cannot play AV1 is not a per-post judgement call, it is whoever
happens to open the thread on an iPhone.

So publish-media.sh now emits two encodings and points the post at the
compatible one:

  * Every video transcode also produces <hash>.h264.mp4 (x264 crf 23,
    same denoise, same frames), uploaded beside the AV1 under the AV1's
    hash — the poster's sibling-naming trick, reused, so fetch-media.sh
    finds it by name with nothing to look up.
  * The printed URL to paste into the post is the H.264 one. pict-rs
    can thumbnail it too, so instance thumbnails come back as a bonus.

fetch-media.sh adopting an own-origin .h264.mp4 URL swaps the AV1 back
in as the page's primary when it is on the mount, and records the H.264
as `fallback` in posts.json. The renderer turns a non-empty fallback
into a <source> pair: the AV1 first with an explicit codecs parameter —
both files are video/mp4, so the parameter is the only thing that lets
a non-AV1 browser skip to the file it can play — and the H.264 second.
Browsers with AV1 keep downloading the small file; Safari before 17
gets one that plays instead of an element that will not.

e2e gains the matching conditional gate: a page offering an av01
<source> must offer an .h264.mp4 one, so an AV1 video can never again
ship without its fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

deploy/README.md

@@ -132,8 +132,13 @@ pointing at its original URL rather than losing the post.
 
 **The recommended flow is to put a recording on catcrafts.net before writing the
 post, and use that URL as the post's link.** Run `tools/publish-media.sh
-recording.mp4`; it transcodes, uploads to the media mount under its content hash,
-and prints a `https://catcrafts.net/media/<hash>.mp4` URL to paste into the post.
+recording.mp4`; it transcodes to AV1 *and* an H.264 sibling, uploads both to the
+media mount under the AV1's content hash, and prints a
+`https://catcrafts.net/media/<hash>.h264.mp4` URL to paste into the post. The
+H.264 one on purpose: a post's link is fetched raw — Lemmy apps and browsers play
+that exact file, with no negotiation in front of it — so it must be the encoding
+everything can play. (An AV1 link posted before this existed drew "bad media
+error" reports from iPhones within hours.)
 
 `fetch-media.sh` recognises its own origin and **adopts** such a URL: it rewrites
 it to `/media/<hash>` and probes the local file for dimensions, downloading
@@ -143,6 +148,12 @@ file host once hit `MAX_BYTES` (64 MB), so the entry kept its original URL, and
 the deploy then failed the e2e "media origin" check on a file that was sitting on
 our own disk the entire time.
 
+Adopting a `<hash>.h264.mp4` URL swaps the AV1 sibling back in as the page's
+primary `<source>` when it is on the mount, keeping the H.264 as the fallback
+`<source>`. So the post links the compatible file, while browsers that can take
+AV1 download the small one — the `codecs` parameter on the first source is what
+lets the rest skip it.
+
 The transcode matters as much as the hosting. Phone recordings are wildly
 oversized for what they show — that same 167 MB clip was 78 s of a dark room at
 17 Mbps, and denoising into AV1 gives the same picture in 15 MB. It also **bakes
@@ -154,15 +165,18 @@ correct too — but transcoding means nothing downstream has to know.)
 
 Two things to know about publishing AV1:
 
-* Nothing emits a fallback encoding and `<video>` carries a single `src`, so
-  browsers without AV1 (Safari before 17, Apple hardware older than A17/M3) get
-  an element that will not play. Use `publish-media.sh --raw`, or a H.264
-  re-encode, when that matters for a particular post.
+* Browsers without AV1 (Safari before 17, Apple hardware older than A17/M3) get
+  the `<hash>.h264.mp4` sibling: on the site via the second `<source>`, on the
+  fediverse because that sibling *is* the posted URL. `--raw` skips the
+  transcode and the fallback both, so a raw AV1 upload recreates the
+  will-not-play problem — use it for files that are already universally
+  playable.
 * Lemmy's `pict-rs` will not generate a thumbnail from an AV1 file, so a
   self-hosted video usually arrives with no `poster`. `publish-media.sh` uploads
   a poster frame beside the video, named `<video-hash>.poster.webp`, and
   `fetch-media.sh` falls back to that sibling when the instance supplied nothing.
-  A thumbnail the instance *did* provide always wins.
+  A thumbnail the instance *did* provide always wins. (Posting the H.264 URL
+  also means pict-rs can thumbnail it again, so instance thumbnails come back.)
 
 An own-origin URL naming a file that is **not** on the mount is deliberately left
 pointing at its original URL rather than rewritten. That is a post published

shared/interfaces/Catcrafts.Shared-Model.cppm

@@ -50,6 +50,12 @@ export struct PostMedia {
     // play — and these posts are their video, so the black box is the page.
     // Empty for images, and empty when the instance generated no thumbnail.
     std::string poster;
+    // H.264 rendition of an AV1 video, when tools/publish-media.sh uploaded one
+    // beside it. Non-empty means the renderer emits a <source> pair instead of
+    // a bare src, so a browser without AV1 (Safari before 17, Apple hardware
+    // without the decoder) gets a file it can play. Empty for images and for
+    // videos that need no fallback.
+    std::string fallback;
     std::int64_t width = 0;
     std::int64_t height = 0;
 };
@@ -311,6 +317,7 @@ export std::vector<Post> LoadPosts(std::string_view json) {
                 pm.src    = std::string(mv.Str("src"));
                 pm.kind   = std::string(mv.Str("kind", "image"));
                 pm.poster = std::string(mv.Str("poster"));
+                pm.fallback = std::string(mv.Str("fallback"));
                 pm.width  = mv.Int("w");
                 pm.height = mv.Int("h");
                 // Only the two kinds the renderer knows how to emit. Anything

shared/interfaces/Catcrafts.Shared-Views.cppm

@@ -353,10 +353,29 @@ SafeHtml RenderPostMedia(std::span<const PostMedia> media) {
             : SafeHtml{};
         if (m.kind == "video") {
             SafeHtml poster = m.poster.empty() ? SafeHtml{} : Url("poster", m.poster);
-            items.push_back(Format(
-                R"(<video class="post-media__item" controls preload="metadata" )"
-                R"(playsinline{}{}{}></video>)",
-                Url("src", m.src), poster, dims));
+            if (m.fallback.empty() || m.fallback == m.src) {
+                items.push_back(Format(
+                    R"(<video class="post-media__item" controls preload="metadata" )"
+                    R"(playsinline{}{}{}></video>)",
+                    Url("src", m.src), poster, dims));
+            } else {
+                // An AV1 video with its H.264 rendition. Both are .mp4, so the
+                // container alone cannot tell them apart: the codecs parameter
+                // on the first <source> is what lets a browser without AV1
+                // (Safari before 17, Apple hardware without the decoder) skip
+                // it and take the H.264 instead of failing on a file it cannot
+                // decode. The string is advisory and used only for selection —
+                // once a source is picked the browser reads the actual stream —
+                // so the canonical profile-0 8-bit form is right for anything
+                // publish-media.sh emits (yuv420p is pinned there).
+                items.push_back(Format(
+                    R"(<video class="post-media__item" controls preload="metadata" )"
+                    R"(playsinline{}{}>)"
+                    R"(<source{} type="video/mp4; codecs=av01.0.08M.08">)"
+                    R"(<source{} type="video/mp4">)"
+                    R"(</video>)",
+                    poster, dims, Url("src", m.src), Url("src", m.fallback)));
+            }
         } else {
             // alt is empty and aria-hidden is absent on purpose: these are
             // screenshots whose meaning is already in the post title and

tools/e2e.sh

@@ -383,6 +383,18 @@ if curl -s "$BASE/posts" | grep -qE '<video class="post-media__item"[^>]*poster=
 else
     bad "video poster" "no video has a poster; a black box shows until play"
 fi
+# A video offering an AV1 <source> must offer an H.264 one after it: the codecs
+# parameter is what lets a browser without AV1 (Safari before 17, Apple hardware
+# without the decoder) skip to a file it can play, and an AV1 source alone is
+# exactly the "element that will not play" the fallback pipeline exists to
+# prevent. Conditional — a build whose posts carry no AV1 has nothing to check.
+if curl -s "$BASE/posts" | grep -q 'codecs=av01'; then
+    if curl -s "$BASE/posts" | grep -qE '<source src="/media/[^"]*\.h264\.mp4" type="video/mp4">'; then
+        ok "AV1 videos carry an H.264 fallback source"
+    else
+        bad "video fallback" "an av01 <source> has no h264 sibling"
+    fi
+fi
 # preload="metadata", not auto: several 5 MB recordings must not all download on
 # page load.
 if curl -s "$BASE/posts" | grep -q 'preload="metadata"'; then

tools/fetch-media.sh

@@ -97,9 +97,11 @@ probe_dims() {
 
 MAP="$(mktemp)"
 POSTERMAP="$(mktemp)"
-trap 'rm -f "$MAP" "$POSTERMAP"' EXIT
+FALLBACKMAP="$(mktemp)"
+trap 'rm -f "$MAP" "$POSTERMAP" "$FALLBACKMAP"' EXIT
 printf '[]' > "$MAP"
 printf '[]' > "$POSTERMAP"
+printf '[]' > "$FALLBACKMAP"
 
 downloaded=0
 reused=0
@@ -132,6 +134,17 @@ while IFS= read -r src; do
                 echo "fetch-media: refusing suspicious own-origin URL: $src" >&2
                 failed=$((failed + 1)); continue ;;
         esac
+        # A .h264.mp4 URL is the fediverse-facing form of an AV1 video: the
+        # post links the encoding everything can play, because that link is
+        # fetched raw by Lemmy apps. Our page can negotiate, so when the AV1
+        # sibling is on the mount it becomes the primary <source> and the
+        # H.264 drops to the fallback (picked up by name below).
+        case "$name" in
+            *.h264.mp4)
+                av1="${name%.h264.mp4}.mp4"
+                [ -f "$MEDIA_DIR/$av1" ] && name="$av1"
+                ;;
+        esac
         dest="$MEDIA_DIR/$name"
         if [ ! -f "$dest" ]; then
             echo "fetch-media: $name not on the media mount, keeping original URL: $src" >&2
@@ -189,12 +202,29 @@ while IFS= read -r src; do
     # box until someone presses play.
     case "$name" in
         *.mp4|*.webm|*.mov)
-            sibling="${name%.*}.poster.webp"
+            # Siblings are named after the AV1's hash, so strip the .h264
+            # marker too — a video adopted as X.h264.mp4 (its AV1 never
+            # published) still finds X.poster.webp.
+            base="${name%.*}"
+            base="${base%.h264}"
+            sibling="$base.poster.webp"
             if [ -f "$MEDIA_DIR/$sibling" ]; then
                 jq --arg k "/media/$name" --arg v "/media/$sibling" \
                    '. + [{key: $k, value: $v}]' "$POSTERMAP" > "$POSTERMAP.new" \
                    && mv "$POSTERMAP.new" "$POSTERMAP"
             fi
+            # The H.264 sibling publish-media.sh uploaded next to the AV1.
+            # Rendered as a second <source> so browsers without AV1 (Safari
+            # before 17, Apple hardware without the decoder) get a file they
+            # can play instead of an element that will not. The guard against
+            # naming itself covers a video adopted as X.h264.mp4 whose AV1 is
+            # not on the mount.
+            fb="$base.h264.mp4"
+            if [ "$fb" != "$name" ] && [ -f "$MEDIA_DIR/$fb" ]; then
+                jq --arg k "/media/$name" --arg v "/media/$fb" \
+                   '. + [{key: $k, value: $v}]' "$FALLBACKMAP" > "$FALLBACKMAP.new" \
+                   && mv "$FALLBACKMAP.new" "$FALLBACKMAP"
+            fi
             ;;
     esac
 
@@ -212,9 +242,11 @@ echo "fetch-media: $downloaded new, $reused already present, $adopted self-hoste
 # failed) keeps its original src, so the page still shows something rather than
 # silently dropping the post's whole point.
 TMP_POSTS="$(mktemp)"
-if jq --slurpfile map "$MAP" --slurpfile posters "$POSTERMAP" '
+if jq --slurpfile map "$MAP" --slurpfile posters "$POSTERMAP" \
+      --slurpfile fallbacks "$FALLBACKMAP" '
     ($map[0] | map({key: .src, value: .}) | from_entries) as $m
     | ($posters[0] | from_entries) as $pm
+    | ($fallbacks[0] | from_entries) as $fm
     | map(.media = ((.media // []) | map(
         . as $item
         | ($m[$item.src] // null) as $hit
@@ -233,6 +265,12 @@ if jq --slurpfile map "$MAP" --slurpfile posters "$POSTERMAP" '
         # empty — a thumbnail the instance did provide always wins.
         | if ((.poster // "") == "") and (($pm[.src] // "") != "")
           then . + { poster: $pm[.src] }
+          else . end
+        # H.264 fallback, keyed by the LOCAL src like the poster map. Only ever
+        # set for media on our own mount — a mirrored third-party file has no
+        # sibling to find.
+        | if (($fm[.src] // "") != "")
+          then . + { fallback: $fm[.src] }
           else . end)))
     ' "$POSTS" > "$TMP_POSTS" 2>/dev/null; then
     # Same reason as the chmod on each mirrored file: mktemp is 0600 and the

tools/publish-media.sh

@@ -30,11 +30,16 @@
 # downstream consumer measures is what the viewer actually sees — see the
 # rotation note in fetch-media.sh for what goes wrong when it does not.
 #
-# NOTE ON AV1: nothing here emits a fallback encoding, and the <video> tag the
-# site renders carries a single src. Browsers without AV1 (Safari before 17, and
-# Apple hardware older than A17/M3) get an element that will not play rather than
-# a degraded one. Pass --raw, or re-encode to H.264, if that audience matters for
-# a particular post.
+# FALLBACK: AV1 alone locks out Safari before 17 and Apple hardware older than
+# A17/M3 — which surfaced the first time an AV1 link was posted and an iPhone
+# viewer got "bad media error" from the raw file. So every transcode also emits
+# an H.264 sibling, uploaded as <hash>.h264.mp4 next to the AV1 (same trick as
+# the poster: named after the AV1's hash so fetch-media.sh finds it with nothing
+# to look up). The URL this script prints for the post is the H.264 one, because
+# a post's link is fetched raw — Lemmy apps and browsers play that exact file,
+# with no <source> negotiation in front of it — and it must be the encoding
+# everything can play. Our own pages recognise the .h264.mp4 name, serve the AV1
+# to browsers that can take it, and keep the H.264 as the <source> fallback.
 
 set -eu
 
@@ -95,6 +100,7 @@ for src in $FILES; do
     esac
 
     poster=""
+    fallback=""
     if [ "$RAW" = 1 ] || [ "$kind" = other ]; then
         payload="$src"
         out_ext="$ext"
@@ -111,6 +117,18 @@ for src in $FILES; do
         # AAC rather than Opus on purpose: Opus-in-MP4 is still uneven in exactly
         # the players most likely to be shaky about AV1 anyway, and the audio is
         # a rounding error next to the video either way.
+        fallback="$WORK/h264.mp4"
+        echo "  transcoding H.264 fallback (crf 23)…"
+        # Same source and same denoise as the AV1, so the two encodings show the
+        # same frames. CRF is x264's own scale (not comparable to SVT-AV1's);
+        # 23 is its "looks the same" default, and this file is the compatibility
+        # rail, not the one most visitors download — bigger is acceptable here.
+        ffmpeg -nostdin -v error -i "$src" \
+               -vf "hqdn3d=4:3:6:4.5" \
+               -c:v libx264 -preset slow -crf 23 -pix_fmt yuv420p \
+               -movflags +faststart \
+               -c:a aac -b:a 96k -ac 1 \
+               "$fallback" -y
         poster="$WORK/poster.webp"
         # A frame a third of the way in, which beats frame 0 — recordings tend to
         # open on a lock screen, a hand moving into place, or a fade.
@@ -130,23 +148,32 @@ for src in $FILES; do
     name="$hash.$out_ext"
     publish_file "$payload" "$name"
 
-    # The poster is named after the VIDEO's hash, not its own. That is the whole
-    # point: fetch-media.sh finds it by name, with nothing to look up, when the
-    # instance did not manage to make a thumbnail — pict-rs will not read AV1, so
-    # for these uploads that is the normal case rather than the exception.
+    # The poster and the H.264 fallback are named after the AV1's hash, not
+    # their own. That is the whole point: fetch-media.sh finds both by name,
+    # with nothing to look up — the poster when the instance made no thumbnail
+    # (pict-rs will not read AV1, so for these uploads that is the normal
+    # case), the fallback for every browser that cannot decode AV1.
     if [ -n "$poster" ] && [ -f "$poster" ]; then
         publish_file "$poster" "$hash.poster.webp"
     fi
+    post_name="$name"
+    if [ -n "$fallback" ] && [ -f "$fallback" ]; then
+        post_name="$hash.h264.mp4"
+        publish_file "$fallback" "$post_name"
+    fi
 
-    url="$SITE_ORIGIN/media/$name"
+    # The postable URL is the H.264 one when it exists — see FALLBACK above.
+    url="$SITE_ORIGIN/media/$post_name"
     # Confirm it is actually being served before handing over a URL that is about
     # to be pasted into a post, where a 404 is public and permanent.
     code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 30 "$url" || echo 000)
     if [ "$code" = 200 ]; then
-        printf '  %s\n' "$url"
+        printf '  post this: %s\n' "$url"
     else
         echo "  WARNING: $url answered HTTP $code — do not post this link yet" >&2
     fi
     printf '  %s -> %s (%s)\n' \
         "$(du -h "$src" | cut -f1)" "$(du -h "$payload" | cut -f1)" "$name"
+    [ -z "$fallback" ] || printf '  fallback: %s (%s)\n' \
+        "$(du -h "$fallback" | cut -f1)" "$post_name"
 done