build.sh: install the newest registry imsd and fingerprintd, verified the way a phone does

Until now both were pinned here by version and sha256, so every release
of either needed a commit and an image run, in the right order
(fingerprintd 0.2.4 would have taken: push, wait for the registry, pin,
push). The pin gated fresh installs only: every installed phone already
takes the newest registry package on 'apk upgrade'.

registry-fetch.py resolves the newest version of each group (imsd + its
systemd unit; fingerprintd + systemd + agent, the subpackages at the
anchor's version, or it fails) and verifies the way apk does on the
phone: the index signature against the key catcrafts-fp6-repo ships --
the signer's name must be that key's too -- each apk's control checksum
against the index, and its data segment against the control's datahash.
Anything that fails is not written. The resolved versions and sha256s go
into the release's build-info.txt, so an image still names its exact
packages.

Tested against the live registry: it resolves imsd 0.3.3-r0 and
fingerprintd 0.2.3-r0 with sha256s identical to the five pins this
removes; a wrong key, a key of another name, a tampered control segment,
a corrupt or swapped data segment, a truncated file and a missing
subpackage are each refused with a reason.

build.sh

@@ -18,37 +18,18 @@ set -eu
 
 KERNEL_REPO=https://forgejo.catcrafts.net/Catcrafts/milos-linux.git
 KERNEL_BRANCH=combined-stable
-# imsd is not built here: the image installs the apk the imsd repo's package
+# imsd and fingerprintd are not built here: each repo's package CI publishes
-# CI publishes to the registry (section 3b), so image and 'apk upgrade' carry
+# its apk to the registry, and the image installs the NEWEST published
-# the same binary. Pinned by version AND by the sha256 of the registry files;
+# version (section 3b), so a fresh install carries the same binary every
-# a bump is these lines (sha256sum the two apks under $IMSD_REGISTRY/aarch64/).
+# installed phone gets from 'apk upgrade', and a release of either needs no
-# Bump deliberately, not via tip-chasing.
+# commit here. Integrity comes from the same place as on the phones: the
-# 0.3.1: 0.3.0 + the ims-pdn-up hardening (mmcli errors logged verbatim,
+# registry index is verified against the key catcrafts-fp6-repo ships
-# registration gate, configurable ip-type) + README carrier updates.
+# (aports/device/, the phones' /etc/apk/keys) and each apk against that
-# 0.3.2: landline callers ring (AMR-NB + G.711 media leg, any playable codec
+# index, by registry-fetch.py. The resolved versions and sha256s are in the
-# accepted, offered codecs named on 488), INVITE validated before the UI is
+# build summary. Until 2026-09-11 both were pinned here by version and
-# told, AMR-NB offered after AMR-WB, CODECS override; built for the A520/A720.
+# sha256; that gated fresh installs only, never upgrades, and cost a commit
-# 0.3.3: 0.3.2 + the media leg resolves the audio user once and thread-safely
+# and an image run per release.
-# (a getpwnam race could aim both PipeWire helpers at /run/user/0: static).
+REGISTRY=https://forgejo.catcrafts.net/api/packages/Catcrafts/alpine/edge/fp6
-IMSD_REGISTRY=https://forgejo.catcrafts.net/api/packages/Catcrafts/alpine/edge/fp6
-IMSD_VERSION=0.3.3-r0
-IMSD_SHA256="
-54755ca4aefaa0a7f98a0ce6c6515cf8e0ed1d86734e3f78a3b58f28c38f75b5  imsd-0.3.3-r0.apk
-226217f5aeea009462757fc89aa030964ba92ac1a22501bf8f75b92fa693f9f9  imsd-systemd-0.3.3-r0.apk
-"
-# fingerprintd (fingerprint unlock) comes from the same registry the same way:
-# its repo's package CI is the only producer, and the same pinning rule holds.
-# Three apks: the daemon, its systemd units, and the session agent (inert
-# until a user writes ~/.config/fingerprintd/fingers.conf). Needs the kernel
-# aport's CONFIG_QCOMTEE=m (pkgrel 101) and fp6-vendor-blobs >= 1-r2, both
-# built in this run. 0.2.3: 0.2.2 (enrol, unlock, agent, actions) + the
-# versioned blobs dependency + a post-upgrade daemon restart.
-FPD_VERSION=0.2.3-r0
-FPD_SHA256="
-3e28f0c1a9a844592ab6878b2dfc0d8f91674549e44bdc1652e7d7d029de1765  fingerprintd-0.2.3-r0.apk
-0cc46eba5c6c77d5bb54cd9f0e2902f7644720f9c98153062ffa33e19ca36889  fingerprintd-systemd-0.2.3-r0.apk
-6dfdbc6f971ba4b8f811f828e5868869c7d71fea6c7045e2bffd51bf2736c040  fingerprintd-agent-0.2.3-r0.apk
-"
 PMAPORTS_REPO=https://gitlab.postmarketos.org/postmarketOS/pmaports.git
 
 cd "$(dirname "$0")"
@@ -270,31 +251,29 @@ retry "build modemmanager" pmbootstrap $NOCROSS build --arch aarch64 modemmanage
 retry "build libcamera" pmbootstrap $NOCROSS build --arch aarch64 libcamera
 # --- 3b. imsd + fingerprintd: the published apks, not local builds -----------
 # Each repo's package CI is the only producer of its apk; the image installs
-# the exact registry package users later get via 'apk upgrade'.
+# the newest registry package, the one users get via 'apk upgrade'.
 # pmbootstrap has no knob for a third-party repository, and after the main
 # 'apk add' it re-adds every package found in its local packages dir BY FILE
 # PATH — which makes apk verify the package's own signature, and registry
 # packages are signed with per-run keys nobody keeps (phones trust the
-# registry-signed index instead). So: fetch, check against the sha256 pin,
+# registry-signed index instead). So: fetch the newest of each, verified the
-# re-sign the envelope with this run's abuild key (control and data streams
+# way a phone verifies them (registry-fetch.py: index signature against the
-# stay byte-identical, so the identity checksum equals the registry's), drop
+# shipped key, control checksum and data hash against the index), re-sign
-# into the local packages dir, re-index. The abuild key exists because the
+# the envelope with this run's abuild key (control and data streams stay
-# builds above initialized the buildroot.
+# byte-identical, so the identity checksum equals the registry's), drop into
+# the local packages dir, re-index. The abuild key exists because the builds
+# above initialized the buildroot. Each group's first name decides the
+# version; its subpackages must exist at that same version.
 REG_DL="$WORK/registry-apks"
 rm -rf "$REG_DL"
 mkdir -p "$REG_DL"
-for _f in "imsd-$IMSD_VERSION.apk" "imsd-systemd-$IMSD_VERSION.apk" \
+REGISTRY_KEY=$(echo aports/device/catcrafts-fp6-repo/*.rsa.pub)
-          "fingerprintd-$FPD_VERSION.apk" "fingerprintd-systemd-$FPD_VERSION.apk" \
+if [ ! -f "$REGISTRY_KEY" ]; then
-          "fingerprintd-agent-$FPD_VERSION.apk"; do
+	echo "expected exactly one registry key in aports/device/catcrafts-fp6-repo" >&2
-	# every fetched file must have a pin: 'grep .' below drops empty lines,
-	# so an empty pin list would otherwise pass the check with nothing checked
-	printf '%s\n' "$IMSD_SHA256" "$FPD_SHA256" | grep -q "  $_f\$" || {
-		echo "no sha256 pin for $_f - add it to IMSD_SHA256/FPD_SHA256" >&2
 	exit 1
-	}
+fi
-	retry "fetch $_f" curl -fsSL -o "$REG_DL/$_f" "$IMSD_REGISTRY/aarch64/$_f"
+retry "fetch registry packages" python3 ./registry-fetch.py "$REGISTRY" "$REGISTRY_KEY" "$REG_DL" \
-done
+	imsd,imsd-systemd fingerprintd,fingerprintd-systemd,fingerprintd-agent
-(cd "$REG_DL" && printf '%s\n' "$IMSD_SHA256" "$FPD_SHA256" | grep . | sha256sum -c -)
 ABUILD_KEY=$(echo "$WORKDIR"/config_abuild/*.rsa)
 if [ ! -f "$ABUILD_KEY" ]; then
 	echo "expected exactly one abuild key in $WORKDIR/config_abuild" >&2
@@ -354,8 +333,9 @@ cp README.md install.sh "$STAGE/fp6-img/"
 	echo "kernel:        $KERNEL_REPO $KERNEL_BRANCH @ $COMMIT"
 	echo "built:         $(date -u +%Y-%m-%dT%H:%M:%SZ)"
 	echo "default login: user / 147147 (same as official postmarketOS images)"
-	echo "imsd:          $IMSD_REGISTRY imsd-$IMSD_VERSION (registry package, sha256-pinned)"
+	while read -r _n _v _s; do
-	echo "fingerprintd:  $IMSD_REGISTRY fingerprintd-$FPD_VERSION (registry package, sha256-pinned)"
+		printf '%-14s %s %s-%s (newest registry package, sha256 %s)\n' "$_n:" "$REGISTRY" "$_n" "$_v" "$_s"
+	done < "$REG_DL/manifest"
 } > "$STAGE/fp6-img/build-info.txt"
 # sums of the extracted contents
 (cd "$STAGE/fp6-img" && sha256sum -- * > sha256sums.txt)

registry-fetch.py

@@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+"""Fetch the newest published versions of our registry packages, verified.
+
+    registry-fetch.py <registry-url> <trusted-key.rsa.pub> <dest-dir> <group>...
+
+<registry-url> is the Alpine repository root the phones carry in
+/etc/apk/repositories (.../alpine/edge/fp6); <trusted-key.rsa.pub> is the key
+they carry in /etc/apk/keys (aports/device/catcrafts-fp6-repo/); a <group> is
+a comma-separated list of package names whose FIRST member decides the
+version: "imsd,imsd-systemd" fetches the newest imsd and the imsd-systemd of
+that same version, and fails if the registry lacks it.
+
+Verification mirrors apk's own, so the image trusts exactly what an installed
+phone trusts: the index signature (.SIGN.RSA*.<key>, over the index's
+compressed tar) against the trusted key, and the signing key's NAME against
+the trusted key's; each package's control segment against the index's C:
+checksum ("Q1" + base64 sha1); each data segment against the control
+segment's datahash (sha256). A package that fails any step is not written.
+Prints one "name version sha256" line per apk and writes the same lines to
+<dest-dir>/manifest.
+
+Version order: apk's rules for the shapes our own packages use
+(X.Y.Z[-rN], numeric components); a suffix like _git is compared as text.
+"""
+import base64
+import gzip
+import hashlib
+import os
+import re
+import subprocess
+import sys
+import tempfile
+import time
+import urllib.request
+import zlib
+
+DIGEST = {"RSA": "sha1", "RSA256": "sha256", "RSA512": "sha512"}
+
+
+def die(msg):
+    sys.exit(f"registry-fetch: {msg}")
+
+
+def gzip_members(data):
+    off = 0
+    while off < len(data):
+        d = zlib.decompressobj(31)
+        d.decompress(data[off:])
+        end = len(data) - len(d.unused_data)
+        if end <= off:
+            raise ValueError("gzip stream did not advance")
+        yield data[off:end]
+        off = end
+
+
+def tar_files(tar):
+    """(name, bytes) for each regular file in a tar image; pax headers skipped."""
+    off = 0
+    while off + 512 <= len(tar):
+        hdr = tar[off:off + 512]
+        if hdr == b"\0" * 512:
+            return
+        size = int(hdr[124:136].split(b"\0")[0].strip() or b"0", 8)
+        name = hdr[:100].rstrip(b"\0").decode()
+        if hdr[156:157] not in (b"x", b"g"):
+            yield name, tar[off + 512:off + 512 + size]
+        off += 512 + (size + 511) // 512 * 512
+
+
+def fetch(url):
+    last = None
+    for attempt in range(3):
+        try:
+            with urllib.request.urlopen(url, timeout=120) as r:
+                return r.read()
+        except Exception as e:  # noqa: BLE001 - any transport failure retries
+            last = e
+            time.sleep(10)
+    die(f"cannot fetch {url}: {last}")
+
+
+def verify_index(index_tgz, keyfile):
+    """Returns the APKINDEX text after checking the signature against keyfile."""
+    try:
+        members = list(gzip_members(index_tgz))
+    except (zlib.error, ValueError) as e:
+        die(f"index: corrupt gzip stream ({e})")
+    if len(members) != 2:
+        die(f"index: expected 2 gzip streams, found {len(members)}")
+    sig_entries = list(tar_files(gzip.decompress(members[0])))
+    if not sig_entries:
+        die("index: no signature entry")
+    name, sig = sig_entries[0]
+    m = re.fullmatch(r"\.SIGN\.(RSA\d*)\.(.+)", name)
+    if not m or m.group(1) not in DIGEST:
+        die(f"index: unexpected signature entry {name!r}")
+    kind, signer = m.groups()
+    if signer != os.path.basename(keyfile):
+        die(f"index: signed by {signer!r}, phones trust {os.path.basename(keyfile)!r}")
+    with tempfile.TemporaryDirectory() as t:
+        sigf, dataf = os.path.join(t, "sig"), os.path.join(t, "data")
+        open(sigf, "wb").write(sig)
+        open(dataf, "wb").write(members[1])
+        r = subprocess.run(["openssl", "dgst", f"-{DIGEST[kind]}", "-verify", keyfile,
+                            "-signature", sigf, dataf], capture_output=True, text=True)
+    if r.returncode != 0 or "Verified OK" not in r.stdout:
+        die(f"index: signature does NOT verify against {keyfile}: {r.stdout.strip()} {r.stderr.strip()}")
+    files = dict(tar_files(gzip.decompress(members[1])))
+    if "APKINDEX" not in files:
+        die("index: no APKINDEX entry")
+    return files["APKINDEX"].decode()
+
+
+def parse_index(text):
+    """{name: {version: fields}} for aarch64 entries."""
+    out = {}
+    for block in text.split("\n\n"):
+        f = dict(line.split(":", 1) for line in block.splitlines() if ":" in line)
+        if f.get("A", "aarch64") != "aarch64" or "P" not in f or "V" not in f:
+            continue
+        out.setdefault(f["P"], {})[f["V"]] = f
+    return out
+
+
+def version_key(v):
+    ver, _, rel = v.partition("-r")
+    parts = tuple((0, int(t)) if t.isdigit() else (1, t) for t in re.split(r"[._]", ver))
+    return parts, int(rel) if rel.isdigit() else 0
+
+
+def verify_apk(blob, fields, name):
+    try:
+        members = list(gzip_members(blob))
+    except (zlib.error, ValueError) as e:
+        die(f"{name}: corrupt gzip stream ({e})")
+    if len(members) != 3:
+        die(f"{name}: expected 3 gzip streams, found {len(members)}")
+    want = fields.get("C", "")
+    if not want.startswith("Q1"):
+        die(f"{name}: index has no Q1 checksum")
+    got = "Q1" + base64.b64encode(hashlib.sha1(members[1]).digest()).decode()
+    if got != want:
+        die(f"{name}: control checksum {got} != index {want}")
+    pkginfo = dict(tar_files(gzip.decompress(members[1]))).get(".PKGINFO", b"").decode()
+    datahash = next((l.split("=", 1)[1].strip() for l in pkginfo.splitlines()
+                     if l.startswith("datahash")), None)
+    if not datahash:
+        die(f"{name}: .PKGINFO has no datahash")
+    if hashlib.sha256(members[2]).hexdigest() != datahash:
+        die(f"{name}: data segment does not match its datahash")
+    if "S" in fields and int(fields["S"]) != len(blob):
+        die(f"{name}: size {len(blob)} != index {fields['S']}")
+
+
+def main(registry, keyfile, dest, groups):
+    registry = registry.rstrip("/")
+    if not os.path.isfile(keyfile):
+        die(f"trusted key {keyfile} not found")
+    os.makedirs(dest, exist_ok=True)
+    index = parse_index(verify_index(fetch(f"{registry}/aarch64/APKINDEX.tar.gz"), keyfile))
+    lines = []
+    for group in groups:
+        names = group.split(",")
+        anchor = names[0]
+        if anchor not in index:
+            die(f"{anchor}: not in the registry index")
+        version = max(index[anchor], key=version_key)
+        for n in names:
+            fields = index.get(n, {}).get(version)
+            if fields is None:
+                die(f"{n}-{version}: not in the registry (newest {anchor} is {version})")
+            fname = f"{n}-{version}.apk"
+            blob = fetch(f"{registry}/aarch64/{fname}")
+            verify_apk(blob, fields, fname)
+            open(os.path.join(dest, fname), "wb").write(blob)
+            lines.append(f"{n} {version} {hashlib.sha256(blob).hexdigest()}")
+    with open(os.path.join(dest, "manifest"), "w") as f:
+        f.write("\n".join(lines) + "\n")
+    print("\n".join(lines))
+
+
+if __name__ == "__main__":
+    if len(sys.argv) < 5:
+        sys.exit(__doc__)
+    main(sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4:])