Catcrafts/catcrafts.net
Watch
2
0
Fork
You've already forked catcrafts.net
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

tests and eurc
All checks were successful
Deploy / build-deploy (push) Successful in 4m19s
Details

Browse source
This commit is contained in:
Jorijn van der Graaf 2026-08-15 00:54:05 +02:00
commit 749f525f83
44 changed files with 5380 additions and 3532 deletions
Download patch file Download diff file Expand all files Collapse all files

172
deploy/README.md
View file

@ -103,10 +103,12 @@ authority than "restart one service" needs — the last deploy step writes
`/srv/catcrafts-app/.deploy-stamp`, and `catcrafts-deploy.path` on the host
reacts.
`catcrafts-deploy.service` runs `catcrafts-server --selftest` as `ExecStartPre`
before restarting. That self-test exercises the HTML-escaping and JSON layers
that generate every byte of markup the site emits, so **a broken binary leaves
the working server running** instead of replacing it.
`catcrafts-deploy.service` runs `catcrafts-server --render /` as `ExecStartPre`
before restarting. The unit tests already gated the deploy in CI
(`crafter-build test --product=server`); this on-host check covers what CI
cannot — the dynamic loader finding libmsquic on this machine and a page
rendering from the deployed content — so **a broken binary leaves the working
server running** instead of replacing it.
The binary is installed as `catcrafts-server.new` and `mv`d into place, so a
request arriving mid-copy never hits a truncated executable.
@ -208,7 +210,7 @@ it to `/media/<hash>` and probes the local file for dimensions, downloading
nothing. That is not just an optimisation — it removes the whole class of build
failure where the media step depends on a third party. A 167 MB recording on a
file host once hit `MAX_BYTES` (64 MB), so the entry kept its original URL, and
the deploy then failed the e2e "media origin" check on a file that was sitting on
the deploy then failed the "media origin" check (ShouldServePostPages) on a file that was sitting on
our own disk the entire time.
Adopting a `<hash>.h264.mp4` URL swaps the AV1 sibling back in as the page's
@ -243,14 +245,15 @@ Two things to know about publishing AV1:
An own-origin URL naming a file that is **not** on the mount is deliberately left
pointing at its original URL rather than rewritten. That is a post published
without its media, and failing the e2e origin check loudly beats shipping a 404
without its media, and failing the origin check (ShouldServePostPages) loudly beats shipping a 404
inside a `<video>` tag.
**`fetch-media.sh` wants `ffprobe`** (Arch: `ffmpeg`) to read pixel dimensions,
which become the `width`/`height` attributes that stop the page reflowing as
several 5 MB recordings arrive. It degrades to no dimensions without it —
so `tools/e2e.sh` asserts they are present, and a build host missing the package
fails at the e2e gate instead of quietly shipping a janky page.
so the ShouldServePostPages suite asserts they are present, and a build host
missing the package fails at the test gate instead of quietly shipping a janky
page.
**Only the post's main media is used.** Lemmy distinguishes the media a post *is*
(`post.url`, with `post.thumbnail_url` as its generated still) from images merely
@ -272,78 +275,118 @@ hotlinking would send every visitor's IP to the source instance; and these posts
are their media, so a deleted upstream file would gut the page. Filenames are the
content hash, which is why the cache lifetime can be a year.
## Payments: Mollie and CoinGate
## Payments: Mollie and EURC
Checkout offers the buyer **two choices**, each served by its own rail:
| Choice | Rail | Env var | What the buyer gets |
|----------|----------|---------------------|-----------------------------------------|
| `bank` | Mollie | `MOLLIE_API_KEY` | iDEAL, cards, bank transfer |
| `crypto` | CoinGate | `COINGATE_API_KEY` | Bitcoin + Lightning, stablecoins, more |
| `crypto` | EURC | `EURC_CHAINS` | self-hosted EURC on the configured EVM chains |
The slots are independent. Set one key and the form offers only that method;
The slots are independent. Set one and the form offers only that method;
set both and the buyer picks; set neither and checkout answers 503 with an
honest message — the whole site still works, degraded rather than down. A
mode whose key is missing is a **startup refusal**, not a silent downgrade:
a checkout that 502s at the last step is worse than one that never offered.
mode whose credential is missing is a **startup refusal**, not a silent
downgrade: a checkout that 502s at the last step is worse than one that
never offered.
### The EURC rail
No processor, no API key, no account anywhere: the buyer sends EURC (Circle's
euro stablecoin, pegged 1:1 — the token amount IS the euro total) to an
address from a pool this shop generated offline, and the server notices by
polling `balanceOf` over JSON-RPC at a finalized block. The order page is the
payment page. Three pieces of configuration:
- `EURC_CHAINS=/etc/catcrafts/eurc-chains.json` — which chains to watch.
Selecting this rail IS setting this variable. One EVM address is valid on
every chain at once, so the buyer pays on whichever is cheapest; **file
order is display order** — put the cheap chain (Base) first, and its
`note` ("lowest network fees") is the nudge the buyer sees.
```json
{"chains": [
{"name": "base", "rpc": "https://mainnet.base.org",
"contract": "0x60a3E35Cc302bFA44Cb288Bc5a4F316Fdb1adb42",
"chain_id": 8453, "note": "lowest network fees"},
{"name": "ethereum", "rpc": "https://ethereum-rpc.publicnode.com",
"contract": "0x1aBaEA1f7C830bD89Acc67eC4af516284b1bC33c",
"chain_id": 1}
]}
```
Contract addresses come from Circle's own list
(developers.circle.com/stablecoins/eurc-contract-addresses) and nowhere
else — matching the CONTRACT, not the ticker, is what makes a fake "EURC"
worthless here. EURC exists on Ethereum, Base, Avalanche, Cronos and World
Chain (not Arbitrum/Optimism/Polygon/BNB).
- `EURC_POOL` (default `<orders>.eurc-addresses`) — one receiving address
per line, `#` comments allowed. Generated by the wallet at home; the box
holds NO key, NO xpub, and can only hand out addresses it was given.
**Copy addresses, never retype**: the server cannot verify the EIP-55
checksum, so a retyped-but-still-hex address is accepted and published.
**Append only**: `<pool>.cursor` is an index into this file, so reordering
re-issues addresses already bound to old orders. Duplicates and malformed
lines are startup refusals; exhaustion too — the low-water warning fires
at 25 left.
- `EURC_WINDOW_HOURS` (default 24) — how long an order may sit unpaid. A
lapsed window is NOT bounced money: the address stays ours, a late payment
still lands there (the server logs exactly this), and it is settled by
hand with `--mark-paid`.
```sh
# Mollie keys: dashboard -> Developers -> API keys. CoinGate: dashboard ->
# API -> new app token. BOTH have real test modes that work against the real
# endpoints — verify the whole flow before swapping in production keys.
# Mollie keys: dashboard -> Developers -> API keys. Mollie's test mode works
# against the real endpoints — verify the whole flow before swapping in the
# live key. The EURC rail has no key and no test mode: point EURC_CHAINS at a
# testnet chains file (Base Sepolia / Ethereum Sepolia contracts are in
# Circle's list) to rehearse against worthless tokens instead.
install -d -m 0755 /etc/catcrafts
cat > /etc/catcrafts/payments.env <<'ENV'
MOLLIE_API_KEY=test_your-key-here
COINGATE_API_KEY=your-coingate-token
COINGATE_SANDBOX=1
EURC_CHAINS=/etc/catcrafts/eurc-chains.json
ENV
chmod 0600 /etc/catcrafts/payments.env
systemctl restart catcrafts-server
journalctl -u catcrafts-server | tail # "payments: bank=mollie crypto=coingate"
journalctl -u catcrafts-server | tail # "payments: bank=mollie crypto=eurc"
```
Drop `COINGATE_SANDBOX=1` for live crypto payments; it selects
`api-sandbox.coingate.com` and its tokens are not interchangeable with live
ones.
Mechanics worth knowing:
* The reconciler polls each open order against **the rail that issued its
link** — the ledger records `pay_choice` per order for exactly this reason.
Mollie every 10 s, CoinGate every 20 s (a blockchain confirmation will not
arrive faster), both backing off with age. The `?redirect` back from either
provider is ignored by design: only the authenticated poll moves an order
to paid, and the paid event records the method (`ideal`, `creditcard`,
`btc`, …) in the ledger.
* **Both providers EXPIRE unpaid orders**, which lapses them automatically and
the buyer just orders again. CoinGate is far more aggressive about it: two
hours before a coin is picked, twenty minutes after. Expect crypto orders to
lapse routinely; that is the normal case, not a fault.
* CoinGate settles in **EUR** (`receive_currency=EUR` in the rail), so the
money that lands is the money the invoice says, the rate is locked when the
buyer opens the invoice, and no crypto touches the balance sheet. That one
parameter is the whole difference between "a second Mollie" and "the shop
now holds crypto" — changing it is a tax decision, not a code cleanup.
Mollie every 10 s, the EURC rail every 30 s (a finalized block will not
arrive faster), both backing off with age. Arriving back on the order page
is ignored as evidence by design: only the authenticated poll (Mollie) or
the rail's own RPC check (EURC) moves an order to paid, and the paid event
records the method (`ideal`, `creditcard`, `eurc-base`, …) in the ledger.
* **Unpaid orders lapse automatically** and the buyer just orders again.
Mollie expires its own payments; the EURC rail closes its window after
`EURC_WINDOW_HOURS` (24 by default). A lapsed EURC order is NOT bounced
money — see above.
* The shop **holds EURC**: crypto revenue sits at the pool addresses until
swept, and nothing lands on the bank account until it is sold for euros at
an exchange. Bookkeeping note: a EURC sale is still a euro sale on the
invoice — EURC is euro-denominated, so there is no rate and no revaluation —
but the financials bank-callback never sees it, so crypto sales exist in
the ledger only until the sweep's SEPA leg arrives.
* Card money stays disputable for months even after "paid": before shipping a
large or exported order, glance at the `via` column in `--orders`. iDEAL,
bank transfers and crypto are final; `creditcard` is the one with a tail.
bank transfers and EURC are final; `creditcard` is the one with a tail.
This is the one real advantage of the crypto rail — no chargebacks — and it
matters most on exactly the non-EU orders where cards get declined.
* A `refunded` CoinGate order that the reconciler sees while still awaiting
gets **lapsed and logged loudly**: it means a long outage spanned the entire
paid window and the money has since gone back. That is the case where
`--mark-paid` may be the right answer and only a human can tell.
* Mollie onboarding reviews the shop: the imprint (KVK, contact address),
terms and privacy pages must be real before they approve live payments.
They are — and e2e now fails the build if a PLACEHOLDER marker ever
reaches a rendered page again. CoinGate onboarding is a KYB review of the
registered business (KVK, UBO, bank account) and wants the same pages.
They are — and ShouldStayScriptFree fails the build if a PLACEHOLDER marker ever
reaches a rendered page again. The EURC rail has no onboarding, no KYB and
no account anywhere; that is the reason it exists.
* Accepting crypto for goods does **not** make this shop a CASP under MiCA —
no custody, no transfer for third parties, so no licence is required. What
it does require is that the processor holds one: since 1 July 2026 only
MiCA-authorised CASPs may serve EU clients, and CoinGate holds both a MiCA
licence and a Payment Institution licence. Verify any replacement provider
in the ESMA register before switching a key.
no custody for others, no transfer for third parties, so no licence is
required, with or without a processor. Cashing EURC out to the bank goes
through an exchange, and THAT party must be a MiCA-authorised CASP (check
the ESMA register) — but it stands after the checkout, not in it.
* VAT is unchanged by payment method: the sale is priced and invoiced in euro
and taxed on the euro value, whichever rail settled it.
@ -398,14 +441,13 @@ Sendcloud outage keeps selling at the last known prices. Format is
(EU rates are grossed up once, at fetch, so the shop nets the carrier's cost).
That also means a hand-written cache file is a complete rate table, which is how
dev and `tools/e2e.sh` run with no Sendcloud account at all. A cache written by
dev and the black-box suites run with no Sendcloud account at all. A cache written by
an older build (flat `country -> cents`, no weight bands) is ignored on load and
replaced by the next refresh — those numbers were an unknown weight band and
re-serving them would price parcels by guess.
Like the CoinGate rail, this integration is UNTESTED against the live API until
credentials exist — the response parser is covered by `--selftest`, the fetch
around it is thin. **Verify one real fetch before opening the shop**: check that
This integration is UNTESTED against the live API until credentials exist —
the response parser is covered by the ShouldParseSendcloudRates test, the fetch around it is thin. **Verify one real fetch before opening the shop**: check that
the logged country count and the weight bands match what the Sendcloud panel
shows, because that table is now the difference between a shop that sells and
one that refuses everything.
@ -510,9 +552,10 @@ Worth knowing:
* With a signing key configured, a gpg failure means the email WAITS — an
unsigned invoice never leaves by accident, same rule as the download.
* Send a real test: `--rail=fake` locally with `MAIL_COMMAND` pointing at
msmtp and your own address in the order form, or just run `tools/e2e.sh`,
which captures the messages with a fake sendmail and verifies the attached
signature.
msmtp and your own address in the order form, or just run the
ShouldProcessCheckout suite (`crafter-build test ShouldProcessCheckout
--product=server`), which captures the messages with a fake sendmail and
verifies the attached signature.
## Reading the orders ledger
@ -560,8 +603,9 @@ A callback arrives carrying a counterparty name, an IBAN and a description.
None of it is written down. The mutation is classified, its amount is added to
a category total, its opaque id goes in a dedup ledger, and everything else is
dropped before anything touches the disk. There is no file here that could leak
a donor's identity, because no such file is ever written. `tools/e2e.sh` asserts
exactly that, by grepping the whole state directory for a test IBAN afterwards.
a donor's identity, because no such file is ever written. ShouldServeFinancialsLive
asserts exactly that, by scanning the whole state directory for a test IBAN
afterwards.
Classification is **default-deny**: money no rule claims is withheld from the
page and logged for you to write a rule for. It is never published as "other".
@ -603,7 +647,8 @@ page and logged for you to write a rule for. It is never published as "other".
`group` is `donations`, `expense` or `ignore`; first match wins,
and explicit rules beat the donation-account default (which is how your own
transfer between accounts stays out of the donation total). **Ignore your
Mollie and CoinGate payouts** — those are sales, already counted from the
Mollie payouts, and one day the exchange's SEPA leg from selling EURC** —
those are sales, already counted from the
ledger, and letting them through would publish that money twice. A rule with
no criterion, an unknown group, or an expense with no label is dropped at
load rather than allowed to claim everything.
@ -757,11 +802,12 @@ split working as designed.
tools/fetch-posts.sh # pull posts from the allowed communities
tools/fetch-media.sh [dir] # mirror their media locally (run after the above)
tools/fetch-rates.sh # ECB reference rates for the indicative prices
tools/e2e.sh # ~200 HTTP checks against a real server (135 while
# coming-soon; the rest re-arm at launch); the CI gate
crafter-build --local -r # the wasm app alone, no backend, on :8080
<server>/catcrafts-server --selftest # ~260 in-process assertions
crafter-build test --product=server --local # every suite: the unit assertions
# AND the black-box HTTP suites, which spawn the
# real server on scratch ports (the CI gate; the
# checkout lifecycle re-arms at launch)
<server>/catcrafts-server --routes # status + title for every route
<server>/catcrafts-server --render /projects # dump one page's HTML
<server>/catcrafts-server --orders FILE # the orders ledger + manual transitions

20
deploy/catcrafts-deploy.service
View file

@ -11,10 +11,20 @@ Documentation=https://forgejo.catcrafts.net/Catcrafts/catcrafts.net
[Service]
Type=oneshot
# Verify the freshly deployed binary before cutting over. --selftest exercises
# the escaping and JSON layers that produce every byte of markup the site
# emits; if it fails, the currently-running (working) server is left alone
# rather than replaced with a broken one.
ExecStartPre=/srv/catcrafts-app/catcrafts-server --selftest
# Verify the freshly deployed binary before cutting over. The unit tests run
# in CI (crafter-build test) before anything is shipped, so what this gate
# checks is the part CI cannot: that THIS binary executes on THIS host — the
# dynamic loader finds libmsquic (the first real deploy died exactly there,
# exit 127) — and renders a page from the deployed content. If it fails, the
# currently-running (working) server is left alone rather than replaced with
# a broken one.
#
# WorkingDirectory because --render loads ./content; StandardOutput=null so
# the rendered HTML does not land in the journal on every deploy (stderr,
# where failures speak, still does).
WorkingDirectory=/srv/catcrafts-app
StandardOutput=null
StandardError=journal
ExecStartPre=/srv/catcrafts-app/catcrafts-server --render /
ExecStart=/usr/bin/systemctl restart catcrafts-server.service

6
deploy/catcrafts-server.service
View file

@ -76,8 +76,10 @@ ReadOnlyPaths=/srv/catcrafts-app /srv/catcrafts.net
# rsync-wiped, and /srv/catcrafts-app is CI-writable; neither may ever hold a
# credential. /etc/catcrafts/payments.env (root:root 0600) carries:
# MOLLIE_API_KEY=live_... (or test_... while verifying) — the BANK rail
# COINGATE_API_KEY=... the CRYPTO rail; omit and checkout offers only
# COINGATE_SANDBOX=1 bank. Sandbox tokens are not live tokens.
# EURC_CHAINS=/etc/catcrafts/eurc-chains.json — the CRYPTO rail (self-hosted
# EURC; no key). Omit and checkout offers only bank.
# The address pool defaults to
# /var/lib/catcrafts/orders.jsonl.eurc-addresses.
# SENDCLOUD_PUBLIC_KEY / SENDCLOUD_SECRET_KEY / SENDCLOUD_METHOD — REQUIRED
# to sell: no rate table means checkout refuses
# INVOICE_GPG_KEY=... invoice signing (see deploy/README.md)