retire the bunq integration
All checks were successful
Deploy / build-deploy (push) Successful in 3m32s
All checks were successful
Deploy / build-deploy (push) Successful in 3m32s
The webhook is deregistered at bunq and the callback endpoint, its parser, default-deny classifier, dedup ledger and signature check are removed; the code is in git history if a bank feed ever comes back. /financials keeps reading the hand-maintained aggregates file, and sales + shop donations stay live from the order ledger. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
4c49000d46
commit
c31797bd9a
13 changed files with 70 additions and 1694 deletions
147
deploy/README.md
147
deploy/README.md
|
|
@ -15,13 +15,10 @@ and bank credentials.
|
|||
|
||||
Runtime state goes in a third place, `/var/lib/catcrafts`, created by the
|
||||
service's `StateDirectory=`. Today that is `orders.jsonl` (the order event log),
|
||||
`orders.jsonl.shipping.json` (the cached carrier rate table) and three files
|
||||
behind the public /financials page — `orders.jsonl.financials.json` (the
|
||||
published running totals), `orders.jsonl.financials-seen.json` (ingested bunq
|
||||
mutation ids, so a redelivered callback cannot double-count) and
|
||||
`orders.jsonl.financial-rules.json` (the classifier). See "The open financials
|
||||
page and the bunq mutation callback". Neither payment provider needs stored
|
||||
state — both authenticate with a bearer token per request.
|
||||
`orders.jsonl.shipping.json` (the cached carrier rate table) and
|
||||
`orders.jsonl.financials.json` (the published running totals behind the public
|
||||
/financials page — see "The open financials page"). Neither payment provider
|
||||
needs stored state — both authenticate with a bearer token per request.
|
||||
|
||||
**Two things on this box cannot be regenerated.** Everything else — the wasm
|
||||
bundle, the content, the binary — comes back from a rebuild.
|
||||
|
|
@ -39,9 +36,9 @@ cp /var/lib/catcrafts/orders.jsonl \
|
|||
It contains names, addresses and email addresses, so it is personal data: keep
|
||||
it 0600, keep it off the web root, and encrypt it before it leaves the machine.
|
||||
(The cached shipping table is deliberately NOT worth backing up: delete it and
|
||||
the next Sendcloud refresh rebuilds it. Neither are the financials files: the
|
||||
weekly reconciliation regenerates the totals from the bank history, and the
|
||||
rules file is a handful of lines you can rewrite.)
|
||||
the next Sendcloud refresh rebuilds it. The financials aggregates file is a few
|
||||
hundred bytes the owner's tooling rewrites — keep a copy with the ledger backup
|
||||
all the same, since nothing on this box can regenerate it.)
|
||||
|
||||
The second is `/srv/catcrafts-app/media` — the mirrored post images and screen
|
||||
recordings. Usually reproducible from `content/posts.json`, but **not if a source
|
||||
|
|
@ -584,128 +581,22 @@ Each appends a status event to the log — nothing is ever rewritten, so the
|
|||
file remains its own audit trail. Deleting personal data on request is an edit
|
||||
of the fields the seven-year fiscal retention does not cover.
|
||||
|
||||
## The open financials page and the bunq mutation callback
|
||||
## The open financials page
|
||||
|
||||
`/financials` publishes running totals only: sales, donations, and expenses
|
||||
as one flat list of categories. Sales fold out of `orders.jsonl` on every
|
||||
request and need no setup at all — that half works the moment the page ships.
|
||||
This section is about the other half.
|
||||
as one flat list of categories. Sales — and donations made through the shop —
|
||||
fold out of `orders.jsonl` on every request and need no setup at all.
|
||||
|
||||
**No bunq API key belongs on this box.** A bunq key can initiate payments and
|
||||
there is no read-only scope, so a compromised server would be a compromised
|
||||
bank account. Instead the key stays on your own machine, is used there once to
|
||||
register a notification filter, and from then on bunq PUSHES mutations here.
|
||||
The server can learn that money moved without being able to move any.
|
||||
The bank side (donations to the bank account, and the expense categories) is
|
||||
`orders.jsonl.financials.json`, written by the owner's own tooling off this
|
||||
box and re-read on every request: updating the file is all it takes to update
|
||||
the page. Aggregates by construction — category totals and an as-of date are
|
||||
all the file can carry, which is the page's privacy design. No bank credential
|
||||
of any kind lives on this box.
|
||||
|
||||
### What reaches the page, and what never does
|
||||
|
||||
A callback arrives carrying a counterparty name, an IBAN and a description.
|
||||
None of it is written down. The mutation is classified, its amount is added to
|
||||
a category total, its opaque id goes in a dedup ledger, and everything else is
|
||||
dropped before anything touches the disk. There is no file here that could leak
|
||||
a donor's identity, because no such file is ever written. ShouldServeFinancialsLive
|
||||
asserts exactly that, by scanning the whole state directory for a test IBAN
|
||||
afterwards.
|
||||
|
||||
Classification is **default-deny**: money no rule claims is withheld from the
|
||||
page and logged for you to write a rule for. It is never published as "other".
|
||||
|
||||
### Setup
|
||||
|
||||
1. **A dedicated bunq account.** Point donations at a monetary account used
|
||||
for nothing else. That account id is what classifies a donation, because
|
||||
donors are strangers and no IBAN list can know them in advance.
|
||||
|
||||
2. **The secret.** It is the last segment of the callback URL, and setting it
|
||||
is what brings the endpoint into existence — unset, `/api/bunq/*` is an
|
||||
ordinary 404.
|
||||
|
||||
```sh
|
||||
openssl rand -hex 32 # into BUNQ_CALLBACK_SECRET in payments.env
|
||||
systemctl restart catcrafts-server
|
||||
```
|
||||
|
||||
The URL is `https://catcrafts.net/api/bunq/<secret>`. Caddy proxies `/api/*`
|
||||
straight through, so no Caddyfile change is needed, and the analytics ingest
|
||||
censors `/api` out of the **public** report. It is NOT censored from the
|
||||
private tier or from Caddy's own access log, so treat the secret the way you
|
||||
treat the analytics password: rotate it if logs are ever shared.
|
||||
|
||||
3. **The rules**, at `/var/lib/catcrafts/orders.jsonl.financial-rules.json`.
|
||||
Re-read on every callback, so a new rule takes effect without a restart:
|
||||
|
||||
```json
|
||||
{"donation_accounts": [9911],
|
||||
"rules": [
|
||||
{"description_contains": "hetzner", "group": "expense", "label": "Hosting"},
|
||||
{"iban": "NL00INSURER0000000", "group": "expense", "label": "Insurance"},
|
||||
{"iban": "DE00SUPPLIER000000", "group": "expense", "label": "Inventory"},
|
||||
{"iban": "NL00MOLLIE00000000", "group": "ignore"},
|
||||
{"iban": "NL00OWNSELF0000000", "group": "ignore"}]}
|
||||
```
|
||||
|
||||
`group` is `donations`, `expense` or `ignore`; first match wins,
|
||||
and explicit rules beat the donation-account default (which is how your own
|
||||
transfer between accounts stays out of the donation total). **Ignore your
|
||||
Mollie payouts, and one day the exchange's SEPA leg from selling EURC** —
|
||||
those are sales, already counted from the
|
||||
ledger, and letting them through would publish that money twice. A rule with
|
||||
no criterion, an unknown group, or an expense with no label is dropped at
|
||||
load rather than allowed to claim everything.
|
||||
|
||||
4. **Register the filter from your own machine**, with the key that lives
|
||||
there — `tools/bunq-callback.sh` does the whole handshake (installation,
|
||||
device-server, session) and installs a `MUTATION` `NotificationFilterUrl`:
|
||||
|
||||
```sh
|
||||
tools/bunq-callback.sh list # accounts + current filters
|
||||
tools/bunq-callback.sh set <account-id> https://catcrafts.net/api/bunq/<secret>
|
||||
```
|
||||
|
||||
It reads `BUNQ_KEY` from the repo-root `.env`, binds the key to **this
|
||||
machine's** address only (never the server's — `permitted_ips` governs who
|
||||
may CALL the bunq API, not where callbacks are delivered), and refuses to
|
||||
register a URL that is not already answering 200. Run `list` first and put
|
||||
the donations account id in `donation_accounts` in the rules file above.
|
||||
|
||||
The device registration is permanent for that key, and your home address is
|
||||
probably dynamic: when it rotates, this script stops working from here
|
||||
(add the new address to the device in the bunq app). **The callback keeps
|
||||
working regardless** — bunq delivers outbound, so nothing about `permitted_ips`
|
||||
affects it.
|
||||
|
||||
5. **Optional but recommended: signature checking.** Set
|
||||
`BUNQ_CALLBACK_PUBKEY` to a PEM file holding bunq's server public key and
|
||||
every callback must then carry a valid RSA-SHA256 signature over its body.
|
||||
It is off by default deliberately: the header bunq signs with has changed
|
||||
across API generations, and a verifier wrong about the header name rejects
|
||||
every real callback while looking like it works. Turn it on **after** you
|
||||
have seen a real callback arrive carrying `X-Bunq-Server-Signature`, and
|
||||
confirm afterwards that donations still land.
|
||||
|
||||
### Operating it
|
||||
|
||||
* Watch it work: `journalctl -u catcrafts-server -f`. A mutation no rule
|
||||
claimed logs its id, the running count of withheld mutations and their net
|
||||
total — that log line is your to-do list.
|
||||
* Every unauthorised request answers 404, never 401: the endpoint does not
|
||||
confirm its own existence to a prober.
|
||||
* A duplicate, a withheld and an ignored mutation all answer 200. A non-2xx
|
||||
makes bunq redeliver, so only a failed write earns a 500 — the one case
|
||||
where a retry could actually help.
|
||||
* State files, all under `/var/lib/catcrafts` and none worth backing up:
|
||||
`orders.jsonl.financials.json` (the published totals),
|
||||
`orders.jsonl.financials-seen.json` (ingested ids + withheld counters), and
|
||||
the rules file above.
|
||||
|
||||
### The weekly reconciliation is the authority
|
||||
|
||||
Callbacks can be missed, replayed or arrive before a rule exists for them, so
|
||||
this path is allowed to be **lossy but never wrong**: it may withhold, it may
|
||||
not invent. Your home tooling recomputes every total from the full bunq
|
||||
mutation history and overwrites `orders.jsonl.financials.json` wholesale — same
|
||||
format, same file. That is the correction mechanism, and it is what makes it
|
||||
safe for the live path to publish provisionally.
|
||||
(A bunq mutation callback used to keep these numbers live; it was retired
|
||||
2026-08-17 — webhook deregistered, endpoint removed. The code and its
|
||||
documentation are in git history if a bank feed ever comes back.)
|
||||
|
||||
## Verifying a deploy
|
||||
|
||||
|
|
|
|||
|
|
@ -85,14 +85,6 @@ ReadOnlyPaths=/srv/catcrafts-app /srv/catcrafts.net
|
|||
# INVOICE_GPG_KEY=... invoice signing (see deploy/README.md)
|
||||
# MAIL_COMMAND=msmtp -t order confirmation email (see deploy/README.md,
|
||||
# MAIL_FROM=... "Order email"); unset = no email is sent
|
||||
# BUNQ_CALLBACK_SECRET=... the last segment of the bunq mutation callback
|
||||
# URL, and what brings that endpoint into being:
|
||||
# unset, /api/bunq/* is a plain 404. NOT an API
|
||||
# key — no bunq key belongs on this box, because
|
||||
# one can initiate payments (deploy/README.md,
|
||||
# "The bunq mutation callback")
|
||||
# BUNQ_CALLBACK_PUBKEY=... path to bunq's server public key in PEM; set it
|
||||
# to REQUIRE a valid RSA-SHA256 body signature
|
||||
# The '-' prefix makes the file optional: without it the server starts with
|
||||
# payments off and the shop renders but refuses checkout — degraded, not down.
|
||||
EnvironmentFile=-/etc/catcrafts/payments.env
|
||||
|
|
|
|||
Loading…
Reference in a new issue