From c31797bd9a08f72c0323c84904b1a0dd998fc8a6 Mon Sep 17 00:00:00 2001 From: Jorijn van der Graaf Date: Mon, 17 Aug 2026 11:36:54 +0200 Subject: [PATCH] retire the bunq integration The webhook is deregistered at bunq and the callback endpoint, its parser, default-deny classifier, dedup ledger and signature check are removed; the code is in git history if a bank feed ever comes back. /financials keeps reading the hand-maintained aggregates file, and sales + shop donations stay live from the order ledger. Co-Authored-By: Claude Fable 5 --- .gitignore | 7 - deploy/README.md | 147 +---- deploy/catcrafts-server.service | 8 - project.cpp | 16 +- .../Catcrafts.Server-Financials.cpp | 527 +----------------- .../implementations/Catcrafts.Server-Http.cpp | 56 +- server/implementations/main.cpp | 19 +- server/interfaces/Catcrafts.Server.cppm | 101 +--- .../interfaces/Catcrafts.Shared-Content.cppm | 2 +- tests/ShouldPublishFinancials/main.cpp | 294 +--------- tests/ShouldServeFinancialsLive/main.cpp | 137 +---- tests/harness/Catcrafts.E2eHarness.cppm | 4 +- tools/bunq-callback.sh | 442 --------------- 13 files changed, 68 insertions(+), 1692 deletions(-) delete mode 100755 tools/bunq-callback.sh diff --git a/.gitignore b/.gitignore index 646e8e7..b18d24c 100644 --- a/.gitignore +++ b/.gitignore @@ -21,13 +21,6 @@ media/ orders.jsonl *.fake-paid *.financials.json -*.financials-seen.json -*.financial-rules.json -bunq-state.json -# tools/bunq-callback.sh's client keypair (a PRIVATE key) and the bunq server -# public key it saves for BUNQ_CALLBACK_PUBKEY. Home-machine credentials. -bunq-client-key.pem -bunq-server-public-key.pem .env diff --git a/deploy/README.md b/deploy/README.md index 0649680..5955229 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -15,13 +15,10 @@ and bank credentials. Runtime state goes in a third place, `/var/lib/catcrafts`, created by the service's `StateDirectory=`. Today that is `orders.jsonl` (the order event log), -`orders.jsonl.shipping.json` (the cached carrier rate table) and three files -behind the public /financials page — `orders.jsonl.financials.json` (the -published running totals), `orders.jsonl.financials-seen.json` (ingested bunq -mutation ids, so a redelivered callback cannot double-count) and -`orders.jsonl.financial-rules.json` (the classifier). See "The open financials -page and the bunq mutation callback". Neither payment provider needs stored -state — both authenticate with a bearer token per request. +`orders.jsonl.shipping.json` (the cached carrier rate table) and +`orders.jsonl.financials.json` (the published running totals behind the public +/financials page — see "The open financials page"). Neither payment provider +needs stored state — both authenticate with a bearer token per request. **Two things on this box cannot be regenerated.** Everything else — the wasm bundle, the content, the binary — comes back from a rebuild. @@ -39,9 +36,9 @@ cp /var/lib/catcrafts/orders.jsonl \ It contains names, addresses and email addresses, so it is personal data: keep it 0600, keep it off the web root, and encrypt it before it leaves the machine. (The cached shipping table is deliberately NOT worth backing up: delete it and -the next Sendcloud refresh rebuilds it. Neither are the financials files: the -weekly reconciliation regenerates the totals from the bank history, and the -rules file is a handful of lines you can rewrite.) +the next Sendcloud refresh rebuilds it. The financials aggregates file is a few +hundred bytes the owner's tooling rewrites — keep a copy with the ledger backup +all the same, since nothing on this box can regenerate it.) The second is `/srv/catcrafts-app/media` — the mirrored post images and screen recordings. Usually reproducible from `content/posts.json`, but **not if a source @@ -584,128 +581,22 @@ Each appends a status event to the log — nothing is ever rewritten, so the file remains its own audit trail. Deleting personal data on request is an edit of the fields the seven-year fiscal retention does not cover. -## The open financials page and the bunq mutation callback +## The open financials page `/financials` publishes running totals only: sales, donations, and expenses -as one flat list of categories. Sales fold out of `orders.jsonl` on every -request and need no setup at all — that half works the moment the page ships. -This section is about the other half. +as one flat list of categories. Sales — and donations made through the shop — +fold out of `orders.jsonl` on every request and need no setup at all. -**No bunq API key belongs on this box.** A bunq key can initiate payments and -there is no read-only scope, so a compromised server would be a compromised -bank account. Instead the key stays on your own machine, is used there once to -register a notification filter, and from then on bunq PUSHES mutations here. -The server can learn that money moved without being able to move any. +The bank side (donations to the bank account, and the expense categories) is +`orders.jsonl.financials.json`, written by the owner's own tooling off this +box and re-read on every request: updating the file is all it takes to update +the page. Aggregates by construction — category totals and an as-of date are +all the file can carry, which is the page's privacy design. No bank credential +of any kind lives on this box. -### What reaches the page, and what never does - -A callback arrives carrying a counterparty name, an IBAN and a description. -None of it is written down. The mutation is classified, its amount is added to -a category total, its opaque id goes in a dedup ledger, and everything else is -dropped before anything touches the disk. There is no file here that could leak -a donor's identity, because no such file is ever written. ShouldServeFinancialsLive -asserts exactly that, by scanning the whole state directory for a test IBAN -afterwards. - -Classification is **default-deny**: money no rule claims is withheld from the -page and logged for you to write a rule for. It is never published as "other". - -### Setup - -1. **A dedicated bunq account.** Point donations at a monetary account used - for nothing else. That account id is what classifies a donation, because - donors are strangers and no IBAN list can know them in advance. - -2. **The secret.** It is the last segment of the callback URL, and setting it - is what brings the endpoint into existence — unset, `/api/bunq/*` is an - ordinary 404. - - ```sh - openssl rand -hex 32 # into BUNQ_CALLBACK_SECRET in payments.env - systemctl restart catcrafts-server - ``` - - The URL is `https://catcrafts.net/api/bunq/`. Caddy proxies `/api/*` - straight through, so no Caddyfile change is needed, and the analytics ingest - censors `/api` out of the **public** report. It is NOT censored from the - private tier or from Caddy's own access log, so treat the secret the way you - treat the analytics password: rotate it if logs are ever shared. - -3. **The rules**, at `/var/lib/catcrafts/orders.jsonl.financial-rules.json`. - Re-read on every callback, so a new rule takes effect without a restart: - - ```json - {"donation_accounts": [9911], - "rules": [ - {"description_contains": "hetzner", "group": "expense", "label": "Hosting"}, - {"iban": "NL00INSURER0000000", "group": "expense", "label": "Insurance"}, - {"iban": "DE00SUPPLIER000000", "group": "expense", "label": "Inventory"}, - {"iban": "NL00MOLLIE00000000", "group": "ignore"}, - {"iban": "NL00OWNSELF0000000", "group": "ignore"}]} - ``` - - `group` is `donations`, `expense` or `ignore`; first match wins, - and explicit rules beat the donation-account default (which is how your own - transfer between accounts stays out of the donation total). **Ignore your - Mollie payouts, and one day the exchange's SEPA leg from selling EURC** — - those are sales, already counted from the - ledger, and letting them through would publish that money twice. A rule with - no criterion, an unknown group, or an expense with no label is dropped at - load rather than allowed to claim everything. - -4. **Register the filter from your own machine**, with the key that lives - there — `tools/bunq-callback.sh` does the whole handshake (installation, - device-server, session) and installs a `MUTATION` `NotificationFilterUrl`: - - ```sh - tools/bunq-callback.sh list # accounts + current filters - tools/bunq-callback.sh set https://catcrafts.net/api/bunq/ - ``` - - It reads `BUNQ_KEY` from the repo-root `.env`, binds the key to **this - machine's** address only (never the server's — `permitted_ips` governs who - may CALL the bunq API, not where callbacks are delivered), and refuses to - register a URL that is not already answering 200. Run `list` first and put - the donations account id in `donation_accounts` in the rules file above. - - The device registration is permanent for that key, and your home address is - probably dynamic: when it rotates, this script stops working from here - (add the new address to the device in the bunq app). **The callback keeps - working regardless** — bunq delivers outbound, so nothing about `permitted_ips` - affects it. - -5. **Optional but recommended: signature checking.** Set - `BUNQ_CALLBACK_PUBKEY` to a PEM file holding bunq's server public key and - every callback must then carry a valid RSA-SHA256 signature over its body. - It is off by default deliberately: the header bunq signs with has changed - across API generations, and a verifier wrong about the header name rejects - every real callback while looking like it works. Turn it on **after** you - have seen a real callback arrive carrying `X-Bunq-Server-Signature`, and - confirm afterwards that donations still land. - -### Operating it - -* Watch it work: `journalctl -u catcrafts-server -f`. A mutation no rule - claimed logs its id, the running count of withheld mutations and their net - total — that log line is your to-do list. -* Every unauthorised request answers 404, never 401: the endpoint does not - confirm its own existence to a prober. -* A duplicate, a withheld and an ignored mutation all answer 200. A non-2xx - makes bunq redeliver, so only a failed write earns a 500 — the one case - where a retry could actually help. -* State files, all under `/var/lib/catcrafts` and none worth backing up: - `orders.jsonl.financials.json` (the published totals), - `orders.jsonl.financials-seen.json` (ingested ids + withheld counters), and - the rules file above. - -### The weekly reconciliation is the authority - -Callbacks can be missed, replayed or arrive before a rule exists for them, so -this path is allowed to be **lossy but never wrong**: it may withhold, it may -not invent. Your home tooling recomputes every total from the full bunq -mutation history and overwrites `orders.jsonl.financials.json` wholesale — same -format, same file. That is the correction mechanism, and it is what makes it -safe for the live path to publish provisionally. +(A bunq mutation callback used to keep these numbers live; it was retired +2026-08-17 — webhook deregistered, endpoint removed. The code and its +documentation are in git history if a bank feed ever comes back.) ## Verifying a deploy diff --git a/deploy/catcrafts-server.service b/deploy/catcrafts-server.service index fb08ad7..068fcb5 100644 --- a/deploy/catcrafts-server.service +++ b/deploy/catcrafts-server.service @@ -85,14 +85,6 @@ ReadOnlyPaths=/srv/catcrafts-app /srv/catcrafts.net # INVOICE_GPG_KEY=... invoice signing (see deploy/README.md) # MAIL_COMMAND=msmtp -t order confirmation email (see deploy/README.md, # MAIL_FROM=... "Order email"); unset = no email is sent -# BUNQ_CALLBACK_SECRET=... the last segment of the bunq mutation callback -# URL, and what brings that endpoint into being: -# unset, /api/bunq/* is a plain 404. NOT an API -# key — no bunq key belongs on this box, because -# one can initiate payments (deploy/README.md, -# "The bunq mutation callback") -# BUNQ_CALLBACK_PUBKEY=... path to bunq's server public key in PEM; set it -# to REQUIRE a valid RSA-SHA256 body signature # The '-' prefix makes the file optional: without it the server starts with # payments off and the shop renders but refuses checkout — degraded, not down. EnvironmentFile=-/etc/catcrafts/payments.env diff --git a/project.cpp b/project.cpp index 41a7f9e..5207c56 100644 --- a/project.cpp +++ b/project.cpp @@ -87,9 +87,9 @@ static Configuration* SharedLibrary(std::span args) { // Catcrafts.ServerCore — every implementation unit of the server except // main.cpp, as a static library. The split exists for the tests: a test can // only LINK a library-type dependency (crafter-build exports -L/-l for -// libraries alone), and the Mollie/EURC/Sendcloud parsers, the invoice -// builder and the bunq classifier all live here. The exe is main.cpp plus -// this library, so the split costs nothing at deploy time. +// libraries alone), and the Mollie/EURC/Sendcloud parsers and the invoice +// builder all live here. The exe is main.cpp plus this library, so the +// split costs nothing at deploy time. // // Same `static unique_ptr` convention as SharedLibrary, and the same warning: // cfg.dependencies holds a raw pointer, so the Configuration must outlive @@ -121,13 +121,11 @@ static Configuration* ServerCore(std::span args, Configu core->GetInterfacesAndImplementations(ifaces, impls); // Both rails reach their provider over TLS, which is what libssl is for - // here. libcrypto is named again on its own account: the bunq mutation - // callback verifies an RSA-SHA256 body signature through EVP, so this - // library calls libcrypto directly rather than only inheriting it as - // libssl's dependency. On the lib rather than the exe because link flags - // propagate to consumers — the exe and every test get them from here. + // here; libcrypto arrives as its dependency. (It was named explicitly + // while the bunq callback verified RSA body signatures through EVP — that + // integration is retired.) On the lib rather than the exe because link + // flags propagate to consumers — the exe and every test get them from here. core->linkFlags.push_back("-lssl"); - core->linkFlags.push_back("-lcrypto"); return core.get(); } diff --git a/server/implementations/Catcrafts.Server-Financials.cpp b/server/implementations/Catcrafts.Server-Financials.cpp index 92bc201..a398f87 100644 --- a/server/implementations/Catcrafts.Server-Financials.cpp +++ b/server/implementations/Catcrafts.Server-Financials.cpp @@ -6,44 +6,23 @@ The source code of this website is made available for viewing purposes only. No permission is granted to copy, modify, distribute, or create derivative works. */ -// The public financials aggregates, and the bunq callback that keeps them live. +// The public financials aggregates. // -// The page at /financials shows running totals only. Sales fold out of the -// order ledger on every request (Orders.cpp); donations and expenses come from -// the aggregates file this unit owns, and bunq's mutation callback is what -// moves those numbers the moment money does. +// The page at /financials shows running totals only. Sales and shop donations +// fold out of the order ledger on every request (Orders.cpp); the bank-side +// donations and the expenses come from the aggregates file this unit reads — +// written by the owner's own tooling, off this box, and re-read per request so +// updating the file is all it takes to update the page. // -// THE PRIVACY RULE, which is the reason this unit is shaped the way it is: a -// bank mutation arrives here carrying a counterparty name, an IBAN and a -// description. NONE of that is ever written down. A mutation is classified, -// its amount is added to a category total, and its opaque id goes in a -// dedup ledger so a retry cannot double-count it. Everything else is dropped -// on the floor before anything is persisted. There is therefore no file on -// this box that the callback path could leak a donor's identity from, because -// no such file is ever written. -// -// CLASSIFICATION IS DEFAULT-DENY. A mutation that no rule claims is NOT -// published — not as "other", not as a guess. It is counted and logged for the -// operator, and stays out of the totals until a rule exists for it. Getting a -// number wrong on this page is worse than the number being late. -// -// WHY THE KEY IS NOT HERE. A bunq API key can initiate payments; there is no -// read-only scope. So this box never holds one. The notification filter is -// registered once from the owner's own machine (which is where the key lives, -// IP-bound), and from then on bunq PUSHES here. The server can receive money -// news without being able to move money — which is the whole point. -// -// THE CALLBACK IS PROVISIONAL, THE WEEKLY PULL IS AUTHORITATIVE. Callbacks can -// be missed, replayed or arrive out of order, and the classifier can be wrong -// until a rule is added. The owner's home tooling recomputes every total from -// the full bunq mutation history and overwrites the aggregates file wholesale. -// That is the correction mechanism, and it is why this path is allowed to be -// lossy but never wrong: it may withhold, it may not invent. +// This unit once also held the bunq mutation callback that moved those numbers +// live (parser, default-deny classifier, dedup ledger, signature check). That +// integration was retired 2026-08-17 — the webhook is deregistered and the +// endpoint is gone — and the file is in git history if a bank feed ever comes +// back. What survives is the privacy property the page still promises: the +// only bank-derived state on this box is category totals with an as-of date; +// no transaction, counterparty or timestamp ever reaches disk here. module; -#include -#include -#include module Catcrafts.Server; import std; @@ -65,398 +44,8 @@ std::string ReadStateFile(const std::filesystem::path& p) { return buf.str(); } -// Write via a temp file and rename, so a reader (the /financials handler, or -// the owner's tooling) never observes a half-written document. The aggregates -// file is read on every page request, so a torn write would be a visible -// wrong number rather than a transient. -bool WriteStateFileAtomic(const std::filesystem::path& p, std::string_view data) { - if (p.empty()) return false; - std::filesystem::path tmp = p; - tmp += ".tmp"; - { - std::ofstream out(tmp, std::ios::binary | std::ios::trunc); - if (!out) return false; - out << data; - out.flush(); - if (!out) return false; - } - std::error_code ec; - std::filesystem::permissions(tmp, - std::filesystem::perms::owner_read - | std::filesystem::perms::owner_write, - ec); - std::filesystem::rename(tmp, p, ec); - if (ec) { - std::filesystem::remove(tmp, ec); - return false; - } - return true; -} - -std::string JsonEscapeF(std::string_view s) { - std::string out; - out.reserve(s.size() + 8); - for (const char c : s) { - switch (c) { - case '"': out += "\\\""; break; - case '\\': out += "\\\\"; break; - case '\n': out += "\\n"; break; - case '\r': out += "\\r"; break; - case '\t': out += "\\t"; break; - default: - if (static_cast(c) < 0x20) { - out += std::format("\\u{:04x}", static_cast(c)); - } else { - out += c; - } - } - } - return out; -} - -std::string LowerF(std::string_view s) { - std::string out(s); - for (char& c : out) { - if (c >= 'A' && c <= 'Z') c = static_cast(c - 'A' + 'a'); - } - return out; -} - -// ── the ingest ledger ───────────────────────────────────────────────── -// -// Opaque bunq mutation ids and two counters. No amounts against ids, no -// names, nothing that reconstructs a transaction — this file exists purely so -// a redelivered callback is recognised as one already counted. - -struct SeenLedger { - std::vector ids; - std::int64_t pendingCount = 0; // mutations no rule claimed - std::int64_t pendingMinor = 0; // and what they summed to, for the operator - - bool Has(std::string_view id) const { - return std::find(ids.begin(), ids.end(), id) != ids.end(); - } -}; - -SeenLedger LoadSeen(std::string_view json) { - SeenLedger out; - auto doc = Json::Parse(json); - if (!doc || !doc->IsObject()) return out; - if (const Json::Value* a = doc->Find("ids"); a && a->IsArray()) { - for (const Json::Value& v : a->array) { - if (v.type == Json::Type::String && !v.string.empty()) out.ids.push_back(v.string); - } - } - out.pendingCount = doc->Int("pending_count"); - out.pendingMinor = doc->Int("pending_minor"); - return out; -} - -std::string SerialiseSeen(const SeenLedger& s) { - std::string out = "{\"ids\":["; - for (std::size_t i = 0; i < s.ids.size(); ++i) { - if (i) out += ','; - out += std::format("\"{}\"", JsonEscapeF(s.ids[i])); - } - out += std::format("],\"pending_count\":{},\"pending_minor\":{}}}", - s.pendingCount, s.pendingMinor); - return out; -} - -// The aggregates file, in exactly the shape Shared's LoadFinancials reads and -// the owner's tooling writes. One format, three writers, no translation layer. -std::string SerialiseFinancials(const Financials& f) { - auto categories = [](const std::vector& cats) { - std::string out = "["; - for (std::size_t i = 0; i < cats.size(); ++i) { - if (i) out += ','; - out += std::format(R"({{"label":"{}","total_minor":{}}})", - JsonEscapeF(cats[i].label), cats[i].totalMinor); - } - out += ']'; - return out; - }; - return std::format( - R"({{"as_of":"{}",)" - R"("donations":{{"count":{},"total_minor":{}}},)" - R"("expenses":{}}})", - JsonEscapeF(f.asOf), f.donationCount, f.donationsMinor, - categories(f.expenses)); -} - -// ── crypto ──────────────────────────────────────────────────────────── - -struct PkeyDeleter { - void operator()(EVP_PKEY* p) const { EVP_PKEY_free(p); } -}; - -std::optional> Base64Decode(std::string_view in) { - auto sextet = [](char c) -> int { - if (c >= 'A' && c <= 'Z') return c - 'A'; - if (c >= 'a' && c <= 'z') return c - 'a' + 26; - if (c >= '0' && c <= '9') return c - '0' + 52; - if (c == '+') return 62; - if (c == '/') return 63; - return -1; - }; - std::vector out; - std::uint32_t acc = 0; - int bits = 0; - for (const char c : in) { - if (c == '\n' || c == '\r' || c == ' ' || c == '\t') continue; - if (c == '=') break; - const int v = sextet(c); - if (v < 0) return std::nullopt; // not base64: refuse rather than guess - acc = (acc << 6) | static_cast(v); - bits += 6; - if (bits >= 8) { - bits -= 8; - out.push_back(static_cast((acc >> bits) & 0xff)); - } - } - return out; -} - -// RSA-SHA256 over the raw request body against bunq's server public key. -// -// Optional and off unless a key file is configured, for an honest reason: the -// header bunq signs callbacks with has changed across API generations, and a -// verifier that is wrong about the header name rejects every real callback -// while looking like it is working. Enable it once a real callback has been -// observed carrying a signature — see deploy/README.md. When it IS enabled a -// failure is fatal to the request: no signature, no ingest. -bool SignatureValid(std::string_view body, std::string_view signatureB64, - const std::filesystem::path& pubkeyPath) { - const std::string pem = ReadStateFile(pubkeyPath); - if (pem.empty() || signatureB64.empty()) return false; - const auto sig = Base64Decode(signatureB64); - if (!sig || sig->empty()) return false; - - BIO* bio = BIO_new_mem_buf(pem.data(), static_cast(pem.size())); - if (!bio) return false; - EVP_PKEY* raw = PEM_read_bio_PUBKEY(bio, nullptr, nullptr, nullptr); - BIO_free(bio); - if (!raw) return false; - const std::unique_ptr key(raw); - - EVP_MD_CTX* ctx = EVP_MD_CTX_new(); - if (!ctx) return false; - bool ok = false; - do { - if (EVP_DigestVerifyInit(ctx, nullptr, EVP_sha256(), nullptr, key.get()) != 1) break; - ok = EVP_DigestVerify(ctx, sig->data(), sig->size(), - reinterpret_cast(body.data()), - body.size()) == 1; - } while (false); - EVP_MD_CTX_free(ctx); - return ok; -} - -// Length-independent, content-independent comparison. The secret sits in the -// callback URL, so an attacker can probe it one request at a time; a plain == -// would leak the matching prefix through timing. -bool SecretEqual(std::string_view a, std::string_view b) { - if (a.empty() || b.empty()) return false; - unsigned char diff = a.size() == b.size() ? 0 : 1; - const std::size_t n = std::max(a.size(), b.size()); - for (std::size_t i = 0; i < n; ++i) { - const unsigned char ca = i < a.size() ? static_cast(a[i]) : 0; - const unsigned char cb = i < b.size() ? static_cast(b[i]) : 0; - diff |= static_cast(ca ^ cb); - } - return diff == 0; -} - -// Recursively find the object that describes the payment. bunq wraps the -// payload differently across API generations and event types -// (NotificationUrl -> object -> Payment | MutationCreated | …), so this looks -// for the SHAPE rather than a fixed path: an object carrying an "amount" -// object and an "id". Matching on shape is what keeps a wrapper rename from -// silently turning every callback into a no-op. -const Json::Value* FindPaymentObject(const Json::Value& v) { - if (v.IsObject()) { - const Json::Value* amount = v.Find("amount"); - if (amount && amount->IsObject() && amount->Find("value") && v.Find("id")) return &v; - for (const auto& [k, child] : v.object) { - if (const Json::Value* hit = FindPaymentObject(child)) return hit; - } - } else if (v.IsArray()) { - for (const Json::Value& child : v.array) { - if (const Json::Value* hit = FindPaymentObject(child)) return hit; - } - } - return nullptr; -} - } // namespace -// ── pure parsing and classification (exported for the self-test) ────── - -std::optional ParseSignedAmountToMinor(std::string_view s) { - bool negative = false; - if (s.starts_with('-')) { negative = true; s.remove_prefix(1); } - else if (s.starts_with('+')) { s.remove_prefix(1); } - const auto magnitude = ParseAmountToMinor(s); - if (!magnitude) return std::nullopt; - return negative ? -*magnitude : *magnitude; -} - -std::optional ParseBunqMutation(std::string_view json) { - auto doc = Json::Parse(json); - if (!doc) return std::nullopt; - const Json::Value* pay = FindPaymentObject(*doc); - if (!pay) return std::nullopt; - - BankMutation m; - // bunq sends the id as a JSON number; it travels as text from here, like - // every other provider id in this codebase. - if (const Json::Value* id = pay->Find("id")) { - if (id->type == Json::Type::Number) { - m.id = std::format("{}", static_cast(id->number)); - } else if (id->type == Json::Type::String) { - m.id = id->string; - } - } - if (m.id.empty()) return std::nullopt; - - const Json::Value* amount = pay->Find("amount"); - if (!amount) return std::nullopt; - m.currency = std::string(amount->Str("currency")); - const auto minor = ParseSignedAmountToMinor(amount->Str("value")); - if (!minor) return std::nullopt; - m.amountMinor = *minor; - - if (const Json::Value* cp = pay->Find("counterparty_alias"); cp && cp->IsObject()) { - // The IBAN sits either directly on the alias or under its - // "labelMonetaryAccount"/"iban", depending on the payload flavour. - m.counterpartyIban = std::string(cp->Str("iban")); - if (m.counterpartyIban.empty()) { - if (const Json::Value* lma = cp->Find("labelMonetaryAccount"); - lma && lma->IsObject()) { - m.counterpartyIban = std::string(lma->Str("iban")); - } - } - } - m.description = std::string(pay->Str("description")); - if (const std::int64_t acct = pay->Int("monetary_account_id"); acct != 0) { - m.account = std::format("{}", acct); - } - // "2026-08-14 09:31:02.123456" -> "2026-08-14". Only the date is kept, and - // only to stamp the page's as-of line; the time is dropped here, at the - // parser, so no later code can publish it by accident. - if (const std::string_view created = pay->Str("created"); created.size() >= 10) { - m.created = std::string(created.substr(0, 10)); - } - return m; -} - -FinancialRules LoadFinancialRules(std::string_view json) { - FinancialRules out; - auto doc = Json::Parse(json); - if (!doc || !doc->IsObject()) return out; - if (const Json::Value* a = doc->Find("donation_accounts"); a && a->IsArray()) { - for (const Json::Value& v : a->array) { - if (v.type == Json::Type::String) out.donationAccounts.push_back(v.string); - else if (v.type == Json::Type::Number) { - out.donationAccounts.push_back( - std::format("{}", static_cast(v.number))); - } - } - } - if (const Json::Value* a = doc->Find("rules"); a && a->IsArray()) { - for (const Json::Value& v : a->array) { - if (!v.IsObject()) continue; - FinancialRule r; - r.iban = LowerF(v.Str("iban")); - r.descriptionContains = LowerF(v.Str("description_contains")); - r.account = std::string(v.Str("account")); - r.group = std::string(v.Str("group")); - r.label = std::string(v.Str("label")); - // A rule with no criterion would claim every mutation, which is - // the exact opposite of default-deny. A rule whose group is not - // one this code understands is a typo, and a typo must not - // silently become a published category. - const bool hasCriterion = !r.iban.empty() || !r.descriptionContains.empty() - || !r.account.empty(); - const bool knownGroup = r.group == "donations" || r.group == "expense" - || r.group == "ignore"; - if (!hasCriterion || !knownGroup) continue; - // An expense needs a label to render under; donations and ignore - // do not have one. - if (r.group == "expense" && r.label.empty()) continue; - out.rules.push_back(std::move(r)); - } - } - return out; -} - -MutationClass ClassifyMutation(const BankMutation& m, const FinancialRules& rules) { - MutationClass out; - // Only euro. A foreign-currency mutation has no place in a euro total and - // converting one here would invent a rate. - if (m.currency != "EUR") return out; - - const std::string iban = LowerF(m.counterpartyIban); - const std::string description = LowerF(m.description); - // Explicit rules first, so an "ignore" can carve an exception out of a - // donation account — the owner moving money between their own accounts - // must not read as a gift. - for (const FinancialRule& r : rules.rules) { - if (!r.iban.empty() && r.iban != iban) continue; - if (!r.account.empty() && r.account != m.account) continue; - if (!r.descriptionContains.empty() - && description.find(r.descriptionContains) == std::string::npos) { - continue; - } - out.group = r.group; - out.label = r.label; - return out; - } - // The donation-account default: money ARRIVING on an account dedicated to - // donations is a donation. Keyed on the account rather than the sender - // because donors are strangers — an IBAN allowlist cannot know them, and - // this is the one category that must work for someone who has never paid - // this company before. - if (m.amountMinor > 0 - && std::find(rules.donationAccounts.begin(), rules.donationAccounts.end(), m.account) - != rules.donationAccounts.end()) { - out.group = "donations"; - } - return out; -} - -void ApplyMutation(Financials& fin, const MutationClass& cls, const BankMutation& m) { - auto bump = [&](std::vector& cats) { - for (FinCategory& c : cats) { - if (c.label == cls.label) { - // Outgoing money is negative on the wire and an expense is a - // positive total, so the sign flips here. A refund from a - // supplier arrives positive and correctly REDUCES the - // category rather than appearing as income. - c.totalMinor += -m.amountMinor; - return; - } - } - cats.push_back(FinCategory{ cls.label, -m.amountMinor }); - }; - if (cls.group == "donations") { - fin.donationsMinor += m.amountMinor; - // The count follows money in, not money out: a refunded donation - // reduces the total without pretending the gift never happened. - if (m.amountMinor > 0) ++fin.donationCount; - } else if (cls.group == "expense") { - bump(fin.expenses); - } else { - return; // "ignore" and unclassified touch nothing - } - // The page's freshness line. Never moves backwards: callbacks can arrive - // out of order, and an as-of date that jumped back a week would read as a - // stall that never happened. - if (!m.created.empty() && m.created > fin.asOf) fin.asOf = m.created; -} - -// ── configuration and the live aggregates ───────────────────────────── - void ConfigureFinancials(FinancialsConfig config) { std::lock_guard lock(gFinMutex); gFinCfg = std::move(config); @@ -467,94 +56,4 @@ Financials CurrentFinancials() { return LoadFinancials(ReadStateFile(gFinCfg.publicPath)); } -bool BunqCallbackConfigured() { - std::lock_guard lock(gFinMutex); - return !gFinCfg.callbackSecret.empty() && !gFinCfg.publicPath.empty(); -} - -bool BunqCallbackAuthorised(std::string_view pathSecret, std::string_view body, - std::string_view signatureB64) { - std::filesystem::path pubkey; - { - std::lock_guard lock(gFinMutex); - if (gFinCfg.callbackSecret.empty()) return false; - if (!SecretEqual(pathSecret, gFinCfg.callbackSecret)) return false; - pubkey = gFinCfg.publicKeyPem; - } - if (pubkey.empty()) return true; // signature checking not enabled - return SignatureValid(body, signatureB64, pubkey); -} - -BunqIngestResult IngestBunqNotification(std::string_view body) { - std::lock_guard lock(gFinMutex); - if (gFinCfg.publicPath.empty()) return BunqIngestResult::Ignored; - - const auto mutation = ParseBunqMutation(body); - if (!mutation) { - // Deliberately NOT an error status: bunq retries a non-2xx, so a - // payload shape this parser does not understand would become an - // endless redelivery loop. It is logged instead, and the weekly pull - // is what recovers the money. The log names no field VALUES — the - // point of the log is which shape arrived, not what it said. - std::println(std::cerr, - "catcrafts-server: bunq callback carried no recognisable " - "mutation ({} bytes); the weekly reconciliation will pick " - "it up", body.size()); - return BunqIngestResult::Ignored; - } - - SeenLedger seen = LoadSeen(ReadStateFile(gFinCfg.seenPath)); - if (seen.Has(mutation->id)) return BunqIngestResult::Duplicate; - - const FinancialRules rules = LoadFinancialRules(ReadStateFile(gFinCfg.rulesPath)); - const MutationClass cls = ClassifyMutation(*mutation, rules); - - // Recorded as seen either way: an unclassified mutation must not be - // re-counted as pending on every redelivery, and once a rule exists for - // it the weekly pull is what brings the money in. - seen.ids.push_back(mutation->id); - - BunqIngestResult result = BunqIngestResult::Withheld; - if (cls.group.empty()) { - ++seen.pendingCount; - seen.pendingMinor += mutation->amountMinor; - std::println(std::cerr, - "catcrafts-server: bunq mutation {} matched no rule and is " - "WITHHELD from /financials ({} awaiting classification, " - "{} cents net). Add a rule to {}.", - mutation->id, seen.pendingCount, seen.pendingMinor, - gFinCfg.rulesPath.string()); - } else if (cls.group == "ignore") { - result = BunqIngestResult::Ignored; - } else { - Financials fin = LoadFinancials(ReadStateFile(gFinCfg.publicPath)); - ApplyMutation(fin, cls, *mutation); - // First publication: a file that has never been written has no as-of - // date, and a mutation with no usable created stamp still has to - // produce one or the page would keep saying nothing is published. - if (fin.asOf.empty()) { - fin.asOf = mutation->created.empty() ? std::string("1970-01-01") - : mutation->created; - } - if (!WriteStateFileAtomic(gFinCfg.publicPath, SerialiseFinancials(fin))) { - std::println(std::cerr, - "catcrafts-server: could not write the financials " - "aggregates to {}", gFinCfg.publicPath.string()); - return BunqIngestResult::Failed; - } - result = BunqIngestResult::Applied; - } - - if (!WriteStateFileAtomic(gFinCfg.seenPath, SerialiseSeen(seen))) { - // The money is already published; losing the dedup entry only risks a - // double count on redelivery, so say so loudly rather than fail the - // request and guarantee that redelivery. - std::println(std::cerr, - "catcrafts-server: could not write the financials ingest " - "ledger to {} — a redelivered callback may double-count", - gFinCfg.seenPath.string()); - } - return result; -} - } // namespace Catcrafts::Server diff --git a/server/implementations/Catcrafts.Server-Http.cpp b/server/implementations/Catcrafts.Server-Http.cpp index 5795f9a..5e4080c 100644 --- a/server/implementations/Catcrafts.Server-Http.cpp +++ b/server/implementations/Catcrafts.Server-Http.cpp @@ -76,10 +76,10 @@ PaymentRails gRails; std::string gRedirectBase = "https://catcrafts.net"; // The bank-derived aggregates for /financials live in Catcrafts.Server- -// Financials.cpp, which owns their file and the bunq callback that updates -// them. They are read through CurrentFinancials() per request rather than -// cached: unlike the content files they CAN change under a running process, -// and live is the page's whole promise. +// Financials.cpp, which owns their file. They are read through +// CurrentFinancials() per request rather than cached: unlike the content +// files they CAN change under a running process, and live is the page's +// whole promise. // The reconciler's sweep cadence: the shortest interval any configured rail // asks for. Each order is still paced by ITS OWN rail's interval inside the @@ -98,12 +98,6 @@ std::chrono::seconds SweepInterval() { return out; } -// The callback URL's fixed prefix; everything after it is the shared secret. -// Under /api because Caddy proxies that prefix straight through and the -// analytics ingest censors it out of the public report (deploy/README.md) — -// a URL carrying a secret must not end up on a page anyone can read. -inline constexpr std::string_view kBunqCallbackPrefix = "/api/bunq/"; - std::string ReadFile(const std::filesystem::path& p) { std::ifstream in(p, std::ios::binary); if (!in) return {}; @@ -1132,48 +1126,6 @@ int Serve(std::uint16_t port) { }; auto fallback = [](const HTTPRequest& req) -> HTTPResponse { - // The bunq mutation callback. Handled here rather than through - // ParseRoute because the path carries a SECRET — the shared route - // table is compiled into the wasm bundle that ships to every browser, - // and a secret has no business being in it. - // - // Everything unauthorised answers 404, never 401: the endpoint should - // not confirm its own existence to a prober, exactly as an unknown - // order token does not confirm the shape of a real one. - if (const std::string_view path = PathWithoutQueryHTTP(req.path); - path.starts_with(kBunqCallbackPrefix)) { - HTTPResponse res; - res.headers["content-type"] = "text/plain; charset=utf-8"; - res.headers["cache-control"] = "no-store"; - res.headers["x-robots-tag"] = "noindex, nofollow"; - const std::string_view secret = path.substr(kBunqCallbackPrefix.size()); - if (!BunqCallbackConfigured() || req.method != "POST" - || req.body.size() > Form::kMaxBodyBytes) { - res.status = "404"; - res.body = "Not found\n"; - return res; - } - std::string_view signature; - if (const auto h = req.headers.find("x-bunq-server-signature"); - h != req.headers.end()) { - signature = h->second; - } - if (!BunqCallbackAuthorised(secret, req.body, signature)) { - res.status = "404"; - res.body = "Not found\n"; - return res; - } - // 200 for everything the endpoint understood, including a - // withheld or duplicate mutation: those are correct outcomes, and - // a non-2xx would make bunq redeliver a callback that was already - // handled exactly as intended. Only a failed WRITE earns a 500, - // because a retry of that genuinely could succeed. - const BunqIngestResult result = IngestBunqNotification(req.body); - res.status = result == BunqIngestResult::Failed ? "500" : "200"; - res.body = result == BunqIngestResult::Failed ? "Could not record\n" : "OK\n"; - return res; - } - // A POST to a product page is a checkout submission. if (req.method == "POST") { const Route route = ParseRoute(PathWithoutQueryHTTP(req.path)); diff --git a/server/implementations/main.cpp b/server/implementations/main.cpp index 03bc119..3aa309b 100644 --- a/server/implementations/main.cpp +++ b/server/implementations/main.cpp @@ -243,26 +243,13 @@ int main(int argc, char** argv) { } Server::SetOrdersPath(ordersPath); - // The /financials aggregates and the bunq callback that feeds them. - // Same derivation convention as the rail marker and the shipping - // cache: state hangs off the orders path. The secret is the last - // segment of the callback URL and is what enables the endpoint at - // all; unset means /api/bunq/* is a plain 404. No bunq API KEY is - // ever read here — see Catcrafts.Server-Financials.cpp for why. + // The /financials aggregates. Same derivation convention as the rail + // marker and the shipping cache: state hangs off the orders path. The + // file is written by the owner's tooling and only read here. { Server::FinancialsConfig finCfg; finCfg.publicPath = ordersPath; finCfg.publicPath += ".financials.json"; - finCfg.seenPath = ordersPath; - finCfg.seenPath += ".financials-seen.json"; - finCfg.rulesPath = ordersPath; - finCfg.rulesPath += ".financial-rules.json"; - if (const char* v = std::getenv("BUNQ_CALLBACK_SECRET"); v && *v) { - finCfg.callbackSecret = v; - } - if (const char* v = std::getenv("BUNQ_CALLBACK_PUBKEY"); v && *v) { - finCfg.publicKeyPem = v; - } Server::ConfigureFinancials(std::move(finCfg)); } Server::LoadContent(contentDir, bundleIndex); diff --git a/server/interfaces/Catcrafts.Server.cppm b/server/interfaces/Catcrafts.Server.cppm index 01d7339..9b2b7eb 100644 --- a/server/interfaces/Catcrafts.Server.cppm +++ b/server/interfaces/Catcrafts.Server.cppm @@ -139,36 +139,17 @@ export namespace Catcrafts::Server { // ── the bank side of /financials ────────────────────────────────── // - // Donations and expenses come from a bank mutation callback rather than - // from an API key on this box: a bunq key can INITIATE PAYMENTS and has - // no read-only scope, so the key stays on the owner's machine (IP-bound) - // and is used there once to register a notification filter. From then on - // bunq pushes mutations here, and the server can learn that money moved - // without being able to move any. - // - // What crosses this boundary is deliberately small. A mutation is - // classified, its amount lands in a category total, its opaque id goes in - // a dedup ledger, and the name, IBAN and description are dropped before - // anything is written. Classification is default-deny: money no rule - // claims is withheld from the page and logged, never published as a guess. - // The owner's weekly pull recomputes every total from the full bunq - // history and overwrites the aggregates file, which is what makes this - // path safe to be lossy. + // Bank-side donations and expenses come from an aggregates file written + // by the owner's own tooling, off this box — no bank credential and no + // bank callback exists here (the bunq mutation callback was retired + // 2026-08-17; its code is in git history). Aggregates by construction: + // category totals and an as-of date are all the file can carry, which is + // the page's privacy design. struct FinancialsConfig { std::filesystem::path publicPath; // .financials.json — what - // the page reads; also written by - // the owner's reconciliation - std::filesystem::path seenPath; // .financials-seen.json — - // ingested mutation ids, so a - // redelivery cannot double-count - std::filesystem::path rulesPath; // .financial-rules.json — - // the classifier, authored by hand - std::string callbackSecret; // BUNQ_CALLBACK_SECRET; the last - // path segment of the callback - // URL. Empty = endpoint disabled - std::filesystem::path publicKeyPem; // BUNQ_CALLBACK_PUBKEY; empty = - // signature checking off + // the page reads; written by the + // owner's reconciliation }; void ConfigureFinancials(FinancialsConfig config); @@ -177,72 +158,6 @@ export namespace Catcrafts::Server { // published yet, which the page states rather than showing zeros. Financials CurrentFinancials(); - // One bank mutation, reduced to what a total needs. Everything - // identifying is dropped by the classifier and never persisted. - struct BankMutation { - std::string id; // bunq's opaque id — the dedup key - std::int64_t amountMinor = 0; // SIGNED: negative is money leaving - std::string currency; // only EUR is ever counted - std::string counterpartyIban; // classifier input; never stored - std::string description; // classifier input; never stored - std::string account; // monetary account id - std::string created; // ISO date only — the time is - // dropped at the parser - }; - - // One classification rule. A rule matches when every criterion it states - // matches; the first matching rule wins. `group` is "donations", - // "expense" or "ignore" — anything else is a typo and the rule is dropped - // at load rather than inventing a category. (Expenses were once split - // into recurring/one-off; see Financials::expenses for why that went.) - struct FinancialRule { - std::string iban; // exact, case-insensitive - std::string descriptionContains; // substring, case-insensitive - std::string account; // monetary account id - std::string group; - std::string label; // the page's category name - }; - - struct FinancialRules { - // Accounts whose INCOMING money is a donation by default. Keyed on - // the account because donors are strangers: no IBAN list can know - // them in advance. - std::vector donationAccounts; - std::vector rules; - }; - - // Empty group = no rule claimed it. That is the default-deny answer, and - // the caller must withhold rather than guess. - struct MutationClass { - std::string group; - std::string label; - }; - - // Pure and exported for the self-test. - std::optional ParseSignedAmountToMinor(std::string_view s); - std::optional ParseBunqMutation(std::string_view json); - FinancialRules LoadFinancialRules(std::string_view json); - MutationClass ClassifyMutation(const BankMutation& m, const FinancialRules& rules); - void ApplyMutation(Financials& fin, const MutationClass& cls, const BankMutation& m); - - // Whether the callback endpoint exists at all. Unconfigured, the path is - // a plain 404 — an endpoint that is off should not announce itself. - bool BunqCallbackConfigured(); - - // Constant-time secret check, plus the RSA-SHA256 body signature when a - // public key is configured. False means 404, for the same reason an - // unknown order token is: a probe learns nothing from the shape. - bool BunqCallbackAuthorised(std::string_view pathSecret, std::string_view body, - std::string_view signatureB64); - - enum class BunqIngestResult { Applied, Duplicate, Withheld, Ignored, Failed }; - - // Parse, classify, and fold one notification into the aggregates. Never - // reports a transport-level error for an unparseable body: bunq retries - // non-2xx, so an unknown payload shape would redeliver forever. It is - // logged and left to the weekly reconciliation instead. - BunqIngestResult IngestBunqNotification(std::string_view body); - // ── invoices ────────────────────────────────────────────────────── // // A paid order's invoice: plain markdown, clearsigned with the shop's diff --git a/shared/interfaces/Catcrafts.Shared-Content.cppm b/shared/interfaces/Catcrafts.Shared-Content.cppm index 6c8f26c..f1772a1 100644 --- a/shared/interfaces/Catcrafts.Shared-Content.cppm +++ b/shared/interfaces/Catcrafts.Shared-Content.cppm @@ -246,7 +246,7 @@ export const LegalPage& FinancialsPage() { .sections = { { "How this page works", { - "Sales, and donations made through the shop, come straight from the shop's order ledger and update the moment they are paid. Donations to the bank account and expenses are aggregated from the business bank account by category and carry the date they were last brought up to date. Anything the categoriser does not recognise is held back until it has been classified, never published as a guess.", + "Sales, and donations made through the shop, come straight from the shop's order ledger and update the moment they are paid. Donations to the bank account and expenses are aggregated from the business bank account by category, brought up to date by hand, and carry the date that last happened.", "Everything is a running total in euros on a cash basis: money counts when it moves, not when an invoice says it should. Amounts include VAT where VAT was charged.", } }, { "What is never published", diff --git a/tests/ShouldPublishFinancials/main.cpp b/tests/ShouldPublishFinancials/main.cpp index 6ecdc0c..caa260c 100644 --- a/tests/ShouldPublishFinancials/main.cpp +++ b/tests/ShouldPublishFinancials/main.cpp @@ -6,12 +6,10 @@ The source code of this website is made available for viewing purposes only. No permission is granted to copy, modify, distribute, or create derivative works. */ -// The financials page and the bunq mutation ingest behind it. The callback is -// the only path by which a stranger's money reaches a public number on this -// site, so its parser, its classifier and above all its default-deny -// behaviour are pinned here. A rule that accidentally claims everything, or a -// classifier that treats an unrecognised transfer as a donation, would -// publish a figure that is simply untrue. +// The financials page: the aggregates loader, the renderer's two donation +// sources (bank file + shop ledger), and the sales/donations split over the +// order fold. The bank-callback ingest this suite once pinned was retired +// with the bunq integration — see git history. import std; import Catcrafts.Shared; @@ -185,294 +183,10 @@ void FinancialsPage() { "financials: empty ledger sums to zero"); } -// ── the bunq mutation callback ──────────────────────────────────────── -void BunqIngest() { - using Server::ParseSignedAmountToMinor; - Check(ParseSignedAmountToMinor("25.00") == 2500, "bunq: positive amount"); - Check(ParseSignedAmountToMinor("-12.50") == -1250, "bunq: outgoing is negative"); - Check(ParseSignedAmountToMinor("+5") == 500, "bunq: explicit plus"); - Check(!ParseSignedAmountToMinor("1.234").has_value(), "bunq: too many decimals"); - Check(!ParseSignedAmountToMinor("nonsense").has_value(), "bunq: non-numeric"); - Check(!ParseSignedAmountToMinor("").has_value(), "bunq: empty amount"); - - // A realistic payload: the mutation is nested two wrappers deep, and - // the parser finds it by SHAPE so a wrapper rename cannot silently - // turn every callback into a no-op. - constexpr std::string_view kPayload = - R"({"NotificationUrl":{"target_url":"https://catcrafts.net/api/bunq/s",)" - R"("category":"MUTATION","event_type":"MUTATION_CREATED","object":{"Payment":{)" - R"("id":4823,"created":"2026-08-14 09:31:02.123456","monetary_account_id":9911,)" - R"("amount":{"currency":"EUR","value":"25.00"},)" - R"("description":"Thanks for imsd!",)" - R"("counterparty_alias":{"iban":"NL55BUNQ2025123456","display_name":"A Donor"}}}}})"; - const auto m = Server::ParseBunqMutation(kPayload); - Check(m.has_value(), "bunq: nested payload parses"); - if (m) { - Check(m->id == "4823", "bunq: numeric id travels as text"); - Check(m->amountMinor == 2500 && m->currency == "EUR", "bunq: amount and currency"); - Check(m->account == "9911", "bunq: monetary account"); - Check(m->counterpartyIban == "NL55BUNQ2025123456", "bunq: counterparty iban"); - // The time of day never survives the parser: an exact timestamp - // is the one field that would let a watcher pin a donation to a - // person who mentioned donating. - Check(m->created == "2026-08-14", "bunq: only the date is kept"); - } - Check(!Server::ParseBunqMutation("garbage").has_value(), "bunq: malformed payload"); - Check(!Server::ParseBunqMutation(R"({"NotificationUrl":{"category":"MUTATION"}})") - .has_value(), - "bunq: a notification with no mutation yields nothing"); - - // The same payload with one field swapped, so every case below differs - // from the parsing case above by exactly the thing under test. - auto payloadWith = [](std::string_view amountObject, std::string_view alias) { - std::string out; - out += R"({"NotificationUrl":{"category":"MUTATION","object":{"Payment":{)"; - out += R"("id":4823,"created":"2026-08-14 09:31:02.123456",)"; - out += R"("monetary_account_id":9911,"amount":)"; - out += amountObject; - out += R"(,"description":"Thanks for imsd!","counterparty_alias":)"; - out += alias; - out += R"(}}}})"; - return out; - }; - constexpr std::string_view kDonorAlias = - R"({"iban":"NL55BUNQ2025123456","display_name":"A Donor"})"; - - // FindPaymentObject matches on the SHAPE — an amount object carrying a - // value, plus an id — and never looks at what the value SAYS. So a - // locale-mangled or hostile amount reaches the parser inside an otherwise - // perfectly well-formed mutation, and the refusal has to happen here. If - // it ever softened to a zero fallback the mutation would be recorded as - // seen, permanently deduped, with the money dropped from the totals and - // nothing in the operator log to say so. - Check(!Server::ParseBunqMutation( - payloadWith(R"({"currency":"EUR","value":"25,00"})", kDonorAlias)) - .has_value(), - "bunq: a comma decimal is refused rather than read as zero"); - Check(!Server::ParseBunqMutation( - payloadWith(R"({"currency":"EUR","value":"1.234"})", kDonorAlias)) - .has_value(), - "bunq: a third fraction digit is refused rather than truncated"); - Check(!Server::ParseBunqMutation( - payloadWith(R"({"currency":"EUR","value":"abc"})", kDonorAlias)) - .has_value(), - "bunq: a non-numeric amount is refused"); - - const Server::FinancialRules rules = Server::LoadFinancialRules( - R"({"donation_accounts":[9911],)" - R"("rules":[)" - R"({"iban":"NL01OWNSELF0000000","group":"ignore"},)" - R"({"description_contains":"hetzner","group":"expense","label":"Hosting"},)" - R"({"iban":"DE02SUPPLIER000000","group":"expense","label":"Inventory"},)" - R"({"group":"expense","label":"Claims everything"},)" - R"({"iban":"NL03TYPO0000000000","group":"nonsense","label":"X"},)" - R"({"iban":"NL04NOLABEL0000000","group":"expense"}]})"); - Check(rules.donationAccounts.size() == 1 && rules.donationAccounts[0] == "9911", - "bunq: numeric donation account loads as text"); - // Three of the six survive: the criterion-less rule would claim every - // mutation, the typo'd group is not a category, and an expense with - // no label has nothing to render as. - Check(rules.rules.size() == 3, "bunq: unsafe rules are dropped at load"); - - // Incoming on the donation account, claimed by no explicit rule. - Check(m && Server::ClassifyMutation(*m, rules).group == "donations", - "bunq: incoming on the donation account is a donation"); - - Server::BankMutation x = *m; - // Money LEAVING the donation account is not a gift to this company. - x.amountMinor = -2500; - Check(Server::ClassifyMutation(x, rules).group.empty(), - "bunq: outgoing on the donation account is not a donation"); - // An explicit ignore beats the donation-account default, which is how - // the owner's own transfer between accounts stays out of the total. - x = *m; - x.counterpartyIban = "nl01ownself0000000"; - Check(Server::ClassifyMutation(x, rules).group == "ignore", - "bunq: an explicit rule beats the donation default, case-insensitively"); - // Foreign currency is never folded into a euro total. - x = *m; - x.currency = "USD"; - Check(Server::ClassifyMutation(x, rules).group.empty(), - "bunq: non-euro is never counted"); - // Default-deny: an ordinary transfer from a stranger, on an account - // that is not the donation one, is withheld rather than guessed at. - x = *m; - x.account = "1234"; - x.counterpartyIban = "NL99UNKNOWN0000000"; - x.description = ""; - Check(Server::ClassifyMutation(x, rules).group.empty(), - "bunq: an unmatched mutation is withheld, not guessed"); - - // A payload with no "currency" key at all still has the shape the parser - // needs, so it parses — and is then refused by the classifier, which is - // where the euro-only rule lives. Nothing reaches a euro total on the - // strength of a field that was never sent. - const auto noCurrency = - Server::ParseBunqMutation(payloadWith(R"({"value":"25.00"})", kDonorAlias)); - Check(noCurrency && noCurrency->amountMinor == 2500 && noCurrency->currency.empty(), - "bunq: an amount with no currency still parses"); - Check(noCurrency && Server::ClassifyMutation(*noCurrency, rules).group.empty(), - "bunq: an unstated currency is never assumed to be euro"); - - // bunq's other alias flavour nests the IBAN one level down, under - // "labelMonetaryAccount". If that fallback broke, the IBAN would come - // back empty, the owner's own transfer INTO the donation account would - // stop matching its ignore rule, and the donation-account default would - // publish the owner's own money as a stranger's gift — on the one page - // whose entire promise is that the number is true. - const auto nested = Server::ParseBunqMutation(payloadWith( - R"({"currency":"EUR","value":"25.00"})", - R"({"labelMonetaryAccount":{"iban":"NL01OWNSELF0000000","display_name":"Self"}})")); - Check(nested && nested->counterpartyIban == "NL01OWNSELF0000000", - "bunq: the nested alias flavour still yields an iban"); - Check(nested && Server::ClassifyMutation(*nested, rules).group == "ignore", - "bunq: the owner's own transfer in is ignored, whichever alias shape carries it"); - - Server::BankMutation bill; - bill.currency = "EUR"; - bill.amountMinor = -1200; - bill.description = "HETZNER ONLINE GMBH invoice"; - bill.created = "2026-08-15"; - const Server::MutationClass billClass = Server::ClassifyMutation(bill, rules); - Check(billClass.group == "expense" && billClass.label == "Hosting", - "bunq: description matching, case-insensitively"); - - // Folding into the aggregates. - Financials fin; - Server::ApplyMutation(fin, Server::ClassifyMutation(*m, rules), *m); - Check(fin.donationCount == 1 && fin.donationsMinor == 2500, - "bunq: a donation moves the count and the total"); - Check(fin.asOf == "2026-08-14", "bunq: as-of follows the mutation date"); - Server::ApplyMutation(fin, billClass, bill); - Check(fin.expenses.size() == 1 && fin.expenses[0].label == "Hosting" - && fin.expenses[0].totalMinor == 1200, - "bunq: an outgoing bill becomes a positive expense"); - Check(fin.asOf == "2026-08-15", "bunq: as-of advances"); - // A supplier refund reduces the category rather than appearing as - // income, and never drags the as-of date backwards. - Server::BankMutation refund = bill; - refund.amountMinor = 500; - refund.created = "2026-08-01"; - Server::ApplyMutation(fin, billClass, refund); - Check(fin.expenses[0].totalMinor == 700, "bunq: a refund reduces its category"); - Check(fin.asOf == "2026-08-15", "bunq: as-of never moves backwards"); - // An unclassified mutation touches nothing at all. - const Financials before = fin; - Server::ApplyMutation(fin, Server::MutationClass{}, *m); - Check(fin.donationCount == before.donationCount - && fin.ExpensesMinor() == before.ExpensesMinor(), - "bunq: an unclassified mutation changes no total"); - - // A REFUNDED gift. Reachable because an explicit rule may name a group - // outright, so "donations" is not the exclusive property of the - // incoming-only account default tested above. - const Server::FinancialRules donationRules = Server::LoadFinancialRules( - R"({"rules":[{"iban":"NL55BUNQ2025123456","group":"donations"}]})"); - Server::BankMutation giftBack = *m; - giftBack.amountMinor = -1000; - const Server::MutationClass backClass = - Server::ClassifyMutation(giftBack, donationRules); - Check(backClass.group == "donations", - "bunq: an explicit rule can classify outgoing money as a donation"); - // The count follows money IN, never money out: 2500 - 1000 = 1500, and - // the one person who gave still gave. Decrementing here would put the - // published donor count below the number of people who actually donated, - // and the weekly reconciliation folds through this same function — it - // would reproduce the wrong figure rather than correct it. - Financials gifts; - gifts.donationsMinor = 2500; - gifts.donationCount = 1; - Server::ApplyMutation(gifts, backClass, giftBack); - Check(gifts.donationsMinor == 1500 && gifts.donationCount == 1, - "bunq: a refunded gift reduces the total and leaves the count alone"); - - // Two expenses under different labels are two rows, in first-seen order. - // Merging them would hide what the money went on behind one bigger - // number, which is the opposite of what this page is for. - Server::BankMutation supplier; - supplier.currency = "EUR"; - supplier.amountMinor = -5000; - supplier.counterpartyIban = "DE02SUPPLIER000000"; - supplier.created = "2026-08-16"; - const Server::MutationClass supplierClass = Server::ClassifyMutation(supplier, rules); - Check(supplierClass.group == "expense" && supplierClass.label == "Inventory", - "bunq: iban matching picks the supplier's category"); - Financials twoCats; - Server::ApplyMutation(twoCats, billClass, bill); // -1200 out → +1200 Hosting - Server::ApplyMutation(twoCats, supplierClass, supplier); // -5000 out → +5000 Inventory - Check(twoCats.expenses.size() == 2 - && twoCats.expenses[0].label == "Hosting" - && twoCats.expenses[0].totalMinor == 1200 - && twoCats.expenses[1].label == "Inventory" - && twoCats.expenses[1].totalMinor == 5000, - "bunq: distinct labels become distinct rows, in first-seen order"); - Check(twoCats.ExpensesMinor() == 6200, "bunq: the expense total is the sum of its rows"); -} - -// ── the callback gate ───────────────────────────────────────────────── -// -// The one endpoint that writes public money figures, and the only thing -// standing in front of it. Driven through ConfigureFinancials because that is -// how the real server reaches it; no key material and no network are needed -// to pin the parts that matter. -void CallbackGate() { - // Unconfigured: the path is a plain 404 and nothing authorises. An - // endpoint that is off should not announce itself by answering - // differently to a well-formed guess than to an empty one. - Server::ConfigureFinancials(Server::FinancialsConfig{}); - Check(!Server::BunqCallbackConfigured(), - "callback: with no secret the endpoint does not exist"); - Check(!Server::BunqCallbackAuthorised("", "{}", ""), - "callback: an empty secret authorises nothing while unconfigured"); - Check(!Server::BunqCallbackAuthorised("s3cret-not-real", "{}", ""), - "callback: even a well-formed secret is refused while unconfigured"); - - Server::FinancialsConfig cfg; - cfg.callbackSecret = "s3cret-not-real"; // never a live one: the real - // secret only ever comes from - // the environment on the box - Server::ConfigureFinancials(cfg); - Check(Server::BunqCallbackAuthorised("s3cret-not-real", "{}", ""), - "callback: the exact secret is authorised"); - // SecretEqual folds a length mismatch into the same accumulator as the - // byte differences, so neither a prefix nor an extension can return early - // — a plain == would leak the secret one byte at a time through timing, - // and the secret sits in the URL where it can be probed a request at a - // time. - Check(!Server::BunqCallbackAuthorised("s3cret-not-rea", "{}", ""), - "callback: a prefix of the secret is refused"); - Check(!Server::BunqCallbackAuthorised("s3cret-not-realX", "{}", ""), - "callback: an extension of the secret is refused"); - Check(!Server::BunqCallbackAuthorised("", "{}", ""), - "callback: an empty secret never matches a configured one"); - // A secret with nowhere to write the aggregates is still no endpoint: - // this is what keeps the path a 404 on a box that has the env var but - // not the storage. - Check(!Server::BunqCallbackConfigured(), - "callback: a secret without an aggregates path leaves the endpoint off"); - - cfg.publicPath = "/nonexistent-catcrafts/financials.json"; - Server::ConfigureFinancials(cfg); - Check(Server::BunqCallbackConfigured(), - "callback: secret plus aggregates path is what turns the endpoint on"); - - // Turning signature checking ON must never become a no-op. With a key - // path that cannot be read there is no way to verify anything, so the - // CORRECT secret now fails too — closed, not open. - cfg.publicKeyPem = "/nonexistent-catcrafts/bunq-public-key.pem"; - Server::ConfigureFinancials(cfg); - Check(!Server::BunqCallbackAuthorised("s3cret-not-real", "{}", "YWJj"), - "callback: signature checking with an unreadable key fails closed"); - - Server::ConfigureFinancials(Server::FinancialsConfig{}); // leave no global behind -} - } // namespace int main() { FinancialsPage(); - BunqIngest(); - CallbackGate(); if (failures != 0) { std::println(std::cerr, "{} check(s) failed", failures); diff --git a/tests/ShouldServeFinancialsLive/main.cpp b/tests/ShouldServeFinancialsLive/main.cpp index 7d1c447..2e91874 100644 --- a/tests/ShouldServeFinancialsLive/main.cpp +++ b/tests/ShouldServeFinancialsLive/main.cpp @@ -6,58 +6,18 @@ The source code of this website is made available for viewing purposes only. No permission is granted to copy, modify, distribute, or create derivative works. */ -// The financials page over real HTTP, and the bunq mutation callback that -// feeds it. Liveness is the page's promise: the aggregates file appears and -// the very next request reflects it — no restart, no cache, no delay. The -// callback is the only path by which a stranger's money reaches a public -// number, so idempotency, default-deny, and the no-leak guarantees are pinned -// against the real endpoint here. +// The financials page over real HTTP. Liveness is the page's promise: the +// aggregates file appears (written by the owner's tooling — the retired bunq +// callback used to do this; see git history) and the very next request +// reflects it — no restart, no cache, no delay. import std; import Catcrafts.E2eHarness; using namespace Catcrafts::E2e; -namespace { - -// fin_attr -> its value on the live page -std::string FinAttr(TestServer& srv, std::string_view attr) { - const std::string body = srv.Body("/financials"); - std::smatch m; - if (std::regex_search(body, m, std::regex(std::format(R"lit({}="([0-9]*)")lit", attr)))) { - return m[1].str(); - } - return {}; -} - -std::string BunqPayload(std::string_view id, std::string_view account, - std::string_view value, std::string_view iban, - std::string_view description) { - std::string p = R"({"NotificationUrl":{"category":"MUTATION","event_type":"MUTATION_CREATED","object":{"Payment":{"id":)"; - p += id; - p += R"(,"created":"2026-08-15 09:31:02.000000","monetary_account_id":)"; - p += account; - p += R"(,"amount":{"currency":"EUR","value":")"; - p += value; - p += R"("},"description":")"; - p += description; - p += R"(","counterparty_alias":{"iban":")"; - p += iban; - p += R"(","display_name":"Someone"}}}}})"; - return p; -} - -} // namespace - int main(int argc, char** argv) { - // The secret IS the last segment of the callback URL, and setting it is - // what brings the endpoint into existence — unset, the path is an - // ordinary 404. Note what is NOT here: a bunq API key. One could initiate - // payments, so no such key ever reaches the server; it only receives. - constexpr std::string_view kSecret = "e2e-callback-secret-not-a-real-one"; - ServerOptions options; - options.env.emplace_back("BUNQ_CALLBACK_SECRET", std::string(kSecret)); - TestServer srv(argv[1], 8215, options); + TestServer srv(argv[1], 8215); // ── the financials page ─────────────────────────────────────────── // Aggregate-only by construction: totals and counts, machine-readable via @@ -71,8 +31,8 @@ int main(int argc, char** argv) { srv.BodyLacks("/financials", "data-fin-donations-count", "no donation figures before the file exists"); - // The aggregates file appears, exactly as the owner's tooling will write - // it, and the very next request reflects it — this is the liveness the + // The aggregates file appears, exactly as the owner's tooling writes it, + // and the very next request reflects it — this is the liveness the // donation counter depends on. WriteFile(std::filesystem::path(srv.Orders().string() + ".financials.json"), R"({"as_of":"2026-08-14",)" @@ -96,88 +56,5 @@ int main(int argc, char** argv) { srv.BodyHas("/financials", "Inventory", "a second expense category renders"); srv.BodyHas("/financials", "2026-08-14", "bank figures carry their as-of date"); - // ── the bunq mutation callback ──────────────────────────────────── - // The rules are written here rather than at startup on purpose — they are - // re-read per callback, so a new rule takes effect without a restart. - WriteFile(std::filesystem::path(srv.Orders().string() + ".financial-rules.json"), - R"({"donation_accounts":[9911],)" - "\n" - R"( "rules":[{"description_contains":"hetzner","group":"expense","label":"Hosting"},)" - "\n" - R"( {"iban":"NL01OWNSELF0000000","group":"ignore"}]})" - "\n"); - - const std::string cb = std::format("/api/bunq/{}", kSecret); - auto bunqPost = [&](std::string_view id, std::string_view account, - std::string_view value, std::string_view iban, - std::string_view description) { - return srv.Post(cb, BunqPayload(id, account, value, iban, description), - "application/json").status; - }; - - // An endpoint guarded by a secret must not confirm its own existence: - // every unauthorised shape is the same 404 an unknown order token gets. - { - const auto wrong = srv.Post("/api/bunq/wrong-secret", "{}", "application/json"); - Check(wrong.status == "404", "POST /api/bunq/wrong-secret -> 404", wrong.status); - } - srv.CheckStatus(cb, "404"); // GET on the right URL is still not a callback - srv.CheckStatus(cb, "404", "HEAD"); - - // A donation arrives on the donation account. No rule names the sender — - // donors are strangers, which is exactly why the account is what - // classifies. - Check(bunqPost("4823", "9911", "25.00", "NL55BUNQ2025123456", "Thanks for imsd") == "200", - "the callback accepts a mutation"); - Check(FinAttr(srv, "data-fin-donations-count") == "4" - && FinAttr(srv, "data-fin-donations-minor") == "7000", - "a donation ticks the public counter immediately"); - - // bunq redelivers a callback it did not see a 2xx for, and can redeliver - // one it did. Counting that twice would publish money that never arrived. - bunqPost("4823", "9911", "25.00", "NL55BUNQ2025123456", "Thanks for imsd"); - Check(FinAttr(srv, "data-fin-donations-count") == "4" - && FinAttr(srv, "data-fin-donations-minor") == "7000", - "a redelivered mutation is not counted twice"); - - // Default-deny: money no rule claims is WITHHELD from the page. It is - // logged for classification, never published as a guess. - Check(bunqPost("4824", "1234", "90.00", "NL99UNKNOWN00000000", "unlabelled transfer") == "200", - "an unclassifiable mutation is still accepted (no redelivery loop)"); - Check(FinAttr(srv, "data-fin-donations-count") == "4" - && FinAttr(srv, "data-fin-expenses-minor") == "231200", - "an unclassified mutation is withheld from every total"); - - // An outgoing bill matched by description becomes a positive expense. - bunqPost("4825", "9911", "-12.00", "DE00HETZNER00000000", "HETZNER ONLINE GMBH"); - Check(FinAttr(srv, "data-fin-expenses-minor") == "232400", - "an outgoing bill lands in its expense category"); - srv.BodyHas("/financials", "2026-08-15", "the as-of date advances with the mutations"); - - // The page still publishes nothing but aggregates: no counterparty, no - // description, no id, no timestamp. This is the assertion that would - // catch a well-meant future edit adding a "recent activity" list. - for (std::string_view leak : { "NL55BUNQ2025123456", "Someone", "Thanks for imsd", - "4823", "09:31" }) { - srv.BodyLacks("/financials", std::string(leak), - std::format("financials leak no transaction detail ({})", leak)); - } - - // And nothing identifying was written to disk either — the ingest ledger - // holds opaque ids and counters, and no other file learned the donor - // exists. - { - bool persisted = false; - for (const auto& entry : - std::filesystem::recursive_directory_iterator(srv.Work())) { - if (!entry.is_regular_file()) continue; - if (ReadFile(entry.path()).find("NL55BUNQ2025123456") != std::string::npos) { - persisted = true; - std::println(std::cerr, " IBAN found in {}", entry.path().string()); - } - } - Check(!persisted, "no counterparty IBAN is persisted anywhere"); - } - return Finish(); } diff --git a/tests/harness/Catcrafts.E2eHarness.cppm b/tests/harness/Catcrafts.E2eHarness.cppm index 0345a82..b062b6d 100644 --- a/tests/harness/Catcrafts.E2eHarness.cppm +++ b/tests/harness/Catcrafts.E2eHarness.cppm @@ -108,7 +108,7 @@ struct ServerOptions { // attach the signed invoice, shell out — runs with zero network. Each // accepted message lands as its own mail-.eml. bool mailer = false; - // Extra environment for the server (e.g. BUNQ_CALLBACK_SECRET). + // Extra environment for the server. std::vector> env; }; @@ -129,7 +129,7 @@ public: // suites assert. for (const char* v : { "MOLLIE_API_KEY", "EURC_CHAINS", "EURC_POOL", "SENDCLOUD_PUBLIC_KEY", "SENDCLOUD_SECRET_KEY", - "SENDCLOUD_METHOD", "BUNQ_CALLBACK_SECRET", + "SENDCLOUD_METHOD", "INVOICE_GPG_KEY", "MAIL_COMMAND", "MAIL_FROM" }) { ::unsetenv(v); } diff --git a/tools/bunq-callback.sh b/tools/bunq-callback.sh deleted file mode 100755 index 5609fe9..0000000 --- a/tools/bunq-callback.sh +++ /dev/null @@ -1,442 +0,0 @@ -#!/bin/sh -# Register the /financials mutation callback with bunq. -# -# RUN THIS FROM THE MACHINE THE KEY BELONGS TO — your own, not the server. -# That is the whole point: a bunq API key can initiate payments and has no -# read-only scope, so it never goes on the internet-facing box. This script -# uses it once, here, to tell bunq "push mutations to that URL". Afterwards -# bunq talks to the server and the key stays home. -# -# tools/bunq-callback.sh list # accounts + current filters -# tools/bunq-callback.sh set # install the filter -# -# The key comes from BUNQ_KEY in the repo-root .env (gitignored). Session -# state — the RSA keypair, the installation token, whether the device was -# registered — persists in ./bunq-state.json (gitignored, 0600), so re-runs -# reuse the registration instead of making a new one every time. -# -# PERMITTED IPS: the device registration binds the key to the addresses that -# may USE it. This defaults to this machine's current public address, NOT the -# server's — bunq delivers callbacks outbound to an HTTPS URL, which has -# nothing to do with this list, so whitelisting the server would grant -# bank-API access to the box most exposed to attack and buy nothing. Override -# with BUNQ_PERMITTED_IP if your address has moved. -# -# Note your home address is probably dynamic: when the ISP rotates it, API -# calls from here start failing with a permission error. Re-running is not -# enough — a device registration is per-key and cannot be re-pointed — so the -# recovery is to add the new address to the existing device via the bunq app, -# or to accept "*" and rely on the key secret alone. The callback itself keeps -# working throughout; only your ability to run this script from here breaks. -set -eu - -API_HOST="${BUNQ_API_HOST:-api.bunq.com}" -STATE="${BUNQ_STATE:-bunq-state.json}" -UA='catcrafts.net-tools/1.0 (+https://catcrafts.net)' - -[ -f .env ] || { echo "bunq: no .env in $(pwd) — run from the repo root" >&2; exit 1; } -# shellcheck disable=SC1091 -BUNQ_KEY=$(sed -n 's/^BUNQ_KEY=//p' .env | head -n1 | tr -d '"'"'"'') -[ -n "$BUNQ_KEY" ] || { echo "bunq: BUNQ_KEY is empty in .env" >&2; exit 1; } - -for bin in openssl curl python3; do - command -v "$bin" >/dev/null || { echo "bunq: $bin is required" >&2; exit 1; } -done - -# ── state ───────────────────────────────────────────────────────────── -# One JSON file, 0600: it holds a private key. - -state_get() { - [ -f "$STATE" ] || { echo ""; return; } - python3 -c 'import json,sys -try: print(json.load(open(sys.argv[1])).get(sys.argv[2],"") or "") -except Exception: print("")' "$STATE" "$1" -} - -state_set() { - python3 -c 'import json,os,sys -p=sys.argv[1] -try: d=json.load(open(p)) -except Exception: d={} -d[sys.argv[2]]=sys.argv[3] -fd=os.open(p,os.O_WRONLY|os.O_CREAT|os.O_TRUNC,0o600) -with os.fdopen(fd,"w") as f: json.dump(d,f)' "$STATE" "$1" "$2" - chmod 600 "$STATE" -} - -json_str() { python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()))'; } -json_get() { - # json_get - python3 -c 'import json,sys -def walk(o,k): - if isinstance(o,dict): - if k in o: yield o[k] - for v in o.values(): yield from walk(v,k) - elif isinstance(o,list): - for v in o: yield from walk(v,k) -try: d=json.load(sys.stdin) -except Exception: sys.exit(1) -for hit in walk(d,sys.argv[1]): - print(hit if not isinstance(hit,(dict,list)) else json.dumps(hit)); break' "$1" -} - -# ── one signed API call ─────────────────────────────────────────────── -# bunq stopped REQUIRING body signatures in 2019, but signing costs nothing -# and a signed request is valid whether or not the server checks. - -api() { # api - _m="$1"; _p="$2"; _b="$3"; _t="${4:-}" - set -- -sS -X "$_m" \ - -H "user-agent: $UA" -H 'cache-control: no-cache' \ - -H "x-bunq-client-request-id: $(openssl rand -hex 8)" \ - -H 'x-bunq-geolocation: 0 0 0 0 000' \ - -H 'x-bunq-language: en_US' -H 'x-bunq-region: nl_NL' - if [ -n "$_b" ]; then - _sig=$(printf '%s' "$_b" | openssl dgst -sha256 -sign "$KEYFILE" | openssl base64 -A) - set -- "$@" -H 'content-type: application/json' \ - -H "x-bunq-client-signature: $_sig" --data-binary "$_b" - fi - [ -n "$_t" ] && set -- "$@" -H "x-bunq-client-authentication: $_t" - curl "$@" "https://$API_HOST$_p" -} - -die_on_error() { # reads a response, prints it and exits if it carries an Error - _r="$1" - if printf '%s' "$_r" | grep -q '"Error"'; then - echo "bunq refused the call:" >&2 - printf '%s\n' "$_r" | python3 -m json.tool >&2 2>/dev/null || printf '%s\n' "$_r" >&2 - exit 1 - fi -} - -# ── handshake: installation -> device-server -> session ─────────────── - -KEYFILE="${BUNQ_KEYFILE:-bunq-client-key.pem}" -if [ ! -f "$KEYFILE" ]; then - echo "bunq: generating a client keypair -> $KEYFILE" - openssl genrsa -out "$KEYFILE" 2048 2>/dev/null - chmod 600 "$KEYFILE" -fi - -INSTALL_TOKEN=$(state_get installation_token) -if [ -z "$INSTALL_TOKEN" ]; then - echo "bunq: registering the installation (once, ever)" - _pub=$(openssl rsa -in "$KEYFILE" -pubout 2>/dev/null | json_str) - _resp=$(api POST /v1/installation "{\"client_public_key\":$_pub}" "") - die_on_error "$_resp" - INSTALL_TOKEN=$(printf '%s' "$_resp" | json_get token) - [ -n "$INSTALL_TOKEN" ] || { echo "bunq: no installation token in the response" >&2; exit 1; } - state_set installation_token "$INSTALL_TOKEN" - # bunq's own public key arrives here. Keep it: it is what - # BUNQ_CALLBACK_PUBKEY on the server verifies callback signatures against. - printf '%s' "$_resp" | json_get server_public_key > bunq-server-public-key.pem || true - [ -s bunq-server-public-key.pem ] \ - && echo "bunq: saved bunq-server-public-key.pem (for BUNQ_CALLBACK_PUBKEY)" -fi - -if [ "$(state_get device_registered)" != "yes" ]; then - PERMITTED_IP="${BUNQ_PERMITTED_IP:-$(curl -sS --max-time 10 https://ifconfig.me)}" - [ -n "$PERMITTED_IP" ] || { echo "bunq: could not determine this machine's IP" >&2; exit 1; } - echo "bunq: binding the key to $PERMITTED_IP (this machine only — NOT the server)" - printf 'bunq: this is permanent for this key. Continue? [y/N] ' - read -r _yn; [ "$_yn" = y ] || [ "$_yn" = Y ] || { echo "aborted"; exit 1; } - _resp=$(api POST /v1/device-server \ - "$(python3 -c 'import json,sys -print(json.dumps({"description":"catcrafts.net financials callback registrar", - "secret":sys.argv[1],"permitted_ips":[sys.argv[2]]}))' \ - "$BUNQ_KEY" "$PERMITTED_IP")" "$INSTALL_TOKEN") - die_on_error "$_resp" - state_set device_registered yes -fi - -# Sessions expire, so this one is not cached. -_resp=$(api POST /v1/session-server \ - "$(python3 -c 'import json,sys; print(json.dumps({"secret":sys.argv[1]}))' "$BUNQ_KEY")" \ - "$INSTALL_TOKEN") -die_on_error "$_resp" -SESSION=$(printf '%s' "$_resp" | json_get token) -# NOT a naive search for "id": the session response opens with an Id object of -# its own, and taking that one silently addresses every later call to the -# wrong user. The user is whichever of these three the account type produces. -USER_ID=$(printf '%s' "$_resp" | python3 -c 'import json,sys -d=json.load(sys.stdin) -for item in d.get("Response",[]): - for k in ("UserPerson","UserCompany","UserApiKey"): - u=item.get(k) - if isinstance(u,dict) and "id" in u: - print(u["id"]); sys.exit(0)') -[ -n "$SESSION" ] && [ -n "$USER_ID" ] \ - || { echo "bunq: could not open a session" >&2; exit 1; } - -# ── commands ────────────────────────────────────────────────────────── - -case "${1:-list}" in -list) - echo "bunq: user $USER_ID" - echo - echo "accounts (the donations one goes in donation_accounts in the rules file):" - api GET "/v1/user/$USER_ID/monetary-account?count=50" "" "$SESSION" | python3 -c 'import json,sys -d=json.load(sys.stdin) -for item in d.get("Response",[]): - for kind,acc in item.items(): - if not isinstance(acc,dict) or "id" not in acc: continue - iban=next((a.get("value") for a in acc.get("alias",[]) if a.get("type")=="IBAN"),"") - bal=(acc.get("balance") or {}).get("value","?") - aid=acc.get("id"); st=acc.get("status",""); desc=acc.get("description","") - print(f" id={aid:<10} {st:<8} {kind:<22} {desc} {iban} balance {bal}")' - echo - # Filters registered PER ACCOUNT do not appear in the user-level list, so - # showing only that one reads as "nothing is registered" when in fact - # everything is. Both are printed, per account, or this command lies. - echo "MUTATION filters, per account:" - api GET "/v1/user/$USER_ID/monetary-account?count=50" "" "$SESSION" | python3 -c 'import json,sys -d=json.load(sys.stdin) -ids=[] -for item in d.get("Response",[]): - for acc in item.values(): - if isinstance(acc,dict) and "id" in acc: ids.append(str(acc["id"])) -print(" ".join(ids))' > /tmp/.cc-accts.$$ -for _a in $(cat /tmp/.cc-accts.$$); do - _f=$(api GET "/v1/user/$USER_ID/monetary-account/$_a/notification-filter-url" "" "$SESSION" \ - | python3 -c 'import json,sys -d=json.load(sys.stdin) -out=[] -for item in d.get("Response",[]): - f=item.get("NotificationFilterUrl") or {} - if not f: continue - t=f.get("notification_target") or "" - # never print the callback secret: the last path segment is masked - parts=t.rsplit("/",1) - masked=parts[0]+"/"+(parts[1][:4]+"…" if len(parts)>1 and parts[1] else "") - out.append(f.get("category","?")+" -> "+masked) -print("; ".join(out) if out else "(none)")') - printf ' account %-10s %s\n' "$_a" "$_f" -done -rm -f /tmp/.cc-accts.$$ -echo -echo "user-level filters (apply to ALL accounts):" -api GET "/v1/user/$USER_ID/notification-filter-url" "" "$SESSION" | python3 -c 'import json,sys -d=json.load(sys.stdin) -r=d.get("Response") or [] -if not r: - print(" (none)"); raise SystemExit -for item in r: - f=item.get("NotificationFilterUrl") or {} - t=f.get("notification_target") or "" - # The callback URL ENDS IN A SHARED SECRET. Never print it whole: this - # output gets pasted into issues and terminals that keep scrollback. - parts=t.rsplit("/",1) - masked=parts[0]+"/"+(parts[1][:4]+"…" if len(parts)>1 and parts[1] else "") - accts=f.get("all_monetary_account_id") or [] - cat=f.get("category","?") - print(f" {cat} -> {masked} accounts={sorted(set(accts))}")' - ;; -set) - ACCOUNT="${2:?usage: tools/bunq-callback.sh set }" - URL="${3:?usage: tools/bunq-callback.sh set }" - case "$URL" in - https://*) ;; - *) echo "bunq: the callback URL must be https" >&2; exit 1 ;; - esac - # Refuse to point bunq at an endpoint that is not answering yet: a filter - # whose target keeps failing is a filter bunq may disable, and a mutation - # delivered into a 404 is simply lost until the weekly reconciliation. - echo "bunq: checking the endpoint is live before registering it" - _code=$(curl -s -o /dev/null -w '%{http_code}' --max-time 15 -X POST \ - -H 'content-type: application/json' -d '{}' "$URL" || echo 000) - if [ "$_code" != 200 ]; then - echo "bunq: $URL answered $_code, not 200." >&2 - echo " Deploy the financials build and set BUNQ_CALLBACK_SECRET first." >&2 - echo " (A wrong secret answers 404 by design — check the secret too.)" >&2 - exit 1 - fi - # "all" registers at USER level, covering every account including ones - # created later. Prefer it: bunq treats even per-account registrations as - # user-scoped entries anyway, and repeated per-account POSTs accumulate - # duplicate account ids rather than replacing cleanly. A user-level POST - # replaces the ENTIRE set, which is also how you clear stale URLs. - if [ "$ACCOUNT" = all ]; then - echo "bunq: installing the MUTATION filter for ALL accounts" - _resp=$(api POST "/v1/user/$USER_ID/notification-filter-url" \ - "$(python3 -c 'import json,sys -print(json.dumps({"notification_filters":[ - {"category":"MUTATION","notification_target":sys.argv[1]}]}))' "$URL")" "$SESSION") - die_on_error "$_resp" - echo " done (previous filters replaced)" - exit 0 - fi - echo "bunq: installing the MUTATION filter on account $ACCOUNT" - # POST REPLACES the whole filter set for this account, so this is also how - # you change or clear one. - _resp=$(api POST "/v1/user/$USER_ID/monetary-account/$ACCOUNT/notification-filter-url" \ - "$(python3 -c 'import json,sys -print(json.dumps({"notification_filters":[ - {"category":"MUTATION","notification_target":sys.argv[1]}]}))' "$URL")" "$SESSION") - die_on_error "$_resp" - printf '%s\n' "$_resp" | python3 -m json.tool - echo - echo "bunq: done. Send yourself €0.01 and watch:" - echo " ssh hetzner journalctl -u catcrafts-server -f" - ;; -payments) - # What a real mutation actually looks like, WITHOUT printing what a real - # mutation actually says. This exists to answer "which field tells me a - # payment came from bunq.me?" empirically rather than from memory — so it - # prints the SHAPE (key names, and the values of fields that classify - # rather than identify) and masks everything that names a human. - ACCOUNT="${2:?usage: tools/bunq-callback.sh payments [count]}" - api GET "/v1/user/$USER_ID/monetary-account/$ACCOUNT/payment?count=${3:-25}" \ - "" "$SESSION" | python3 -c 'import json,sys -SAFE={"id","created","type","sub_type","amount","payment_auto_allocate_instance", - "bunqme_fundraiser_result","request_reference_split_the_bill", - "payment_arrival_expected","merchant_reference","batch_id","scheduled_id"} -d=json.load(sys.stdin) -rows=[item["Payment"] for item in d.get("Response",[]) if "Payment" in item] -if not rows: - print(" (no payments on this account)"); raise SystemExit -allkeys=set() -for p in rows: allkeys.update(p.keys()) -print(f" {len(rows)} payment(s). Union of keys present:") -for k in sorted(allkeys): - mark=" <- SAFE to classify on" if k in SAFE else "" - print(f" {k}{mark}") -print() -print(" per payment (identifying fields masked):") -for p in rows: - amt=(p.get("amount") or {}) - val=amt.get("value","?"); cur=amt.get("currency","?") - bm=p.get("bunqme_fundraiser_result") - bmk="yes" if bm else "no" - extra="" - if isinstance(bm,dict): - extra=" bunqme_keys=" + ",".join(sorted(bm.keys())) - pid=p.get("id"); ptype=p.get("type"); psub=p.get("sub_type") - print(f" id={pid} type={ptype} sub_type={psub} " - f"amount={val} {cur} bunqme_fundraiser_result={bmk}{extra}")' - ;; -backfill) - # The categorisation worklist for a date window. READ ONLY — it publishes - # nothing and writes nothing to the server. Incoming and outgoing are - # separated because they ask different questions ("is this a donation?" vs - # "which expense category?"), and outgoing is grouped by counterparty - # because that is the unit a rule matches on. - SINCE="${2:?usage: tools/bunq-callback.sh backfill [YYYY-MM-DD]}" - UNTIL="${3:-9999-12-31}" - _tmp=$(mktemp -d) - api GET "/v1/user/$USER_ID/monetary-account?count=50" "" "$SESSION" | python3 -c 'import json,sys -d=json.load(sys.stdin) -for item in d.get("Response",[]): - for acc in item.values(): - if isinstance(acc,dict) and "id" in acc: - print(acc["id"], (acc.get("description") or "?").replace(" ","_"))' > "$_tmp/accts" - while read -r _id _nm; do - api GET "/v1/user/$USER_ID/monetary-account/$_id/payment?count=200" "" "$SESSION" \ - > "$_tmp/pay-$_id-$_nm.json" - done < "$_tmp/accts" - python3 - "$SINCE" "$UNTIL" "$_tmp" <<'PY' -import glob, json, os, sys -since, until, tmp = sys.argv[1], sys.argv[2], sys.argv[3] -rows=[] -for path in sorted(glob.glob(os.path.join(tmp,"pay-*.json"))): - label=os.path.basename(path)[4:-5] - try: d=json.load(open(path)) - except Exception: continue - for item in d.get("Response",[]): - p=item.get("Payment") - if not isinstance(p,dict): continue - created=(p.get("created") or "")[:10] - if not (since <= created <= until): continue - amt=(p.get("amount") or {}) - if amt.get("currency")!="EUR": continue - try: cents=int(round(float(amt.get("value","0"))*100)) - except Exception: continue - cp=(p.get("counterparty_alias") or {}) - rows.append({"acct":label,"date":created,"cents":cents, - "type":p.get("type"),"sub":p.get("sub_type"), - "name":cp.get("display_name") or "?", - "iban":cp.get("iban") or "", - "desc":(p.get("description") or "").strip()[:44]}) -if not rows: - print(" no EUR payments in", since, "..", until); raise SystemExit -print(f" window {since} .. {until} {len(rows)} payment(s)\n") -ins=[r for r in rows if r["cents"]>0] -outs=[r for r in rows if r["cents"]<0] - -print(f" INCOMING ({len(ins)}) — decide donation / sale payout / other:") -for r in sorted(ins,key=lambda r:r["date"]): - eur=r["cents"]/100 - print(f" {r['date']} {eur:>9.2f} {r['type']:<11}{r['sub']:<9} {r['name'][:22]:<22} {r['desc']}") -print(f" ---- incoming total: {sum(r['cents'] for r in ins)/100:.2f}\n") - -print(f" OUTGOING ({len(outs)}) — grouped by counterparty; each group is one rule:") -groups={} -for r in outs: - key=(r["name"],r["iban"]) - g=groups.setdefault(key,{"cents":0,"n":0,"descs":set()}) - g["cents"]+=r["cents"]; g["n"]+=1 - if r["desc"]: g["descs"].add(r["desc"]) -for (name,iban),g in sorted(groups.items(),key=lambda kv:kv[1]["cents"]): - eur=-g["cents"]/100 - sample=sorted(g["descs"])[0] if g["descs"] else "" - print(f" {eur:>10.2f} x{g['n']:<3} {name[:26]:<26} {iban:<20} {sample[:30]}") -print(f" ---- outgoing total: {-sum(r['cents'] for r in outs)/100:.2f}") -PY - rm -rf "$_tmp" - ;; -payment) - # One payment by id, classification fields only. Used to answer "what does - # a payment that AROSE FROM a bunq.me tab actually look like on the wire?" - ACCOUNT="${2:?usage: tools/bunq-callback.sh payment ...}" - shift 2 - for pid in "$@"; do - api GET "/v1/user/$USER_ID/monetary-account/$ACCOUNT/payment/$pid" "" "$SESSION" \ - | python3 -c 'import json,sys -d=json.load(sys.stdin) -rows=[i["Payment"] for i in d.get("Response",[]) if "Payment" in i] -for p in rows: - amt=(p.get("amount") or {}) - val=amt.get("value","?") - pid=p.get("id"); ptype=p.get("type"); psub=p.get("sub_type") - has_bm="bunqme_fundraiser_result" in p - print(f" id={pid} type={ptype} sub_type={psub} amount={val} " - f"has_bunqme_field={has_bm}")' - done - ;; -bunqme) - # The bunq.me side. A tab records which payments fulfilled it, so this is - # the only authoritative "did this money come from a bunq.me link?" join — - # and note it takes an API CALL WITH THE KEY, which is exactly what the - # server does not have. Prints payment ids so they can be matched against - # `payments` output; no payer names or links are shown in full. - ACCOUNT="${2:?usage: tools/bunq-callback.sh bunqme [count]}" - api GET "/v1/user/$USER_ID/monetary-account/$ACCOUNT/bunqme-tab?count=${3:-25}" \ - "" "$SESSION" | python3 -c 'import json,sys -d=json.load(sys.stdin) -tabs=[i["BunqMeTab"] for i in d.get("Response",[]) if "BunqMeTab" in i] -if not tabs: - print(" (no bunq.me tabs on this account)"); raise SystemExit -for t in tabs: - url=t.get("bunqme_tab_share_url") or "" - slug=url.rsplit("/",1)[-1] - masked=slug[:3]+"…" if slug else "(none)" - entry=t.get("bunqme_tab_entry") or {} - inq=t.get("result_inquiries") or [] - pids=[] - for r in inq: - node=r.get("BunqMeTabResultInquiry") or r - pay=(node.get("payment") or {}) - pay=pay.get("Payment") or pay - if isinstance(pay,dict) and pay.get("id") is not None: - pids.append(str(pay["id"])) - tid=t.get("id"); st=t.get("status") - amt=(entry.get("amount_inquired") or {}).get("value","open") - joined=",".join(pids) - print(f" tab={tid} status={st} link=bunq.me/{masked} asked={amt} " - f"fulfilled_by_payment_ids=[{joined}]")' - ;; -*) - echo "usage: tools/bunq-callback.sh [list | set |" >&2 - echo " payments [count] | bunqme [count]]" >&2 - exit 2 - ;; -esac