This commit is contained in:
parent
94a995d814
commit
aaa7a8ce99
10 changed files with 543 additions and 8 deletions
|
|
@ -351,6 +351,38 @@ BUNQ_API_KEY=… TRANSFER_IBAN=NL.. \
|
||||||
# -> pulled 2 new credit(s) into /var/lib/catcrafts/orders.jsonl.transfer-credits.jsonl
|
# -> pulled 2 new credit(s) into /var/lib/catcrafts/orders.jsonl.transfer-credits.jsonl
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**The key is IP-restricted, and that decides the shape.** Verified the hard way
|
||||||
|
on 2026-08-20: the bunq API key permits one address (the owner's home
|
||||||
|
connection), and **bunq enforces that on every request, not only at device
|
||||||
|
registration**. Registering the server's address in the device's
|
||||||
|
`permitted_ips` does not help — the device registered fine from home with both
|
||||||
|
addresses listed, and every read from the server still came back
|
||||||
|
`"Incorrect API key or IP address"`. So `BUNQ_API_KEY` on the web host cannot
|
||||||
|
work unless the key's allowlist is widened in the bunq app.
|
||||||
|
|
||||||
|
That constraint happens to enforce the right design, so it is now the supported
|
||||||
|
one: **`tools/pull-and-ship-credits.sh`**, run by a systemd USER timer on the
|
||||||
|
machine bunq permits (`deploy/catcrafts-credits.{service,timer}`, install
|
||||||
|
instructions in the service file). It pulls, keeps an accumulating local copy,
|
||||||
|
and ships **incoming credits only** to the file the rail reads. Outgoing lines
|
||||||
|
are supplier payments and card spending: the matcher ignores negative amounts,
|
||||||
|
so shipping them would put the business's outgoing payment history on a
|
||||||
|
public-facing host for no settlement benefit. The script refuses to overwrite
|
||||||
|
the remote with an empty file, so an upstream parse failure leaves the server
|
||||||
|
settling from the last good copy rather than from nothing.
|
||||||
|
|
||||||
|
Latency, so nobody wonders: five minutes for the timer plus the reconciler's own
|
||||||
|
60 s cadence, so a fresh order confirms within about six minutes of the money
|
||||||
|
landing. Orders more than two hours old back off to a ten-minute poll, so an
|
||||||
|
older one can take that long.
|
||||||
|
|
||||||
|
Two operational notes. The timer only runs while that machine is awake, so
|
||||||
|
`Persistent=true` makes it pull once on waking rather than silently skipping
|
||||||
|
every window it missed. And if the home address changes, both the key's
|
||||||
|
allowlist (in the bunq app) and settlement stop working until it is updated —
|
||||||
|
the failure mode is silent from the shop's side, so the credits file's mtime is
|
||||||
|
worth glancing at.
|
||||||
|
|
||||||
**Read this before deciding where to put the key.** A bunq API key can
|
**Read this before deciding where to put the key.** A bunq API key can
|
||||||
**initiate payments**, and bunq offers no read-only scope, so there is no such
|
**initiate payments**, and bunq offers no read-only scope, so there is no such
|
||||||
thing as a key that can only read. That gives two deployment shapes, and they
|
thing as a key that can only read. That gives two deployment shapes, and they
|
||||||
|
|
|
||||||
30
deploy/catcrafts-credits.service
Normal file
30
deploy/catcrafts-credits.service
Normal file
|
|
@ -0,0 +1,30 @@
|
||||||
|
# Read the bank and hand the incoming credits to the server.
|
||||||
|
#
|
||||||
|
# A USER unit, on the machine whose IP the bunq API key permits — NOT on the
|
||||||
|
# web host. The web host cannot call bunq at all (the key is IP-restricted at
|
||||||
|
# the key level and bunq enforces it on every request), and it should not be
|
||||||
|
# able to: a bunq key can initiate payments, because bunq offers no read-only
|
||||||
|
# scope. So the key stays here and the server only ever reads a file.
|
||||||
|
#
|
||||||
|
# Install:
|
||||||
|
# mkdir -p ~/.config/systemd/user
|
||||||
|
# cp deploy/catcrafts-credits.{service,timer} ~/.config/systemd/user/
|
||||||
|
# systemctl --user daemon-reload
|
||||||
|
# systemctl --user enable --now catcrafts-credits.timer
|
||||||
|
# loginctl enable-linger "$USER" # so it runs when nobody is logged in
|
||||||
|
#
|
||||||
|
# Watch: journalctl --user -u catcrafts-credits -f
|
||||||
|
[Unit]
|
||||||
|
Description=Pull bunq credits and ship them to catcrafts.net
|
||||||
|
Documentation=file:deploy/README.md
|
||||||
|
# Pointless without a route to the bank or to the server.
|
||||||
|
After=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
WorkingDirectory=%h/repos/catcrafts.net
|
||||||
|
ExecStart=%h/repos/catcrafts.net/tools/pull-and-ship-credits.sh
|
||||||
|
# A failed pull must not look like a successful one. The script already refuses
|
||||||
|
# to overwrite the remote with an empty file, so a failure here leaves the
|
||||||
|
# server settling from the last good copy rather than from nothing.
|
||||||
|
SuccessExitStatus=0
|
||||||
23
deploy/catcrafts-credits.timer
Normal file
23
deploy/catcrafts-credits.timer
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
# Every five minutes. Two things set the floor and the ceiling:
|
||||||
|
#
|
||||||
|
# * bunq rate-limits reads to roughly 3 GET per 3 seconds per method, so one
|
||||||
|
# call per five minutes is nowhere near it. The cost of going faster is
|
||||||
|
# nothing technical; it is just noise.
|
||||||
|
# * a donor watching the order page wants confirmation while they still care.
|
||||||
|
# Five minutes here plus the reconciler's own 60 s cadence means a fresh
|
||||||
|
# order confirms within about six minutes of the money landing.
|
||||||
|
#
|
||||||
|
# Persistent=true so a laptop that was asleep pulls once on waking instead of
|
||||||
|
# silently skipping every window it missed — which is exactly when a donation
|
||||||
|
# would otherwise sit unacknowledged overnight.
|
||||||
|
[Unit]
|
||||||
|
Description=Pull bunq credits for catcrafts.net every 5 minutes
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnBootSec=2min
|
||||||
|
OnUnitActiveSec=5min
|
||||||
|
Persistent=true
|
||||||
|
AccuracySec=30s
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
|
@ -192,6 +192,37 @@ std::vector<BankCredit> ParseBunqPayments(std::string_view json) {
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
std::optional<BankCredit> ParseBunqCallback(std::string_view json) {
|
||||||
|
const auto doc = Json::Parse(json);
|
||||||
|
if (!doc || !doc->IsObject()) return std::nullopt;
|
||||||
|
const Json::Value* note = doc->Find("NotificationUrl");
|
||||||
|
if (!note || !note->IsObject()) return std::nullopt;
|
||||||
|
const Json::Value* object = note->Find("object");
|
||||||
|
if (!object || !object->IsObject()) return std::nullopt;
|
||||||
|
const Json::Value* p = object->Find("Payment");
|
||||||
|
if (!p || !p->IsObject()) return std::nullopt;
|
||||||
|
|
||||||
|
const Json::Value* amount = p->Find("amount");
|
||||||
|
if (!amount || !amount->IsObject()) return std::nullopt;
|
||||||
|
// Only euro can pay a euro order; see ParseBunqPayments for why a
|
||||||
|
// foreign-currency credit is skipped rather than counted at face value.
|
||||||
|
if (amount->Str("currency") != "EUR") return std::nullopt;
|
||||||
|
const std::optional<std::int64_t> minor =
|
||||||
|
ParseSignedAmountToMinor(amount->Str("value"));
|
||||||
|
if (!minor) return std::nullopt;
|
||||||
|
|
||||||
|
BankCredit c;
|
||||||
|
c.id = std::format("{}", p->Int("id"));
|
||||||
|
c.reference = std::string(p->Str("description"));
|
||||||
|
c.amountMinor = *minor;
|
||||||
|
c.method = std::string(BunqMethodFor(p->Str("type")));
|
||||||
|
// An id is what deduplication rests on. A callback without one cannot be
|
||||||
|
// deduplicated, so accepting it would let a retry credit the same money
|
||||||
|
// twice — refuse instead.
|
||||||
|
if (c.id.empty() || c.id == "0") return std::nullopt;
|
||||||
|
return c;
|
||||||
|
}
|
||||||
|
|
||||||
namespace {
|
namespace {
|
||||||
|
|
||||||
class BunqCreditSource final : public CreditSource {
|
class BunqCreditSource final : public CreditSource {
|
||||||
|
|
|
||||||
|
|
@ -75,6 +75,13 @@ std::string gCssHref = "/styles.css";
|
||||||
PaymentRails gRails;
|
PaymentRails gRails;
|
||||||
std::string gRedirectBase = "https://catcrafts.net";
|
std::string gRedirectBase = "https://catcrafts.net";
|
||||||
|
|
||||||
|
// Where the bunq callback writes, and the secret path that authorises it.
|
||||||
|
// Both empty means the endpoint does not exist at all — an unconfigured
|
||||||
|
// webhook must not answer, or the shop would carry a public write-ish endpoint
|
||||||
|
// nobody asked for.
|
||||||
|
std::filesystem::path gCreditsPath;
|
||||||
|
std::string gWebhookPath;
|
||||||
|
|
||||||
// The bank-derived aggregates for /financials live in Catcrafts.Server-
|
// The bank-derived aggregates for /financials live in Catcrafts.Server-
|
||||||
// Financials.cpp, which owns their file. They are read through
|
// Financials.cpp, which owns their file. They are read through
|
||||||
// CurrentFinancials() per request rather than cached: unlike the content
|
// CurrentFinancials() per request rather than cached: unlike the content
|
||||||
|
|
@ -637,6 +644,65 @@ std::string NowRfc2822() {
|
||||||
// the POST/redirect/GET pattern, and it matters for a real form: a rendered
|
// the POST/redirect/GET pattern, and it matters for a real form: a rendered
|
||||||
// POST response means reloading re-submits, and the back button re-posts. The
|
// POST response means reloading re-submits, and the back button re-posts. The
|
||||||
// redirect leaves the browser on a GET it can safely repeat.
|
// redirect leaves the browser on a GET it can safely repeat.
|
||||||
|
// The bunq callback. See ParseBunqCallback for the security model; the short
|
||||||
|
// version is that bunq does NOT sign these, so this endpoint's only defences
|
||||||
|
// are the ones around it: a secret path segment, a source-IP allowlist in
|
||||||
|
// Caddy for bunq's published range, and the fact that no parcel leaves without
|
||||||
|
// a human. It therefore does the least it possibly can — decode one payment
|
||||||
|
// and append it to the credits file — and makes no settlement decision at all.
|
||||||
|
// The reconciler settles from that file exactly as it does from a pulled one,
|
||||||
|
// so a forged callback can at worst manufacture a credit line, never bypass
|
||||||
|
// the reference match or the covering-amount rule.
|
||||||
|
HTTPResponse HandleBunqCallback(const HTTPRequest& req) {
|
||||||
|
HTTPResponse res;
|
||||||
|
res.headers["content-type"] = "text/plain; charset=utf-8";
|
||||||
|
// Never cache, never index, and say nothing useful in the body: this URL
|
||||||
|
// is a shared secret, so every answer is the same two characters.
|
||||||
|
res.headers["cache-control"] = "no-store";
|
||||||
|
res.headers["x-robots-tag"] = "noindex, nofollow";
|
||||||
|
|
||||||
|
if (req.method != "POST") {
|
||||||
|
res.status = "405";
|
||||||
|
res.headers["allow"] = "POST";
|
||||||
|
res.body = "no\n";
|
||||||
|
return res;
|
||||||
|
}
|
||||||
|
|
||||||
|
const std::optional<BankCredit> credit = ParseBunqCallback(req.body);
|
||||||
|
if (!credit) {
|
||||||
|
// A callback shape this cannot decode is NOT an error to shout about
|
||||||
|
// with a 4xx: bunq sends several notification categories, and only
|
||||||
|
// some carry a Payment. Answer 200 so bunq stops retrying something
|
||||||
|
// that will never decode, and log it so a genuinely new shape is
|
||||||
|
// visible rather than silently dropped.
|
||||||
|
std::println(std::cerr, "bunq callback: no usable payment in body ({} bytes)",
|
||||||
|
req.body.size());
|
||||||
|
res.status = "200";
|
||||||
|
res.body = "ok\n";
|
||||||
|
return res;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Outgoing money cannot pay for an order, and the matcher ignores it
|
||||||
|
// anyway — so refuse to write the shop's own supplier payments into a file
|
||||||
|
// that lives on a public-facing host.
|
||||||
|
if (credit->amountMinor <= 0) {
|
||||||
|
res.status = "200";
|
||||||
|
res.body = "ok\n";
|
||||||
|
return res;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (AppendCreditTo(gCreditsPath, *credit)) {
|
||||||
|
std::println(std::cerr, "bunq callback: credited {} via {} (id {})",
|
||||||
|
Money::FormatMinor(credit->amountMinor), credit->method,
|
||||||
|
credit->id);
|
||||||
|
}
|
||||||
|
// 200 even on a duplicate or a write we skipped: a duplicate IS success
|
||||||
|
// from bunq's side, and making it retry would achieve nothing.
|
||||||
|
res.status = "200";
|
||||||
|
res.body = "ok\n";
|
||||||
|
return res;
|
||||||
|
}
|
||||||
|
|
||||||
HTTPResponse HandleCheckout(const HTTPRequest& req, const Route& route) {
|
HTTPResponse HandleCheckout(const HTTPRequest& req, const Route& route) {
|
||||||
HTTPResponse res;
|
HTTPResponse res;
|
||||||
|
|
||||||
|
|
@ -1064,6 +1130,29 @@ void ConfigurePayments(PaymentRails rails, std::string redirectBase) {
|
||||||
if (!redirectBase.empty()) gRedirectBase = std::move(redirectBase);
|
if (!redirectBase.empty()) gRedirectBase = std::move(redirectBase);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void ConfigureBunqCallback(std::filesystem::path creditsPath, std::string secretPath) {
|
||||||
|
// Both or neither. A secret with nowhere to write, or a path to write with
|
||||||
|
// no secret guarding it, are each worse than not having the endpoint: the
|
||||||
|
// first answers requests it cannot act on, the second answers everyone.
|
||||||
|
if (creditsPath.empty() || secretPath.empty()) return;
|
||||||
|
// A short secret is not a secret. Refuse rather than serve a guessable
|
||||||
|
// write endpoint just because someone set the variable to "hook".
|
||||||
|
if (secretPath.size() < 24) {
|
||||||
|
std::println(std::cerr,
|
||||||
|
"bunq callback: BUNQ_WEBHOOK_PATH is too short to be secret "
|
||||||
|
"({} chars, want 24+) — the endpoint is NOT enabled",
|
||||||
|
secretPath.size());
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (secretPath.front() != '/') secretPath.insert(secretPath.begin(), '/');
|
||||||
|
gCreditsPath = std::move(creditsPath);
|
||||||
|
gWebhookPath = std::move(secretPath);
|
||||||
|
// Deliberately does NOT log the path: it would land in the journal, and
|
||||||
|
// from there in any log shipping or analytics that reads it.
|
||||||
|
std::println(std::cerr, "bunq callback: enabled on a secret path ({} chars)",
|
||||||
|
gWebhookPath.size());
|
||||||
|
}
|
||||||
|
|
||||||
bool CryptoPaymentAvailable() { return gRails.crypto != nullptr; }
|
bool CryptoPaymentAvailable() { return gRails.crypto != nullptr; }
|
||||||
|
|
||||||
bool BankPaymentAvailable() { return gRails.bank != nullptr; }
|
bool BankPaymentAvailable() { return gRails.bank != nullptr; }
|
||||||
|
|
@ -1280,6 +1369,31 @@ int Serve(std::uint16_t port) {
|
||||||
};
|
};
|
||||||
|
|
||||||
auto fallback = [](const HTTPRequest& req) -> HTTPResponse {
|
auto fallback = [](const HTTPRequest& req) -> HTTPResponse {
|
||||||
|
// The bunq callback, before anything else looks at the path. It is
|
||||||
|
// matched here rather than added to the shared route table on purpose:
|
||||||
|
// ParseRoute is shared with the wasm frontend, and a secret URL has no
|
||||||
|
// business being compiled into a bundle served to browsers.
|
||||||
|
//
|
||||||
|
// Compared in CONSTANT TIME. This is a shared secret in a URL, so a
|
||||||
|
// timing oracle on a byte-by-byte compare would let it be recovered
|
||||||
|
// one character at a time — and unlike a password there is no rate
|
||||||
|
// limit or lockout behind it.
|
||||||
|
// POST only, and non-POST deliberately falls through to ordinary page
|
||||||
|
// handling rather than answering 405. A distinctive answer here would
|
||||||
|
// be an oracle: a GET returning 405 where every other unknown URL
|
||||||
|
// returns 404 confirms a guessed path is the right one, which is
|
||||||
|
// exactly the signal a secret-in-the-URL scheme cannot afford to give.
|
||||||
|
// bunq only ever POSTs, so nothing legitimate is lost.
|
||||||
|
if (!gWebhookPath.empty() && req.method == "POST") {
|
||||||
|
const std::string_view path = PathWithoutQueryHTTP(req.path);
|
||||||
|
if (path.size() == gWebhookPath.size()) {
|
||||||
|
unsigned char diff = 0;
|
||||||
|
for (std::size_t i = 0; i < path.size(); ++i) {
|
||||||
|
diff |= static_cast<unsigned char>(path[i] ^ gWebhookPath[i]);
|
||||||
|
}
|
||||||
|
if (diff == 0) return HandleBunqCallback(req);
|
||||||
|
}
|
||||||
|
}
|
||||||
// A POST to a product page is a checkout submission.
|
// A POST to a product page is a checkout submission.
|
||||||
if (req.method == "POST") {
|
if (req.method == "POST") {
|
||||||
const Route route = ParseRoute(PathWithoutQueryHTTP(req.path));
|
const Route route = ParseRoute(PathWithoutQueryHTTP(req.path));
|
||||||
|
|
|
||||||
|
|
@ -369,6 +369,37 @@ std::unique_ptr<CreditSource> MakeFileCreditSource(std::filesystem::path path) {
|
||||||
return std::make_unique<FileCreditSource>(std::move(path));
|
return std::make_unique<FileCreditSource>(std::move(path));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
bool AppendCreditTo(const std::filesystem::path& creditsPath,
|
||||||
|
const BankCredit& credit) {
|
||||||
|
if (credit.id.empty()) return false;
|
||||||
|
// Deduplicate on the bank's own payment id. bunq retries a callback about
|
||||||
|
// six times, and a periodic --pull-credits reads an overlapping window, so
|
||||||
|
// the SAME payment arrives more than once by design. Appending it twice
|
||||||
|
// would double the money against one reference — enough, on a part-paid
|
||||||
|
// order, to settle it without the balance ever arriving.
|
||||||
|
{
|
||||||
|
std::ifstream in(creditsPath, std::ios::binary);
|
||||||
|
std::string line;
|
||||||
|
while (std::getline(in, line)) {
|
||||||
|
if (line.empty()) continue;
|
||||||
|
const auto doc = Json::Parse(line);
|
||||||
|
if (!doc || !doc->IsObject()) continue;
|
||||||
|
if (doc->Str("id") == credit.id) return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
std::ofstream out(creditsPath, std::ios::app | std::ios::binary);
|
||||||
|
if (!out) {
|
||||||
|
std::println(std::cerr, "transfer: cannot append to {}", creditsPath.string());
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
out << std::format(
|
||||||
|
R"({{"id":"{}","reference":"{}","amount_minor":{},"method":"{}"}})" "\n",
|
||||||
|
EscT(credit.id), EscT(credit.reference), credit.amountMinor,
|
||||||
|
EscT(credit.method));
|
||||||
|
out.flush();
|
||||||
|
return static_cast<bool>(out);
|
||||||
|
}
|
||||||
|
|
||||||
std::optional<int> PullCreditsInto(CreditSource& source,
|
std::optional<int> PullCreditsInto(CreditSource& source,
|
||||||
const std::filesystem::path& creditsPath) {
|
const std::filesystem::path& creditsPath) {
|
||||||
const std::optional<std::vector<BankCredit>> fresh = source.Recent();
|
const std::optional<std::vector<BankCredit>> fresh = source.Recent();
|
||||||
|
|
|
||||||
|
|
@ -406,12 +406,28 @@ int main(int argc, char** argv) {
|
||||||
// warning is loud, the listening line below reports
|
// warning is loud, the listening line below reports
|
||||||
// crypto=off, and tools/enable-eurc.sh refuses to call an
|
// crypto=off, and tools/enable-eurc.sh refuses to call an
|
||||||
// enable successful without the rail's own load line.
|
// enable successful without the rail's own load line.
|
||||||
std::println(std::cerr,
|
// Name the right files. This message used to describe EURC's
|
||||||
"catcrafts-server: WARNING: the '{}' rail could not load "
|
// chains file and address pool whichever rail had failed,
|
||||||
"its chains file ({}) or address pool ({}) — see above. "
|
// which is actively misleading for the transfer rail: the
|
||||||
"CONTINUING WITHOUT IT: that payment choice is off and "
|
// likeliest way IT fails is a bad BUNQ_API_KEY, and being
|
||||||
"the rest of the site is unaffected.",
|
// told to look at a chains file sends the reader away from
|
||||||
mode, eurcChainsPath.string(), eurcPoolPath.string());
|
// the actual cause.
|
||||||
|
if (mode == "transfer") {
|
||||||
|
std::println(std::cerr,
|
||||||
|
"catcrafts-server: WARNING: the 'transfer' rail could "
|
||||||
|
"not load — see above. Usual causes: TRANSFER_IBAN or "
|
||||||
|
"TRANSFER_BENEFICIARY unset, or a BUNQ_API_KEY that "
|
||||||
|
"bunq refused. CONTINUING WITHOUT IT: bank transfer is "
|
||||||
|
"off, so checkout offers only the other rail, and the "
|
||||||
|
"rest of the site is unaffected.");
|
||||||
|
} else {
|
||||||
|
std::println(std::cerr,
|
||||||
|
"catcrafts-server: WARNING: the '{}' rail could not load "
|
||||||
|
"its chains file ({}) or address pool ({}) — see above. "
|
||||||
|
"CONTINUING WITHOUT IT: that payment choice is off and "
|
||||||
|
"the rest of the site is unaffected.",
|
||||||
|
mode, eurcChainsPath.string(), eurcPoolPath.string());
|
||||||
|
}
|
||||||
out.reset();
|
out.reset();
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
@ -425,6 +441,12 @@ int main(int argc, char** argv) {
|
||||||
if (!build(cryptoMode, rails.crypto)) return 2;
|
if (!build(cryptoMode, rails.crypto)) return 2;
|
||||||
|
|
||||||
Server::ConfigurePayments(std::move(rails), redirectBase);
|
Server::ConfigurePayments(std::move(rails), redirectBase);
|
||||||
|
// The bunq callback, which exists because the API key's IP allowlist
|
||||||
|
// forbids this host from asking bunq anything. Off unless a secret path
|
||||||
|
// is configured.
|
||||||
|
if (const char* v = std::getenv("BUNQ_WEBHOOK_PATH"); v && *v) {
|
||||||
|
Server::ConfigureBunqCallback(transferCreditsPath, v);
|
||||||
|
}
|
||||||
|
|
||||||
// Invoice signing: the GPG key uid/fingerprint; GNUPGHOME decides the
|
// Invoice signing: the GPG key uid/fingerprint; GNUPGHOME decides the
|
||||||
// keyring. Unset means unsigned dev invoices with a visible marker.
|
// keyring. Unset means unsigned dev invoices with a visible marker.
|
||||||
|
|
@ -496,7 +518,14 @@ int main(int argc, char** argv) {
|
||||||
if (const char* v = std::getenv("BUNQ_STATE"); v && *v) {
|
if (const char* v = std::getenv("BUNQ_STATE"); v && *v) {
|
||||||
statePath = v;
|
statePath = v;
|
||||||
} else {
|
} else {
|
||||||
statePath = ordersPath;
|
// Hung off the CREDITS file, not the orders file, because that
|
||||||
|
// is what MakeRail does — and the two MUST agree. They did not
|
||||||
|
// at first, and the cost is not cosmetic: a machine that ran
|
||||||
|
// both this command and the server would onboard twice against
|
||||||
|
// bunq, and bunq allows as few as TEN setup calls per DAY. Two
|
||||||
|
// conventions for one file is a way to spend that budget on
|
||||||
|
// nothing.
|
||||||
|
statePath = creditsPath;
|
||||||
statePath += ".bunq-context.json";
|
statePath += ".bunq-context.json";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -562,7 +591,18 @@ int main(int argc, char** argv) {
|
||||||
std::println(std::cerr, "could not append to {}", file.string());
|
std::println(std::cerr, "could not append to {}", file.string());
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
if (status == "paid") Server::AssignInvoiceNumber(token, now);
|
// Never for a donation, exactly as the automatic paid transition
|
||||||
|
// refuses: nothing was supplied, so there is no invoice, and a
|
||||||
|
// number burned on one leaves a gap-shaped question in a
|
||||||
|
// customer's series. This manual path was missing the guard and
|
||||||
|
// minted a number for the donation CC-16083E on 2026-08-20 while
|
||||||
|
// settling a stuck test payment by hand. That number is spent and
|
||||||
|
// an append-only ledger cannot recall it, which is exactly why
|
||||||
|
// the check belongs on every path that can transition to paid,
|
||||||
|
// not only the one the reconciler takes.
|
||||||
|
if (status == "paid" && !order->donation) {
|
||||||
|
Server::AssignInvoiceNumber(token, now);
|
||||||
|
}
|
||||||
std::println("{}: {} -> {}", order->reference, order->status, status);
|
std::println("{}: {} -> {}", order->reference, order->status, status);
|
||||||
return 0;
|
return 0;
|
||||||
};
|
};
|
||||||
|
|
|
||||||
|
|
@ -602,6 +602,43 @@ export namespace Catcrafts::Server {
|
||||||
// "sepa" is final, "card" can be reversed for months.
|
// "sepa" is final, "card" can be reversed for months.
|
||||||
std::string_view BunqMethodFor(std::string_view paymentType);
|
std::string_view BunqMethodFor(std::string_view paymentType);
|
||||||
|
|
||||||
|
// ── the bunq callback (webhook) ───────────────────────────────────
|
||||||
|
//
|
||||||
|
// bunq pushes a notification when the account changes, which is the only
|
||||||
|
// way this shop learns about money when the API key's IP allowlist forbids
|
||||||
|
// the web host from asking. Shape:
|
||||||
|
//
|
||||||
|
// {"NotificationUrl":{"category":"MUTATION",
|
||||||
|
// "object":{"Payment":{…the same Payment object…}}}}
|
||||||
|
//
|
||||||
|
// SECURITY, stated plainly because the design depends on understanding it:
|
||||||
|
// **bunq does not sign these.** Verified against doc.bunq.com — no HMAC, no
|
||||||
|
// server signature; certificate pinning authenticates US to bunq, not bunq
|
||||||
|
// to us. So the body is an unauthenticated claim that money arrived, and
|
||||||
|
// the only things standing between it and the ledger are transport-level:
|
||||||
|
// a source-IP allowlist for bunq's published range (185.40.108.0/22, which
|
||||||
|
// bunq warns may change), a secret path segment, and the fact that nothing
|
||||||
|
// ships without a human. Treat a callback as evidence exactly as strong as
|
||||||
|
// those controls, and keep a periodic --pull-credits as the backstop:
|
||||||
|
// bunq retries roughly six times and then drops the notification forever,
|
||||||
|
// so a backend that was down during a deploy loses that payment silently.
|
||||||
|
std::optional<BankCredit> ParseBunqCallback(std::string_view json);
|
||||||
|
|
||||||
|
// Append one credit to the file the transfer rail reads, unless an entry
|
||||||
|
// with the same id is already there. Returns false on a write failure or a
|
||||||
|
// duplicate — the caller answers the webhook 200 either way, because a
|
||||||
|
// duplicate is a SUCCESS from bunq's point of view and retrying it would
|
||||||
|
// achieve nothing.
|
||||||
|
bool AppendCreditTo(const std::filesystem::path& creditsPath,
|
||||||
|
const BankCredit& credit);
|
||||||
|
|
||||||
|
// Enable the bunq callback endpoint. Both arguments are required and the
|
||||||
|
// secret must be at least 24 characters, or the endpoint stays off — see
|
||||||
|
// ParseBunqCallback for why the path IS the authentication here, and why
|
||||||
|
// that is only acceptable alongside Caddy's source-IP allowlist.
|
||||||
|
void ConfigureBunqCallback(std::filesystem::path creditsPath,
|
||||||
|
std::string secretPath);
|
||||||
|
|
||||||
// One pull: read the account and append every credit not already in the
|
// One pull: read the account and append every credit not already in the
|
||||||
// file to it, newest last. Returns the number appended, or nullopt if the
|
// file to it, newest last. Returns the number appended, or nullopt if the
|
||||||
// bank could not be reached. This is what `--pull-credits` runs, and it is
|
// bank could not be reached. This is what `--pull-credits` runs, and it is
|
||||||
|
|
|
||||||
|
|
@ -148,6 +148,109 @@ int main() {
|
||||||
R"("amount":{"currency":"EUR","value":"1.234"}}}]})").empty(),
|
R"("amount":{"currency":"EUR","value":"1.234"}}}]})").empty(),
|
||||||
"an unparseable amount is skipped rather than read as zero");
|
"an unparseable amount is skipped rather than read as zero");
|
||||||
|
|
||||||
|
// ── the callback shape ────────────────────────────────────────────
|
||||||
|
//
|
||||||
|
// bunq does not sign these, so the parser is the only thing between an
|
||||||
|
// arbitrary POST body and a line in the credits file. It must therefore be
|
||||||
|
// strict about what it accepts and must never invent a field.
|
||||||
|
{
|
||||||
|
constexpr std::string_view kHook = R"({
|
||||||
|
"NotificationUrl": {
|
||||||
|
"category": "MUTATION",
|
||||||
|
"object": {"Payment": {
|
||||||
|
"id": 4155999,
|
||||||
|
"type": "EBA_SCT",
|
||||||
|
"description": "CC-2B6457",
|
||||||
|
"amount": {"currency": "EUR", "value": "570.43"}
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
})";
|
||||||
|
const auto c = Server::ParseBunqCallback(kHook);
|
||||||
|
Check(c.has_value(), "a MUTATION callback decodes to a credit");
|
||||||
|
if (c) {
|
||||||
|
Check(c->id == "4155999", "the payment id is kept", c->id);
|
||||||
|
Check(c->amountMinor == 57043, "the amount decodes",
|
||||||
|
std::format("{}", c->amountMinor));
|
||||||
|
Check(c->reference == "CC-2B6457", "the description is the reference",
|
||||||
|
c->reference);
|
||||||
|
Check(c->method == "sepa", "the type maps to a via", c->method);
|
||||||
|
}
|
||||||
|
|
||||||
|
// An id is what deduplication rests on, and bunq RETRIES callbacks. A
|
||||||
|
// credit with no id could therefore be applied twice, so it must be
|
||||||
|
// refused outright rather than stored with a blank.
|
||||||
|
Check(!Server::ParseBunqCallback(R"({"NotificationUrl":{"object":{"Payment":{)"
|
||||||
|
R"("type":"EBA_SCT","description":"x",)"
|
||||||
|
R"("amount":{"currency":"EUR","value":"1.00"}}}}})").has_value(),
|
||||||
|
"a payment with no id is refused, because retries need one");
|
||||||
|
|
||||||
|
// Everything that is not a payment notification: bunq sends several
|
||||||
|
// categories, and none of the others may produce a credit.
|
||||||
|
for (const std::string_view junk : {
|
||||||
|
std::string_view(""),
|
||||||
|
std::string_view("not json"),
|
||||||
|
std::string_view(R"({})"),
|
||||||
|
std::string_view(R"({"NotificationUrl":{}})"),
|
||||||
|
std::string_view(R"({"NotificationUrl":{"object":{}}})"),
|
||||||
|
std::string_view(R"({"NotificationUrl":{"object":{"Payment":{}}}})"),
|
||||||
|
// A Payment whose amount is another currency, or unparseable.
|
||||||
|
std::string_view(R"({"NotificationUrl":{"object":{"Payment":{"id":1,)"
|
||||||
|
R"("amount":{"currency":"USD","value":"1.00"}}}}})"),
|
||||||
|
std::string_view(R"({"NotificationUrl":{"object":{"Payment":{"id":1,)"
|
||||||
|
R"("amount":{"currency":"EUR","value":"1.234"}}}}})"),
|
||||||
|
}) {
|
||||||
|
Check(!Server::ParseBunqCallback(junk).has_value(),
|
||||||
|
"a body with no usable euro payment yields nothing", junk);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The sign survives here too: an outgoing payment must not be able to
|
||||||
|
// arrive through the webhook as income.
|
||||||
|
const auto out = Server::ParseBunqCallback(
|
||||||
|
R"({"NotificationUrl":{"object":{"Payment":{"id":7,"type":"EBA_SCT",)"
|
||||||
|
R"("description":"supplier","amount":{"currency":"EUR","value":"-99.00"}}}}})");
|
||||||
|
Check(out.has_value() && out->amountMinor == -9900,
|
||||||
|
"an outgoing callback keeps its sign for the caller to reject",
|
||||||
|
out ? std::format("{}", out->amountMinor) : "nullopt");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── appending, and refusing to append twice ───────────────────────
|
||||||
|
//
|
||||||
|
// bunq retries a callback about six times, and a periodic pull re-reads an
|
||||||
|
// overlapping window, so the same payment WILL arrive more than once. If a
|
||||||
|
// second copy landed in the file it would double the money against one
|
||||||
|
// reference — enough to settle a part-paid order whose balance never came.
|
||||||
|
{
|
||||||
|
const std::filesystem::path dir =
|
||||||
|
std::filesystem::temp_directory_path() / "cc-bunq-append-test";
|
||||||
|
std::error_code ec;
|
||||||
|
std::filesystem::remove_all(dir, ec);
|
||||||
|
std::filesystem::create_directories(dir, ec);
|
||||||
|
const std::filesystem::path credits = dir / "credits.jsonl";
|
||||||
|
|
||||||
|
Server::BankCredit c;
|
||||||
|
c.id = "999";
|
||||||
|
c.reference = "CC-2B6457";
|
||||||
|
c.amountMinor = 2500;
|
||||||
|
c.method = "sepa";
|
||||||
|
|
||||||
|
Check(Server::AppendCreditTo(credits, c), "the first append succeeds");
|
||||||
|
Check(!Server::AppendCreditTo(credits, c),
|
||||||
|
"the same payment id is refused the second time");
|
||||||
|
auto src = Server::MakeFileCreditSource(credits);
|
||||||
|
const auto all = src->Recent();
|
||||||
|
Check(all && all->size() == 1, "the file holds exactly one copy",
|
||||||
|
all ? std::format("{}", all->size()) : "nullopt");
|
||||||
|
if (all) {
|
||||||
|
Check(Server::MatchCredits(*all, "CC-2B6457").paidMinor == 2500,
|
||||||
|
"so the order sees 25.00 once, not twice");
|
||||||
|
}
|
||||||
|
// A credit with no id cannot be deduplicated and must be refused.
|
||||||
|
Server::BankCredit noId = c;
|
||||||
|
noId.id.clear();
|
||||||
|
Check(!Server::AppendCreditTo(credits, noId), "a credit with no id is refused");
|
||||||
|
std::filesystem::remove_all(dir, ec);
|
||||||
|
}
|
||||||
|
|
||||||
if (failures != 0) {
|
if (failures != 0) {
|
||||||
std::println(std::cerr, "{} check(s) failed", failures);
|
std::println(std::cerr, "{} check(s) failed", failures);
|
||||||
return 1;
|
return 1;
|
||||||
|
|
|
||||||
94
tools/pull-and-ship-credits.sh
Executable file
94
tools/pull-and-ship-credits.sh
Executable file
|
|
@ -0,0 +1,94 @@
|
||||||
|
#!/bin/sh
|
||||||
|
# Read the bank from a machine bunq's API key actually permits, and hand the
|
||||||
|
# result to the server that cannot read it itself.
|
||||||
|
#
|
||||||
|
# WHY THIS EXISTS, so nobody "simplifies" it back:
|
||||||
|
#
|
||||||
|
# The bunq API key is IP-restricted at the KEY level, and bunq enforces that on
|
||||||
|
# EVERY call — not only at device registration. Registering the server's IP in
|
||||||
|
# the device's permitted_ips does NOT help; that was tried on 2026-08-20 and
|
||||||
|
# every read from the server still came back "Incorrect API key or IP address".
|
||||||
|
# So the only two options are to widen the key's allowlist in the bunq app, or
|
||||||
|
# to call from an address it already permits. This script is the second.
|
||||||
|
#
|
||||||
|
# It also happens to be the shape the project's key policy wants: a bunq key
|
||||||
|
# can INITIATE PAYMENTS (bunq has no read-only scope), so the public web host
|
||||||
|
# should never hold one. Here it never does — the server only ever reads a file
|
||||||
|
# of incoming credits.
|
||||||
|
#
|
||||||
|
# tools/pull-and-ship-credits.sh
|
||||||
|
#
|
||||||
|
# Environment (BUNQ_KEY is read from the repo-root .env, which is gitignored):
|
||||||
|
# CREDITS_HOST ssh destination (default: hetzner)
|
||||||
|
# CREDITS_REMOTE the file the rail reads (default: the production path)
|
||||||
|
# CREDITS_LOCAL local accumulating copy (default: ~/.cache/catcrafts)
|
||||||
|
#
|
||||||
|
# BUNQ_PERMITTED_IPS is read from .env if set. Leaving it unset registers the
|
||||||
|
# device with "*", which sounds worse than it is HERE: bunq already enforces the
|
||||||
|
# key's own IP allowlist on every call, so "*" on the device is overridden by
|
||||||
|
# the stricter thing. Pinning it as well is defence in depth, at the price of
|
||||||
|
# breaking on a DHCP change — and a residential address does change. Note the
|
||||||
|
# registration happens ONCE, so switching later means deleting the context file
|
||||||
|
# and spending two more of the day's setup calls.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
HOST="${CREDITS_HOST:-hetzner}"
|
||||||
|
REMOTE="${CREDITS_REMOTE:-/var/lib/catcrafts/orders.jsonl.transfer-credits.jsonl}"
|
||||||
|
LOCAL_DIR="${CREDITS_LOCAL:-$HOME/.cache/catcrafts}"
|
||||||
|
LOCAL="$LOCAL_DIR/orders.jsonl"
|
||||||
|
|
||||||
|
[ -f .env ] || { echo "$0: no .env — the bunq key lives there" >&2; exit 1; }
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
set -a; . ./.env; set +a
|
||||||
|
|
||||||
|
# The .env still calls it BUNQ_KEY, from the previous integration. Accept both
|
||||||
|
# rather than making someone rename a working secret.
|
||||||
|
KEY="${BUNQ_API_KEY:-${BUNQ_KEY:-}}"
|
||||||
|
[ -n "$KEY" ] && [ -n "${TRANSFER_IBAN:-}" ] || {
|
||||||
|
echo "$0: need BUNQ_KEY (or BUNQ_API_KEY) and TRANSFER_IBAN in .env" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Exactly one server binary, or fail loudly: a variant directory embeds a
|
||||||
|
# config hash, so two matches means picking one would be a coin flip.
|
||||||
|
matches=$(find bin -maxdepth 1 -type d -name 'Catcrafts.Server-*' 2>/dev/null | sort)
|
||||||
|
count=$(printf '%s\n' "$matches" | grep -c . || true)
|
||||||
|
[ "$count" = 1 ] || { echo "$0: expected one Catcrafts.Server-* under bin/, found $count" >&2; exit 1; }
|
||||||
|
BIN="$matches/catcrafts-server"
|
||||||
|
|
||||||
|
mkdir -p "$LOCAL_DIR"
|
||||||
|
chmod 700 "$LOCAL_DIR"
|
||||||
|
|
||||||
|
# Pull. Appends only what is new, deduplicated on bunq's own payment id, so
|
||||||
|
# running this every few minutes over an overlapping window cannot double-count
|
||||||
|
# a payment into settling an order twice.
|
||||||
|
BUNQ_API_KEY="$KEY" "$BIN" --pull-credits --orders "$LOCAL" >/dev/null
|
||||||
|
|
||||||
|
CREDITS="$LOCAL.transfer-credits.jsonl"
|
||||||
|
[ -f "$CREDITS" ] || { echo "$0: nothing pulled, not shipping" >&2; exit 1; }
|
||||||
|
# This file is the business's bank statement in miniature, and the context file
|
||||||
|
# beside it holds a private key. The directory is 0700 already; narrow the files
|
||||||
|
# too rather than inheriting whatever umask happened to apply.
|
||||||
|
chmod 600 "$CREDITS" "$LOCAL.transfer-credits.jsonl.bunq-context.json" 2>/dev/null || true
|
||||||
|
|
||||||
|
# Ship INCOMING credits only. Outgoing lines are supplier payments and card
|
||||||
|
# spending: the matcher ignores negative amounts anyway, so sending them would
|
||||||
|
# put the business's outgoing payment history on a public-facing host for no
|
||||||
|
# settlement benefit at all.
|
||||||
|
TMP=$(mktemp); trap 'rm -f "$TMP"' EXIT
|
||||||
|
python3 - "$CREDITS" >"$TMP" <<'PY'
|
||||||
|
import json, sys
|
||||||
|
for line in open(sys.argv[1]):
|
||||||
|
line = line.strip()
|
||||||
|
if line and json.loads(line)["amount_minor"] > 0:
|
||||||
|
print(line)
|
||||||
|
PY
|
||||||
|
|
||||||
|
# Refuse to ship an empty file over a good one: a parse failure upstream must
|
||||||
|
# not blank the evidence the server settles from.
|
||||||
|
[ -s "$TMP" ] || { echo "$0: no incoming credits — refusing to overwrite the remote" >&2; exit 1; }
|
||||||
|
|
||||||
|
ssh "$HOST" "cat > $REMOTE && chown catcrafts:catcrafts $REMOTE && chmod 600 $REMOTE" < "$TMP"
|
||||||
|
echo "$0: shipped $(wc -l < "$TMP") incoming credit(s) to $HOST:$REMOTE"
|
||||||
Loading…
Reference in a new issue