analytics: GoAccess reports from the Caddy logs, public and private tiers
Server-side only, per the privacy policy: no client-side analytics. Rotated logs ingest exactly once into per-tier DBs; the public tier is anonymized at ingest and never receives censored paths. Log retention raised to a year. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
parent
c1b0c29af6
commit
8cd4d55d00
6 changed files with 251 additions and 0 deletions
|
|
@ -343,6 +343,60 @@ curl -s https://catcrafts.net/projects | grep -c '<script' # 0
|
|||
curl -s https://catcrafts.net/projects | grep -o '<title>[^<]*'
|
||||
```
|
||||
|
||||
## Analytics
|
||||
|
||||
Server-side only — the privacy policy promises request logging and nothing
|
||||
else, so there is no client-side analytics anywhere on the site. GoAccess
|
||||
(Debian package) turns Caddy's JSON access logs into two static HTML
|
||||
reports, each with its own persistent DB and ingest ledger:
|
||||
|
||||
- `https://catcrafts.net/analytics/` — **public, censored.** Visitor IPs are
|
||||
anonymized at ingest (last octet zeroed before anything reaches its DB),
|
||||
no HOSTS or full-URL REFERRERS panels, and log lines matching `CENSOR_RE`
|
||||
in the script never enter its DB at all — the public tier cannot leak
|
||||
what it never ingested. Extend `CENSOR_RE` when the shop launches so
|
||||
order/payment URLs can never surface; keep secrets out of URL *paths*
|
||||
regardless (query strings are already stripped).
|
||||
- `https://catcrafts.net/analytics/private/` — **uncensored** (basic auth,
|
||||
hash in the Caddyfile): full IPs, all panels.
|
||||
|
||||
Raw logs keep full IPs either way — that is the request logging the privacy
|
||||
policy declares; per-IP forensics work from the logs and the private tier,
|
||||
never from the public page.
|
||||
|
||||
Three pieces, all in `deploy/`:
|
||||
|
||||
- `catcrafts-analytics` → `/usr/local/bin/` — ingests each rotated
|
||||
`catcrafts.net-*.log.gz` exactly once into a persistent GoAccess DB
|
||||
(`/var/lib/goaccess/db`, tracked in `/var/lib/goaccess/ingested`), then
|
||||
renders the report from DB + live log. The live file is never persisted,
|
||||
so its lines don't double-count when Caddy rotates it. History therefore
|
||||
survives log deletion: the DB keeps aggregates forever.
|
||||
- `catcrafts-analytics.service` — oneshot, runs as `caddy` (owner of the
|
||||
0600 logs).
|
||||
- `catcrafts-analytics.timer` — hourly at :07.
|
||||
|
||||
Bot filtering is the load-bearing part: measured on real traffic, 57% of
|
||||
requests were headerless vulnerability scanners and another 19% self-declared
|
||||
bots (mostly ClaudeBot) — only ~24% human. `--ignore-crawlers
|
||||
--unknowns-as-crawlers` drops both groups. The flags in the script apply at
|
||||
ingest time and the DB stores aggregated data, so changing filters later only
|
||||
affects new lines — re-ingesting history means deleting
|
||||
`/var/lib/goaccess/{db,ingested}` and letting the next run rebuild from
|
||||
whatever raw logs retention still holds (a year, per the Caddyfile).
|
||||
|
||||
```sh
|
||||
apt install goaccess
|
||||
install -m 755 deploy/catcrafts-analytics /usr/local/bin/
|
||||
install -m 644 deploy/catcrafts-analytics.{service,timer} /etc/systemd/system/
|
||||
mkdir -p /etc/goaccess /var/lib/goaccess /var/www/analytics /var/www/analytics-private
|
||||
install -m 644 deploy/goaccess-browsers.list /etc/goaccess/browsers.list
|
||||
# own IPs to keep out of the numbers - host-only file, NOT in this repo
|
||||
echo "203.0.113.7" > /etc/goaccess/exclude-ips
|
||||
chown -R caddy:caddy /var/lib/goaccess /var/www/analytics /var/www/analytics-private
|
||||
systemctl daemon-reload && systemctl enable --now catcrafts-analytics.timer
|
||||
```
|
||||
|
||||
## Running it locally
|
||||
|
||||
```sh
|
||||
|
|
|
|||
Loading…
Reference in a new issue