refactor: call the benchmark's comparison the legacy format

The README, a comment in :Reflect and tests/Benchmark named the project
whose transaction log this library replaces. That name means nothing to
people using the library, so the benchmark's reference implementation is
now the `Legacy` namespace, its column reads `legacy ms`, and the prose
describes it as the hand-rolled iostream format Crafter.DB replaces.
Timings and logic are unchanged; the suite still passes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jorijn van der Graaf 2026-10-31 20:51:14 +01:00
commit eb720058f5
Signed by: jorijnvdgraaf
GPG key ID: 2937E59CDCC1BCFB
3 changed files with 33 additions and 33 deletions

View file

@ -1,6 +1,6 @@
# Crafter.DB # Crafter.DB
An embedded, typed table store for C++26: a table is a plain struct, a file is a memory-mapped array of fixed-width rows, queries are C++. Built for the Catcrafts projects (it replaces Nostro's hand-rolled transaction log) and distributed as one C++ module, `Crafter.DB`. An embedded, typed table store for C++26: a table is a plain struct, a file is a memory-mapped array of fixed-width rows, queries are C++. Built for the Catcrafts projects (it replaces a hand-rolled, iostream-based transaction log) and distributed as one C++ module, `Crafter.DB`.
- **Structs are the schema.** Any aggregate with a `version` is a table. No macros, no registration, no code generation: member names, types and offsets are derived at compile time. - **Structs are the schema.** Any aggregate with a `version` is a table. No macros, no registration, no code generation: member names, types and offsets are derived at compile time.
- **Fixed-width rows, out-of-line strings.** Every row is `rowSize` bytes. Strings and blobs live in a heap region; the row holds a 16-byte handle that keeps values of up to 12 bytes, and the first 4 bytes of longer ones, inline, so `iban == x` is one 8-byte compare per row. - **Fixed-width rows, out-of-line strings.** Every row is `rowSize` bytes. Strings and blobs live in a heap region; the row holds a 16-byte handle that keeps values of up to 12 bytes, and the first 4 bytes of longer ones, inline, so `iban == x` is one 8-byte compare per row.
@ -194,18 +194,18 @@ One file per table: a 4 KiB header (magic, format and schema version, fingerprin
## Performance ## Performance
`tests/Benchmark` compares Crafter.DB with a cleaned-up copy of Nostro's format (iostream, 2-byte length-prefixed strings) on 100 000 transactions, warm cache, one run on a Xeon w7-3465X: `tests/Benchmark` compares Crafter.DB with a cleaned-up copy of the legacy format it replaces (iostream, 2-byte length-prefixed strings, linear scans) on 100 000 transactions, warm cache, one run on a Xeon w7-3465X:
| Operation | Nostro | Crafter.DB | Note | | Operation | Legacy | Crafter.DB | Note |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| append 100 000 rows | 16.5 ms | 11.1 ms | Crafter.DB includes `fdatasync`; Nostro syncs nothing | | append 100 000 rows | 16.5 ms | 11.1 ms | Crafter.DB includes `fdatasync`; the legacy format syncs nothing |
| last timestamp, 1000 × open + read | 1.9 ms | 30.0 ms | opening a mapped table costs one `mmap` per extent (about 20 for this size); open once per process | | last timestamp, 1000 × open + read | 1.9 ms | 30.0 ms | opening a mapped table costs one `mmap` per extent (about 20 for this size); open once per process |
| load all, materialised | 36.0 ms | 10.4 ms | zero-copy scan of two columns: 0.8 ms | | load all, materialised | 36.0 ms | 10.4 ms | zero-copy scan of two columns: 0.8 ms |
| time range, 10 % of rows | 10.0 ms | 0.9 ms | binary search + contiguous slice | | time range, 10 % of rows | 10.0 ms | 0.9 ms | binary search + contiguous slice |
| `iban == x` | 16.2 ms | 0.23–0.30 ms | 2–3 ns per row | | `iban == x` | 16.2 ms | 0.23–0.30 ms | 2–3 ns per row |
| time range + `iban == x` | 11.2 ms | 0.03 ms | | | time range + `iban == x` | 11.2 ms | 0.03 ms | |
File size for that table: 17.2 MB against Nostro's 9.3 MB (fixed-width handles plus the slack of the last extents; `Compact()` right-sizes). File size for that table: 17.2 MB against the legacy format's 9.3 MB (fixed-width handles plus the slack of the last extents; `Compact()` right-sizes).
## Building ## Building

View file

@ -72,7 +72,7 @@ export namespace Crafter::DB {
return raw.substr(dot + 1, end - dot - 1); return raw.substr(dot + 1, end - dot - 1);
} }
// The type's qualified name as the compiler spells it ("Nostro::Transaction"). // The type's qualified name as the compiler spells it ("Ledger::Transaction").
// Used in diagnostics only, so the exact spelling is not load-bearing. // Used in diagnostics only, so the exact spelling is not load-bearing.
template<class T> template<class T>
consteval std::string_view TypeName() { consteval std::string_view TypeName() {

View file

@ -1,9 +1,9 @@
// SPDX-License-Identifier: LGPL-3.0-only // SPDX-License-Identifier: LGPL-3.0-only
// SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts® // SPDX-FileCopyrightText: Copyright (C) 2026 Catcrafts®
// Crafter.DB against a cleaned-up copy of Nostro's hand-rolled format // Crafter.DB against a cleaned-up copy of the legacy format it replaces
// (iostream, uint16-prefixed strings inline, linear scans) on the five // (iostream, uint16-prefixed strings inline, linear scans) on the five
// operations Nostro performs. Prints a table; never fails. Runs on 100k // operations that log served. Prints a table; never fails. Runs on 100k
// rows so a full suite stays fast; raise `rowCount` for a serious run. // rows so a full suite stays fast; raise `rowCount` for a serious run.
import Crafter.DB; import Crafter.DB;
@ -49,8 +49,8 @@ namespace {
return rows; return rows;
} }
// --- the Nostro format, cleaned up ----------------------------------------- // --- the legacy format, cleaned up -----------------------------------------
namespace Nostro { namespace Legacy {
void WriteString(std::ofstream& file, std::string_view str) { void WriteString(std::ofstream& file, std::string_view str) {
const std::uint16_t size = static_cast<std::uint16_t>(str.size()); const std::uint16_t size = static_cast<std::uint16_t>(str.size());
file.write(reinterpret_cast<const char*>(&size), sizeof(size)); file.write(reinterpret_cast<const char*>(&size), sizeof(size));
@ -139,7 +139,7 @@ namespace {
struct Line { struct Line {
std::string name; std::string name;
double nostro; double legacy;
double crafter; double crafter;
std::string note; std::string note;
}; };
@ -149,7 +149,7 @@ int main() {
const fs::path dir = fs::temp_directory_path() / "crafter-db-benchmark"; const fs::path dir = fs::temp_directory_path() / "crafter-db-benchmark";
fs::remove_all(dir); fs::remove_all(dir);
fs::create_directories(dir); fs::create_directories(dir);
const fs::path nostroPath = dir / "nostro.bin"; const fs::path legacyPath = dir / "legacy.bin";
const fs::path crafterPath = dir / "crafter.cdb"; const fs::path crafterPath = dir / "crafter.cdb";
const std::vector<Transaction> rows = MakeRows(); const std::vector<Transaction> rows = MakeRows();
const std::string_view wantedIban = rows[rowCount / 2].iban; const std::string_view wantedIban = rows[rowCount / 2].iban;
@ -160,84 +160,84 @@ int main() {
// append // append
{ {
const double nostro = Millis([&] { Nostro::Save(nostroPath, rows); }); const double legacy = Millis([&] { Legacy::Save(legacyPath, rows); });
const double crafter = Millis([&] { const double crafter = Millis([&] {
DB::Table<Transaction> table(crafterPath); DB::Table<Transaction> table(crafterPath);
table.Append(rows); table.Append(rows);
table.Flush(); table.Flush();
}); });
lines.push_back({ std::format("append {} rows", rowCount), nostro, crafter, "incl. open, Crafter.DB incl. fdatasync" }); lines.push_back({ std::format("append {} rows", rowCount), legacy, crafter, "incl. open, Crafter.DB incl. fdatasync" });
} }
// last timestamp (1000 opens each) // last timestamp (1000 opens each)
{ {
const double nostro = Millis([&] { for (std::int32_t i = 0; i < 1000; i++) sink += Nostro::LastTime(nostroPath).time_since_epoch().count() & 1; }); const double legacy = Millis([&] { for (std::int32_t i = 0; i < 1000; i++) sink += Legacy::LastTime(legacyPath).time_since_epoch().count() & 1; });
const double crafter = Millis([&] { const double crafter = Millis([&] {
for (std::int32_t i = 0; i < 1000; i++) { for (std::int32_t i = 0; i < 1000; i++) {
const DB::Table<Transaction> table(crafterPath, { .readOnly = true }); const DB::Table<Transaction> table(crafterPath, { .readOnly = true });
sink += table.Back()[&Transaction::timePoint].time_since_epoch().count() & 1; sink += table.Back()[&Transaction::timePoint].time_since_epoch().count() & 1;
} }
}); });
lines.push_back({ "last timestamp (1000x open+read)", nostro, crafter, "open dominates both" }); lines.push_back({ "last timestamp (1000x open+read)", legacy, crafter, "open dominates both" });
} }
DB::Table<Transaction> table(crafterPath, { .readOnly = true }); DB::Table<Transaction> table(crafterPath, { .readOnly = true });
// load all // load all
{ {
const double nostro = Millis([&] { sink += Nostro::LoadAll(nostroPath).size(); }); const double legacy = Millis([&] { sink += Legacy::LoadAll(legacyPath).size(); });
const double crafter = Millis([&] { sink += table.Materialize().size(); }); const double crafter = Millis([&] { sink += table.Materialize().size(); });
double zeroCopy = Millis([&] { double zeroCopy = Millis([&] {
std::int64_t total = 0; std::int64_t total = 0;
for (DB::Row<Transaction> row : table) total += row[&Transaction::amount] + static_cast<std::int64_t>(row[&Transaction::iban].Length()); for (DB::Row<Transaction> row : table) total += row[&Transaction::amount] + static_cast<std::int64_t>(row[&Transaction::iban].Length());
sink += static_cast<std::size_t>(total & 1); sink += static_cast<std::size_t>(total & 1);
}); });
lines.push_back({ "load all (materialised)", nostro, crafter, std::format("zero-copy scan of two columns: {:.2f} ms", zeroCopy) }); lines.push_back({ "load all (materialised)", legacy, crafter, std::format("zero-copy scan of two columns: {:.2f} ms", zeroCopy) });
} }
// time range (10% of rows) // time range (10% of rows)
{ {
std::size_t nostroCount = 0; std::size_t legacyCount = 0;
std::size_t crafterCount = 0; std::size_t crafterCount = 0;
const double nostro = Millis([&] { nostroCount = Nostro::LoadRange(nostroPath, from, to).size(); }); const double legacy = Millis([&] { legacyCount = Legacy::LoadRange(legacyPath, from, to).size(); });
const double crafter = Millis([&] { crafterCount = table.Range<&Transaction::timePoint>(from, to).Materialize().size(); }); const double crafter = Millis([&] { crafterCount = table.Range<&Transaction::timePoint>(from, to).Materialize().size(); });
lines.push_back({ "time range (10% of rows, materialised)", nostro, crafter, std::format("{} / {} rows", nostroCount, crafterCount) }); lines.push_back({ "time range (10% of rows, materialised)", legacy, crafter, std::format("{} / {} rows", legacyCount, crafterCount) });
} }
// iban == // iban ==
{ {
std::size_t nostroCount = 0; std::size_t legacyCount = 0;
std::size_t keyCount = 0; std::size_t keyCount = 0;
std::size_t exprCount = 0; std::size_t exprCount = 0;
std::size_t viewCount = 0; std::size_t viewCount = 0;
const double nostro = Millis([&] { nostroCount = Nostro::LoadByIban(nostroPath, wantedIban).size(); }); const double legacy = Millis([&] { legacyCount = Legacy::LoadByIban(legacyPath, wantedIban).size(); });
const double withKey = Millis([&] { const double withKey = Millis([&] {
const DB::Key key(wantedIban); const DB::Key key(wantedIban);
keyCount = table.Where([&](DB::Row<Transaction> r) { return r[&Transaction::iban] == key; }).Count(); keyCount = table.Where([&](DB::Row<Transaction> r) { return r[&Transaction::iban] == key; }).Count();
}); });
const double withExpression = Millis([&] { exprCount = table.Where(iban == wantedIban).Count(); }); const double withExpression = Millis([&] { exprCount = table.Where(iban == wantedIban).Count(); });
const double withView = Millis([&] { viewCount = table.Where([&](DB::Row<Transaction> r) { return r[&Transaction::iban] == wantedIban; }).Count(); }); const double withView = Millis([&] { viewCount = table.Where([&](DB::Row<Transaction> r) { return r[&Transaction::iban] == wantedIban; }).Count(); });
lines.push_back({ "iban == x (lambda + Key)", nostro, withKey, std::format("{} / {} rows, {:.1f} ns/row", nostroCount, keyCount, withKey * 1e6 / rowCount) }); lines.push_back({ "iban == x (lambda + Key)", legacy, withKey, std::format("{} / {} rows, {:.1f} ns/row", legacyCount, keyCount, withKey * 1e6 / rowCount) });
lines.push_back({ "iban == x (expression)", nostro, withExpression, std::format("{} rows, {:.1f} ns/row", exprCount, withExpression * 1e6 / rowCount) }); lines.push_back({ "iban == x (expression)", legacy, withExpression, std::format("{} rows, {:.1f} ns/row", exprCount, withExpression * 1e6 / rowCount) });
lines.push_back({ "iban == x (lambda + string_view)", nostro, withView, std::format("{} rows, {:.1f} ns/row", viewCount, withView * 1e6 / rowCount) }); lines.push_back({ "iban == x (lambda + string_view)", legacy, withView, std::format("{} rows, {:.1f} ns/row", viewCount, withView * 1e6 / rowCount) });
} }
// range + iban // range + iban
{ {
std::size_t nostroCount = 0; std::size_t legacyCount = 0;
std::size_t crafterCount = 0; std::size_t crafterCount = 0;
const double nostro = Millis([&] { const double legacy = Millis([&] {
for (const Transaction& tr : Nostro::LoadRange(nostroPath, from, to)) nostroCount += tr.iban == wantedIban ? 1 : 0; for (const Transaction& tr : Legacy::LoadRange(legacyPath, from, to)) legacyCount += tr.iban == wantedIban ? 1 : 0;
}); });
const double crafter = Millis([&] { crafterCount = table.Where(timePoint >= from && timePoint <= to && iban == wantedIban).Count(); }); const double crafter = Millis([&] { crafterCount = table.Where(timePoint >= from && timePoint <= to && iban == wantedIban).Count(); });
lines.push_back({ "time range + iban == x", nostro, crafter, std::format("{} / {} rows", nostroCount, crafterCount) }); lines.push_back({ "time range + iban == x", legacy, crafter, std::format("{} / {} rows", legacyCount, crafterCount) });
} }
std::println("{:<42} {:>12} {:>12} {:>8} {}", "operation", "nostro ms", "crafter ms", "speedup", "note"); std::println("{:<42} {:>12} {:>12} {:>8} {}", "operation", "legacy ms", "crafter ms", "speedup", "note");
for (const Line& line : lines) { for (const Line& line : lines) {
std::println("{:<42} {:>12.3f} {:>12.3f} {:>7.1f}x {}", line.name, line.nostro, line.crafter, line.nostro / std::max(line.crafter, 0.0001), line.note); std::println("{:<42} {:>12.3f} {:>12.3f} {:>7.1f}x {}", line.name, line.legacy, line.crafter, line.legacy / std::max(line.crafter, 0.0001), line.note);
} }
std::println("files: nostro {} bytes, crafter {} bytes", fs::file_size(nostroPath), fs::file_size(crafterPath)); std::println("files: legacy {} bytes, crafter {} bytes", fs::file_size(legacyPath), fs::file_size(crafterPath));
std::println("(sink {})", sink & 1); std::println("(sink {})", sink & 1);
fs::remove_all(dir); fs::remove_all(dir);