A full copy-on-write disk ends the process with SIGBUS: the commit record and in-place leaves are stores into MAP_SHARED pages #1

New issue

Open

opened 2026-11-01 20:51:20 +00:00 by jorijnvdgraaf · 0 comments

jorijnvdgraaf commented

2026-11-01 20:51:20 +00:00

Owner

A full copy-on-write disk ends the process with SIGBUS, outside every IOError. Crafter.DB makes its writes as CPU stores into MAP_SHARED mappings: the commit record each Flush and each open writes into the header pages (TableFile::WriteCommit → memcpy into headerPages.Data()), Set's in-place leaves (CopyLeaves into MutableSlot), Append's row and heap bytes. On a filesystem that rewrites a written page in place (ext4, f2fs) that is fine, and fallocate makes a full disk an IOError at the extent ("real blocks, so a full disk is an error here and never a SIGBUS on first touch of a mapped page", File.cppm). On a copy-on-write filesystem (btrfs with the default datacow) rewriting a page whose extent is already written needs new data space: btrfs_page_mkwrite fails its data reservation with ENOSPC, can_nocow_file_extent refuses a REG extent of a COW inode, and the fault returns VM_FAULT_SIGBUS. The store dies in the write — in TableFile's constructor already, since opening writes a commit record into a header page that Create wrote with pwrite.

How it bites a consumer (imsd-messages' pre-push review, round 37, 2026-11-01): the app's store catches IOError at one gate and tells its banner ("Cannot save messages (storage full?)"); on a full btrfs data directory the resident instance dies instead, and every relaunch dies the same way until space is freed. No text is lost (imsd keeps its copy; the write died before the delete), but a sent word being written becomes a retry at the next start. The app's claim was narrowed to filesystems that rewrite in place and the case is banked as a known 1.0 limitation on Jorijn's ruling; this issue is the follow-up.

Where (at bfd0cc2): implementations/Crafter.DB-File.cpp — the header mapping mmap(…, PROT_READ | PROT_WRITE, MAP_SHARED, …) (~~:241) and the extents' (~~:189); implementations/Crafter.DB-Storage.cpp — WriteCommit (~~:442–451), Flush (~~:453), the constructor's WriteCommit(false) (~~:306), MarkDeleted (~~:418); interfaces/Crafter.DB-Table.cppm — Set's CopyLeaves (~~:148), Update's memcpy (~~:114).

Ask: write what must land — the commit record at least, and the in-place leaves and tombstones — with File::WriteAt (pwrite) rather than stores into the shared mapping, so a full CoW disk surfaces as the IOError a consumer already catches; or document the limitation in the README's durability section and offer FS_NOCOW_FL on the files the library creates (btrfs honours it on an empty file; metadata ENOSPC and snapshot-shared extents remain). A test needs a btrfs loop device filled to ENOSPC; a reproducer is the consumer's store opened on one.

**A full copy-on-write disk ends the process with SIGBUS, outside every IOError.** Crafter.DB makes its writes as CPU stores into MAP_SHARED mappings: the commit record each Flush and each open writes into the header pages (`TableFile::WriteCommit` → memcpy into `headerPages.Data()`), `Set`'s in-place leaves (`CopyLeaves` into `MutableSlot`), `Append`'s row and heap bytes. On a filesystem that rewrites a written page in place (ext4, f2fs) that is fine, and `fallocate` makes a full disk an `IOError` at the extent ("real blocks, so a full disk is an error here and never a SIGBUS on first touch of a mapped page", File.cppm). On a copy-on-write filesystem (btrfs with the default datacow) rewriting a page whose extent is already written needs new data space: `btrfs_page_mkwrite` fails its data reservation with ENOSPC, `can_nocow_file_extent` refuses a REG extent of a COW inode, and the fault returns `VM_FAULT_SIGBUS`. The store dies in the write — in `TableFile`'s constructor already, since opening writes a commit record into a header page that `Create` wrote with pwrite. **How it bites a consumer** (imsd-messages' pre-push review, round 37, 2026-11-01): the app's store catches `IOError` at one gate and tells its banner ("Cannot save messages (storage full?)"); on a full btrfs data directory the resident instance dies instead, and every relaunch dies the same way until space is freed. No text is lost (imsd keeps its copy; the write died before the delete), but a `sent` word being written becomes a retry at the next start. The app's claim was narrowed to filesystems that rewrite in place and the case is banked as a known 1.0 limitation on Jorijn's ruling; this issue is the follow-up. **Where** (at bfd0cc2): `implementations/Crafter.DB-File.cpp` — the header mapping `mmap(…, PROT_READ | PROT_WRITE, MAP_SHARED, …)` (~:241) and the extents' (~:189); `implementations/Crafter.DB-Storage.cpp` — `WriteCommit` (~:442–451), `Flush` (~:453), the constructor's `WriteCommit(false)` (~:306), `MarkDeleted` (~:418); `interfaces/Crafter.DB-Table.cppm` — `Set`'s `CopyLeaves` (~:148), `Update`'s memcpy (~:114). **Ask**: write what must land — the commit record at least, and the in-place leaves and tombstones — with `File::WriteAt` (pwrite) rather than stores into the shared mapping, so a full CoW disk surfaces as the `IOError` a consumer already catches; or document the limitation in the README's durability section and offer `FS_NOCOW_FL` on the files the library creates (btrfs honours it on an empty file; metadata ENOSPC and snapshot-shared extents remain). A test needs a btrfs loop device filled to ENOSPC; a reproducer is the consumer's store opened on one.