server/paged_heap library

Slotted-page row heap built on top of PagedFile.

This is the row-storage layer of the out-of-core engine: arbitrary byte payloads ("rows") are inserted into the heap, get back a stable 64-bit RowId, and can be fetched, replaced, deleted, or full-scanned without ever loading the whole file into memory.

Page layout

Page 0 — heap header (16 bytes used):

  offset  field
  ------  -----
       0  u32 magic = 'DHP1'
       4  u32 row count (informational; recomputed on scan)
       8  u32 next-allocation hint page (or 0)
      12  u32 reserved

Page N (N >= 1) — slotted data page:

  offset  field
  ------  -----
       0  u8  page kind (1 = data, 2 = overflow)
       1  u8  reserved
       2  u16 slot count
       4  u16 free-space start  (header + slots*4)
       6  u16 free-space end    (offset of lowest payload byte)
       8  slot[0..slotCount-1]: u16 payload offset, u16 payload length
      ..  free space ..
      ..  payloads (growing from end backward)

A slot whose length == 0xFFFF is a tombstone — the slot id is retired and may not be reused (so RowIds remain stable across re-opens).

A slot whose payload begins with 0xFF 0xFF 0xFF 0xFF followed by [u32 firstOverflowPage][u32 totalLength][bytes…] is an overflow pointer: the row was too big to fit on a single data page and its real bytes live in a chain of overflow pages, each of which is:

  offset  field
  ------  -----
       0  u8  page kind = 2
       1  u8  reserved
       2  u16 bytes-on-this-page
       4  u32 next overflow page (0 = end of chain)
       8  payload bytes (up to pageSize - 8)

RowId

A RowId is (pageNo << 16) | slotIdx, so we support up to 65536 slots per page and 2^48 / pageSize rows in total — vastly more than any in-RAM workload.

Concurrency

Same model as PagedFile: callers must serialise mutations externally (the executor's AsyncRwLock write lock already does this). Reads are safe to run concurrently with other reads on the same handle.