How it's built, one chunk at a time.

Phase 1 is the brain with no storage yet — just the in-memory drivers. It establishes the data model every later tier has to honor: route an ID to its chunk, keep deletes as tombstones, cache decoded chunks under a hard ceiling, and guard concurrent writers with optimistic concurrency.

SafeBitmap wraps the native roaring addon but only ever uses its safe deserializer behind a size cap — every byte from any tier is treated as untrusted input. The warm row for a chunk is a delta { adds, removes }; writes carry an opaque, never-reused OCC token, so two writers to the same chunk can't silently lose an update (one gets a WriteConflictError and re-reads).

Phase 2 makes state survive a restart and defines the on-disk cold format. A .crbm archive is a sequence of per-chunk roaring blobs followed by a footer index — every chunk, the index, and the footer each carry a CRC32C, so corruption is caught before a byte reaches the native decoder. Cold objects are generation-keyed and immutable: segment.<gen>.crbm, never overwritten in place.

This phase also builds the correctness moat: a FoundationDB-style deterministic simulator that runs the real core/ against fake in-memory drivers under a seeded scheduler, injecting latency, errors, reordering and crashes — and checks every result against an oracle (a trivial Set). A mismatch prints the seed; the seed replays the failure byte-for-byte.

Phase 3 puts the cold tier on S3 (or any S3-compatible store like MinIO) and delivers the feature the whole project exists for. The S3ColdDriver lives at the @cloudbitmaps/roaring/s3 subpath, so the AWS SDK is an optional peer dependency — the core package never pulls it in. Generations are write-once via a conditional If-None-Match:* PUT.

The crown jewel is intersect(). To AND two segments, you only need the chunks whose 16-bit key appears in both — a key missing from either side contributes nothing and is never downloaded. So two billion-ID segments that barely overlap transfer almost nothing, and the whole thing streams under a bounded in-flight window: it runs in a small / serverless process regardless of segment size.

The warm tier moves to DynamoDB (subpath @cloudbitmaps/roaring/dynamodb, optional peer dep). A single table — PK = ns#…|seg#…, SK = chunk#… — holds the delta rows, and every write is a conditional write. Two processes mutating the same chunk now get genuine optimistic concurrency: one wins, the other gets a WriteConflictError and re-reads. The OCC token of Phase 1 is now backed by a real distributed compare-and-set.

Cloud storage throttles, returns 5xx, and drops connections. Every warm/cold call now automatically retries transient faults with bounded exponential backoff and full jitter — you don't wire anything. Crucially, deterministic errors are never retried: a ValidationError, IntegrityError, or WriteConflictError can't be helped by trying again, so they surface immediately.

Safety across a retry is the subtle part: a write that times out after it committed is caught on retry (the OCC token has advanced → a clean conflict, never a double-apply), and cold generations are write-once so there's no half-written object a reader could pick up. A transient outage costs you latency, not correctness.

Cold is a series of immutable, numbered generations; a reader needs to know which one is current. Without a registry, the cold source lists every generation and takes the max — a storage scan per segment. The registry replaces that with a single authoritative record (currentGen) you read once. It's optional (omit it, fall back to the scan), with memory / LocalFs / DynamoDB implementations. The record also reserves the slots later phases fill: status, dirtyChunkCount, and the wrapped-DEK list.

Every add/remove grows the warm tier; left alone, reads slow down as more deltas merge. Compaction folds the warm deltas into a fresh immutable cold generation and clears the warm rows — in a separate background process that never touches your app path. The hard part is doing it without ever losing a write that lands mid-compaction. The answer is a 2-phase commit, fenced on the per-chunk version it archived, that recovers cleanly from a crash at any step.

Encrypt the cold objects so a leaked bucket reveals neither IDs nor cardinality (payloads and the chunk index are encrypted; the footer's count fields are zeroed and re-derived after decryption). It's envelope encryption: you hold one root KEK (32 bytes, bring-your-own — no required cloud dependency); each segment gets a random DEK, wrapped under your KEK and stored in the registry. Each ciphertext's AAD binds it to (namespace, segment, generation, chunkKey) so a blob can't be lifted into another context.

That same design gives crypto-shred — GDPR "right to erasure" that works even on immutable or backed-up storage. Delete the wrapped DEK and the encrypted bytes are permanently unreadable everywhere, no overwrite needed. Encryption is fully opt-in: pass a keystore and it's on; set requireEncryption: true to forbid any cleartext write or read.

The final piece closes the project's founding promise: compaction must not buffer a whole segment in RAM. The merge became an async generator feeding a streaming cold sink; on S3 that's multipart upload with at most one part buffered (default 8 MiB). Peak write memory is ~one part whether the segment is 2 GiB or 50 GiB.

Write-once survived the rewrite: a small object is a single conditional PutObject; a large one finishes with a conditional CompleteMultipartUpload (If-None-Match:*). And the object ceiling is honest — S3 caps multipart at 10,000 parts, so the cap is partBytes × 10,000 (≈80 GiB at the default, growing via partBytes up to S3's 5 TiB), enforced with a typed error rather than a silent overflow.

There's no new API to call — it's the same runCompactionCycle and the same S3ColdDriver from §4d/§3. Large objects simply upload via multipart automatically.