Flavors · one engine, pluggable codecs
Pick the codec that matches your ids.
@cloudbitmaps/core is codec-agnostic. The cache over immutable generations, the load-and-publish write path, chunk-skipping intersection, generation GC, erasure-by-rewrite, encryption and retention live there, with the in-memory and local-disk drivers and the ports the S3, GCS and Azure Blob packages implement. None of it changes with the codec, and neither do the four storage drivers. A flavor decides one thing — how a chunk of ids is represented on the wire and at rest.
Choose one
One of these ships. The other two are here so the choice is visible, and why there is only one: a plain bitset is not planned, and the third is research with nothing to install. Neither has a page, because there is no page for a package you cannot install.
| Flavor | Status | Take it when | Cost of choosing it |
|---|---|---|---|
| @cloudbitmaps/roaring Adaptive containers — array, bitset, run | Shipped · v0.27.0 | Ids are sparse or clustered and adaptive containers do real compression work. The default: if you are moving off Redis sets, start here. | Dense data gives containers nothing to compress. |
| @cloudbitmaps/bitset One bit per id — no containers, no branches | Not planned | Nothing, and that is a measured result rather than an opinion. Above about 6% density a Roaring chunk of scattered ids is an uncompressed bitset, and a contiguous one is smaller still as runs, so there is no size to win: on the shape built to favour a plain bitset it comes out 2% ahead, and Roaring wins the other three by 543×, 63× and 1.88×. | It would be a plausible wrong turn — picked for dense ids, where Roaring has already become the same bitset. |
| @cloudbitmaps/soaring A separate algorithm, unproven | Research | Under research, with nothing to install. It ships only if the algorithm proves out against roaring on real workloads — and if it does not, this row goes away. | — |
The flavors are the ones this project ships, and a new one is one row here and one figure on its own page. The codec seam is internal to the engine, so there is no supported way to add a flavor from outside.
Where the choice does not matter
Every flavor is a codec plugged into the same store. Swapping one is an import change and a re-load of your segments — the drivers, the compare-and-swap pointer protocol, the cache and the container format around the payload all stay put.
Cost
$2.40 per million cold intersects of two segments sharing 100 of 1,999 chunks (6 GETs, measured on S3), and $11.20 per million first single-part store.load() calls (2 PUT + 3 GET, measured on S3), pointer included, against $346/mo standing for a three-node Redis-HA cluster. Billed by the request either way — a codec changes bytes, not the pricing model.
The moat
100 of 2,000 chunks fetched when intersecting two 2,000,000-id segments — the shared keys only. Key alignment happens above the codec, so it survives every flavor. Step through it.
When to walk away
No flavor fixes this. Sub-millisecond p99 on a working set that fits a bounded cache is a different machine; a membership check that misses the cache costs one ranged GET against object storage, once the segment's pointer and index are read, where a RAM store costs a memory read. Use Redis.
The API, whichever you pick
has · count · iterate, and the full set algebra — intersect, union, andNot — plus *Into variants that publish the result as a new generation of a destination segment instead of returning it. iterate and every combine also read an id range, { after, through }, fetching only the chunks it overlaps. Data arrives the one way whichever you pick — store.load, one immutable object then a compare-and-swap of the pointer. A flavor is the codec the engine builds its bitmaps through: the engine hands it each chunk's bytes, above the same four drivers and two tiers, and every call on this card is the engine's, so no flavor changes its signature. The engine's main entry is internal, so the codec seam is not a public extension point.
Start with roaring.
It is the one that ships, and it is the right default unless you already know your ids are dense. The memory drivers need no configuration, so the quickstart runs before you decide anything about storage.
npm i @cloudbitmaps/roaring @cloudbitmaps/s3
# the codec, and the storage you have —
# or @cloudbitmaps/gcs · @cloudbitmaps/azure-blob
# the memory and local drivers need the codec alone