CloudBitmaps benchmarks

Benchmarks · every claim here is CI-gated

See exactly where pay-per-use beats a flat cache — and where it doesn't.

CloudBitmaps bills per request and per byte; a Redis-HA node bills a flat monthly rate. Below a certain sustained write/read rate, pay-per-use is far cheaper; above it, the flat node wins. This page draws that crossover straight from the library's own estimateCost() — and the numbers behind it are turned into build-breaking tests, so the story can never drift ahead of reality.

The crossover

Monthly cost vs sustained load

The shaded green region is where CloudBitmaps costs less than a flat Redis-HA node; the red region is the honest lose-zone where the flat node is cheaper. We never hide it.

Where pay-per-use crosses a flat Redis-HA node aws-us-east-1-ondemand · 8 KiB items · cache off · flat Redis-HA = $346/mo CloudBitmaps (pay-per-use)Redis HA (flat) Sustained writes — the metered cost driver cheaper than Redis lose-zone $0 $200 $400 $600 0 15 30 45 writes / sec crossover ≈ 26.3 /s Sustained reads (Topology-B warm, strongly-consistent) cheaper than Redis lose-zone $0 $200 $400 $600 0 250 500 750 reads / sec crossover ≈ 527 /s
Generated by pnpm bench from the shipped estimateCost() + the default aws-us-east-1-ondemand pricing — so the chart can't drift from the library.
ScenarioValueBasisVerdict
At-rest (1.2 GiB, no traffic)$0.03/mo0.008% of Rediswin-big
Write crossover26.33 writes/s8 KiB itemspast here Redis wins
Read crossover526.64 reads/sTopology-B, cache offpast here Redis wins
Redis-HA baseline$346/moflatthe comparison line

How it stays honest

The claims are tests, not slides

Every number above is asserted in CI as a deterministic, rate-independent anchor. A regression or an overclaim breaks the build — it can't ship:

  • Counting is free. count() on a warm-delta-free segment performs 0 payload reads — it sums cardinality straight from the .crbm index; only chunks with pending writes are fetched.
  • Chunk-skipping works. A 5%-overlap intersection fetches ≤ 10% of a full two-segment download (measured through the metrics sink).
  • Cheap at rest. The reference ~1.2 GiB set with no traffic costs ≤ 10% of a Redis-HA node.
  • We don't understate the loss. The modeled write crossover is ≥ the published rate — the lose-zone is never rosier than reality.
  • The estimator is trustworthy. Its prediction lands within ±20% of the engine's actual measured backend cost for a real workload.

Measured at scale

1K → 10K → 100K segments

Measured, not modeled: a real pnpm bench:scale run builds a fleet of up to 100K segments on local disk and reads across all of it. Machine-dependent — a snapshot, not a CI gate.

FleetLive heap (cap 1024)Peak RSSDiscovery scan
1,000 segments7.8 MiB63.1 MiB80.2 ms
10,000 segments7.9 MiB114.7 MiB1,522.7 ms
100,000 segments7 MiB178 MiB12,954.4 ms

Intersection of two 2,000,000-id segments (2,000 chunks each, 100 shared): fetched only 100 of the 2,000 chunks per segment — the shared keys; the rest skipped by key alignment — in 25.3 ms. Measured on Apple M3 Pro (arm64, node v24.14.1). The bound is the live heap (post-GC), flat at 7.8 MiB @ 1,000 · 7.9 MiB @ 10,000 · 7 MiB @ 100,000. Process peak RSS (for context) is a high-water that also folds in the benchmark's fleet-seeding allocations and isn't returned to the OS after GC, so it grows with fleet here — not a clean read-path footprint. Fleet seeded at ~42–55 durable segments/s (fsync-bound); discovery is LocalFs-bound (the O(total) shape is the point).

  • Memory tracks the working set, not the fleet. The cold-reader cache is capped at maxOpenSegments (default 1024), so retained live heap after reading the entire fleet is flat from 1K to 100K segments — a 100× larger fleet holds the same resident reader set.
  • Discovery is O(total segments) per cycle (the registry enumeration); sharding splits the Warm drain but not the enumeration, whose O(dirty) indexed cursor is a documented deferral.
  • Chunk-skipping holds at scale. Intersecting two large multi-chunk segments fetches only the shared chunks and skips the rest by key alignment.

Read the fine print

What these numbers assume

  • Default pricing. aws-us-east-1-ondemand, 8 KiB items, cache off, strongly-consistent warm reads. Your region, cloud, provisioned capacity, and cache-hit rate move the crossover — feed your own PricingProfile and workload to estimateCost().
  • Model, not a cloud bill. Dollars come from the cost formulas + published rates, not a live AWS invoice. Real-cloud dollar calibration is offline / launch-time.
  • Cost, not latency. Wall-clock latency/throughput are measured offline (too noisy to gate on shared CI runners); this chart is about money.

For the full method and the raw numbers, see the published benchmarks and the getting-started guide.