Storage expansion and cost model

This page records the measured V1 storage behavior on origin/main commit a9cddf6. It is evidence for choosing a policy, not a promise about cloud-provider prices or a final security claim matrix. The latter remains tracked in #52.

Exact byte formula

For a plaintext chunk of p bytes, k = dataShards, m = parityShards, and n = k + m:

ciphertext bytes = p + 16
bytes per shard  = ceil((p + 16) / k)
encoded bytes    = n * ceil((p + 16) / k)
padding bytes    = k * ceil((p + 16) / k) - (p + 16)
raw factor       = n / k

The 16 bytes are the AEAD authentication tag on every encrypted chunk. The raw factor is only the large-chunk asymptote. A file also pays one tag and up to k - 1 padding bytes per chunk, plus any recursive manifest blobs. Manifests are encrypted and erasure-coded with the same policy when the serialized chunk list exceeds the current 8,192-byte threshold.

Measured baseline

The committed harness uses deterministic bytes, a 64 KiB fixed chunker, the production FilePipeline, production convergent encryption and Reed-Solomon, and in-memory providers in one Node.js v24.19.0 process on Darwin arm64. The large case is 16 MiB (256 data chunks). Every input was uploaded and downloaded byte-for-byte. Provider latency, browser or mobile CPU, signer latency, and internet throughput are not measured here.

The manifest contribution in the 16 MiB cases is 43,615 B, 122,558 B, 118,440 B, 197,952 B, and 289,500 B respectively. The sealed root indexes are 510 B, 592 B, 1,037 B, 2,045 B, and 3,720 B respectively; those bytes are the registry payload when execution is configured, rather than object-storage bytes. The default FastCDC run produced 227 chunks for the same deterministic 16 MiB input; its 1+0, 1+1, 2+1, 4+2, and 7+3 factors were 1.00252x, 2.00691x, 1.50660x, 1.51082x, and 1.44424x. Exact results and per-provider splits are preserved in the immutable evidence revision 68e20aa: storage-cost-baseline.json.

Every logical shard causes a provider HEAD. A missing shard then causes a PUT. A full read currently issues GET for every shard candidate, not only the minimum quorum. Each signed provider operation also causes a POST to the customer's signer service. Deduplication can remove PUT bytes and requests, but not the preceding HEAD; the baseline deliberately uses unique chunks.

Provider count is not redundancy

Shards are placed by position using providers[position % providers.length]. For 7+3 across five providers, each provider receives two of ten shards; losing any one provider removes two shards, leaving eight for a quorum of seven. For 4+2 across five providers, the split is 2, 1, 1, 1, 1, so any one provider loss is guaranteed but two are not. Adding a provider changes later uploads only: uploads must construct or refresh the pipeline with the new provider list, each FileIndex retains the ordered provider IDs used when that file was written, and there is no automatic rewrite of existing objects.

Defaults, overrides, and live callers

The selection trace was refreshed on 2026-08-31 against origin/main commit 82cef6f. FilePipeline.upload still requires both shard counts and never inserts a default. Repair, health, download, and purge paths consume the quorum already sealed into each FileIndex; they do not choose a new one.

At the traced revision, the frontend also contained a defaultUploadQuorum(providerCount) helper referenced only by its unit test. It was not a production caller; removing that dead helper leaves upload selection unchanged.

Shard counts remain explicit: callers can pass ...DEFAULT_ATTESTED_SHARD_POLICY for the documented five-provider configuration. The custody policy now defaults to "attested", which checks the actual placement before writing. Both four and five providers pass with 7+3; three are refused because losing the most-loaded provider would leave fewer than seven shards. This policy does not change shard counts or rewrite existing files.

custodyPolicy: "relaxed" is a deliberate opt-out for callers that accept weaker placement properties, such as a single-provider smoke test. It warns when either condition fails. Custody facts are returned for scoped uploads and public uploads with root chunks; empty public uploads omit them. The cost harness opts out explicitly so it can measure under-redundant configurations; those measurements are not a recommendation to use them. The former frontend helper was never the SDK default, and its 1+1 result for two providers was not a recommendation.

The fixtures record these mechanical properties for configurations already discussed in examples or tests:

• One provider: 1+0 gives encryption and integrity without provider-loss tolerance.
• Three providers: 2+1 gives a 1.5x raw factor and survives any one provider loss when each provider holds one shard.
• Five providers: 7+3 gives a 1.42857x raw factor and the placement and request counts measured above.
• With two providers, no shard placement can simultaneously keep each provider below quorum and survive the loss of either provider.
• Caller overrides remain supported; invalid policies are rejected before Reed-Solomon allocation.

Dated provider prices

The following public list prices were retrieved on 2026-08-31. They are marginal inputs before free tiers, rounding, tax, discounts, or committed-use pricing. Units are not normalized in the source data: AWS, Google, and Azure storage use binary GiB (2^30 bytes); R2 and Scaleway use the providers' listed GB units. Azure internet egress remains decimal GB because its bandwidth table defines 1 TB as 1,000 GB. The machine-readable record in packages/sdk/benchmarks/storage-pricing-inputs.json preserves the exact unit size, region, tier, route, currency, source, and source publication/effective date when the provider exposes one.

AWS's machine-readable price list was published 2026-08-18; Cloudflare's page was last updated 2026-08-07. Microsoft reports the selected Hot LRS meter as effective from 2017-02-03. Google and Scaleway do not expose a page update date, so the retrieval date is the only freshness claim for those rows.

Exact 16 MiB example

In the measured 16 MiB 7+3 / five-provider run, every provider leg held exactly 4,852,780 B and saw 520 PUT, 520 upload HEAD, and 520 GET for one full download. Applying each provider row to one leg gives:

Using one leg from each row would therefore be $0.0134948236 plus €0.0001264634 at marginal rates. This is arithmetic evidence, not the amount on a bill: at this size free tiers and whole-unit rounding dominate.

Capacity-planning example

For 1 TB of plaintext retained continuously, the raw 7+3 factor gives 1.4285714286 TB before per-chunk tags, padding, and recursive manifests. With the current even two-shards-per-provider placement, each of five providers gets 0.2857142857 TB. The storage-only marginal estimate for the same provider mix is $21.26 plus €4.59 per month. Use the measured large-file factor instead of the raw factor when a workload's chunk size and manifest depth are known.

Neither example includes signer-service POSTs, registry gas, taxes, minimum object sizes or storage durations, retries, provider-side replication, or account-wide billing rules. A full read currently fetches every shard candidate, so retrieval and egress can be charged on every provider leg rather than only a minimum quorum. Repair can add health-check reads, retrieval/egress, existence checks, and replacement writes. Before quoting a customer, refresh every row and record the provider, region, tier, currency, unit convention, request classes, egress route, free tier, and rounding rules for that customer's actual mix.