# DataPrism > Documentation ## Architecture DataPrism combines client-side encryption, platform-managed object storage, and an on-chain index. Users write the *encrypted index* of their files into **prisms** (on-chain namespaced vaults) funded with **DPC** credits, while encrypted file shards are stored through short-lived, project-scoped URLs. The system spans three layers, **smart contracts** on the blockchain, off-chain **services** (API, storage, relayers), and the **client** (SDK + frontend). ### System map ``` ┌────────────────┐ create account / buy credits / create prism │ Enterprise │──────────────────────────────┐ │ (user) │ ▼ └───────┬────────┘ ┌──────────────────────┐ │ │ dataprism-frontend │ │ uses the SDK └──────────┬───────────┘ ▼ │ card payment ┌──────────────────┐ ▼ │ dataprism-sdk │ ┌─────────────────────────────┐ │ ┌─────────────┐ │ │ Stripe │ │ │ encryption │ │ └──────────────┬──────────────┘ │ │ chunk + RS │ │ │ webhook │ │ storage │ │ ▼ │ └─────────────┘ │ ┌─────────────────────────────┐ │ execution layer │ │ dataprism-payment-relayer │ │ wallet │ api │ └──────────────┬──────────────┘ └────┬─────────┬───┘ │ buy/transfer DPC │ │ │ shards│ on-chain ▼ ▼ │ ┌─────────────────────────────────┐ ┌─────────┐ │ (api) │ dataprism-api → dataprism-relayer │ Managed │◄───┤ signed URLs │ ──────────────────► on-chain │ │ storage │ └──► dataprism-api ──► │ │ └─────────┘ ┌───────────▼────────────────┐ │ (encrypted) │ dataprism-protocol │ │ │ Token (DPC) │ │ │ DataPrism / DataPrismCloud │ │ │ Forwarder (meta-tx) │ │ │ FeeController / Exchange │ │ └────────────────────────────┘ │ (prisms + encrypted index) ``` In the `dataprism-sdk` block, "encryption / chunk + RS / storage" is the full [file pipeline](/concepts/file-pipeline): **FastCDC chunking → convergent encryption → Reed-Solomon → content-addressing (dedup) → managed storage → encrypted Merkle index on-chain**. On the write side, the **execution layer** routes either to the wallet (gas paid in ETH) or to the API (gas fronted by the relayer, reimbursed in DPC). ### The two write paths Every on-chain write, creating a prism, writing a slot, storing a file index, goes through one of two interchangeable paths. The application code is identical; only the [execution layer](/sdk/execution) differs. ``` ┌──────── does the user hold ETH for gas? ────────┐ │ YES NO │ ▼ ▼ direct contract call call to dataprism-api (DataPrismCloud.*) with a signed request user pays the gas (EIP-712, gasless) │ │ │ dataprism-relayer executes │ the transaction on-chain │ (pays gas, takes a DPC fee) ▼ ▼ write committed on-chain ◄───────────────────────────┘ ``` ### Layers #### Smart contracts ([Protocol](/protocol/overview)) The on-chain core. `DataPrism` stores raw data per prism, `DataPrismCloud` adds the application layer (cloud IDs, writers, two-step transfers, enumeration, read-through getters), and `DataPrismCloudForwarder` enables gasless meta-transactions. `Token` (DPC) pays fees, `FeeController` prices actions, and `Exchange` lets users swap ERC-20 tokens (USDC) for DPC. A strict rule holds across the stack: every off-chain component talks **only** to `DataPrismCloud` / `Forwarder`, never to the core `DataPrism` contract directly. #### Client SDK ([SDK](/sdk/overview)) The client-side core, in TypeScript, runnable anywhere (browser, Node, scripts). It owns encryption (keys derived from a wallet signature, never stored), the file pipeline (chunking + Reed-Solomon + managed storage), and the unified execution layer for on-chain writes. The frontend uses it; integrators embed it directly. #### REST API ([API](/api/overview)) A serverless API (AWS Lambda + API Gateway + DynamoDB) with two responsibilities. It authorizes managed-storage operations and returns short-lived signed URLs, and it queues EIP-712 requests between users and relayers. Relayers poll for `pending` requests, claim them atomically, and report the outcome. The API never submits blockchain transactions directly. #### Relayers Two distinct services: * **[Relayer](/relayer/overview)** submits user data meta-transactions to `DataPrismCloudForwarder` in exchange for a `relayerFee` in DPC. * **[Payment Relayer](/payment-relayer/overview)** handles the fiat on-ramp: after a Stripe card payment, it buys DPC on the Exchange and transfers it to the user. #### Frontend ([Frontend](/frontend/overview)) A Next.js dashboard where users manage projects, storage, API keys, team members, and buy DPC tokens with a card or USDC. Identity is wallet-based; private keys never leave the browser. ### End-to-end flow: writing data (`WRITE_DATA_FOR`) This is the gasless (API) path. The wallet path collapses steps 2-7 into a single direct `DataPrismCloud.writeData(...)` call paid by the user. ``` 1. OFF-CHAIN SIGNING User computes the EIP-712 hash: WriteData(cloudId, dataId, keccak256(payload), relayerFee, timestamp, nonce) Signs with their private key → (v, r, s) 2. REQUEST SUBMISSION POST /requests { "chainName": "demo", "action": "WRITE_DATA_FOR", "params": { "cloudId": "0x...", "dataId": "0x...", "payload": "0x...", "relayerFee": "1000000000000000000", "timestamp": "1749600000", "signer": "0xUserAddress", "v": 27, "r": "0x...", "s": "0x..." } } → API validates, stores in DynamoDB with status=pending, returns requestId 3. RELAYER POLL GET /requests?status=pending → receives the request 4. CLAIM PATCH /requests/{id} { status: "processing" } → Atomic DynamoDB update (condition: status=pending) 5. PROFITABILITY CHECK estimateGas(writeDataFor) × gasPrice × ethPrice ≤ relayerFee ? 6. ON-CHAIN CALL Forwarder.writeDataFor(cloudId, dataId, payload, relayerFee, timestamp, signer, v, r, s) 7. INSIDE THE FORWARDER a. Checks the priority window b. Recomputes the EIP-712 hash using the signer's current nonce c. Verifies the signature (ECDSA or ERC-1271) d. Calls DataPrismCloud.writeDataAs(cloudId, signer, dataId, payload, relayerFee) 8. INSIDE DATAPRISMCLOUD a. Deducts protocol fees from the prism's DPC balance b. Transfers relayerFee DPC to the Forwarder c. Calls DataPrism.writeData(prismId, dataId, payload) d. Increments the signer's nonce 9. REPORT PATCH /requests/{id} { status: "success", txHash: "0x..." } ``` The same flow applies to other actions by swapping the signed typehash, the request `action`, and the `*For` function called. In particular, a combined **`CREATE_PRISM_WITH_DATA_FOR`** flow follows exactly these steps: it signs a `CreatePrismWithData` request, the relayer calls `Forwarder.createPrismWithDataFor(...)`, and `DataPrismCloud.createPrismWithDataAs(...)` creates the prism and writes the initial batch of entries in a single atomic transaction (one nonce increment). ### Fee model Two distinct fee levels apply to every operation. See the [SDK fee model](/sdk/execution#fees) for how the SDK surfaces them. #### Protocol fees * Paid in DPC, deducted from the **prism's token balance**. * Defined per action (`ADD_DATA`, `MODIFY_DATA`, `DELETE_DATA`, …) by the [FeeController](/protocol/contracts#feecontroller). * `DataPrismCloud` benefits from negotiated custom (zero) fees. * Sent to the FeeController's `treasury`. #### Relayer fees (`relayerFee`) * Set by the **user** at signing time (only relevant on the API path). * Deducted from the prism's balance and forwarded to the Forwarder. * Distributed based on relayer status: **official** relayers receive 100%; **non-official** relayers receive `relayerShareBps`% (default 70%), the remainder goes to the `protocolTreasury`. * Accumulated as **vouchers** and claimed via `claimVouchers()`. ### Request lifecycle (API path) ``` pending → processing → success ↘ failed ``` The `pending → processing` transition is **atomic** via a DynamoDB conditional expression, preventing two relayers from processing the same request simultaneously. ## Introduction **DataPrism** is a privacy-first storage layer with client-side encryption, managed object storage, and an on-chain index. It gives applications three core properties: 1. **Private by design**: encryption happens before upload. DataPrism stores encrypted shards, while the project key stays with the user. 2. **Storage that works out of the box**: the platform manages the bucket and authorizes each upload, download, check, or delete with a short-lived project-scoped URL. Users never paste cloud credentials into the app. 3. **An on-chain trust registry**: the index that says *where to find each fragment* is stored on a blockchain, **encrypted**, inside a namespaced vault called a **prism**. Nobody can alter or delete that index without your key. > Privacy is not given, it is built. Code is the only law that cannot be lobbied. ``` ┌─────────────────────────────────────────┐ │ DataPrism │ │ (encryption + managed storage + │ │ on-chain index + payments) │ └─────────────────────────────────────────┘ │ ▼ ┌────────────────────────┐ │ Managed object storage │ │ encrypted shards │ └────────────────────────┘ ``` ### Who it is for The target user is the **enterprise**. A company creates an account, tops up a balance of credits (by card or crypto), and then drives its storage programmatically through the [SDK](/sdk/overview). DataPrism handles encryption, managed storage access, payments, and the on-chain index; the company keeps control of its project keys and data. ### The pieces DataPrism is a monorepo of independent packages, each with a single responsibility. | Package | Role | | -------------------------------------------------------- | -------------------------------------------------------------------------------------- | | [`dataprism-protocol`](/protocol/overview) | Solidity smart contracts: the credit token (DPC), prisms, meta-transactions, exchange. | | [`dataprism-sdk`](/sdk/overview) | TypeScript SDK: encryption, managed storage, and on-chain calls. | | [`dataprism-api`](/api/overview) | Serverless REST API: storage authorization and the relayer request queue. | | [`dataprism-relayer`](/relayer/overview) | Service that executes on-chain transactions on behalf of users. | | [`dataprism-payment-relayer`](/payment-relayer/overview) | Listens for Stripe payments and credits users in on-chain tokens. | | [`dataprism-frontend`](/frontend/overview) | Web dashboard: accounts, credit purchases, prism management. | ### How a file is stored The heavy data lives in managed object storage; the **prism only holds a tiny, encrypted index**. ``` file (any size) │ ① chunk (content-defined, FastCDC) │ ② convergent-encrypt each chunk → storage only ever sees ciphertext │ ③ erasure-code (Reed-Solomon) → k data + m parity shards │ ④ content-address + dedup → identical shards stored once │ ⑤ store → managed storage via signed URLs │ ⑥ build a Merkle manifest → roll the route list up to a tiny root ▼ on-chain: a ~few-hundred-byte encrypted index (constant size, any file size) ``` To read the file back, the SDK reads the on-chain root, decrypts it with your key, fetches at least a **quorum** of shards from managed storage, reconstructs each chunk with Reed-Solomon, and reassembles the original file. See [File Storage & Resilience](/concepts/file-pipeline) for the full pipeline. ### Core concepts * **Prism**: a namespaced, on-chain vault that stores the **encrypted index** of your files (the reconstruction routes), not the files themselves. Identified by a `bytes32` ID (`keccak256(owner, salt)`); a shortened `bytes16` **cloud ID** is used on-chain to save calldata. It holds a DPC balance and a set of data slots. * **DPC**: the protocol's ERC-20 credit token. It pays protocol fees (every on-chain write) and relayer fees (when a relayer fronts the gas). * **The Cloud Standard**: a convention (like Ethereum's EIPs, but for cloud) that reserves named slots in every prism: `dataprism.key`, `dataprism.encryption`, `dataprism.metadata`, and `dataprism.providers`. See [Prisms & the Cloud Standard](/concepts/prisms). * **Meta-transaction**: users sign an EIP-712 message off-chain; a relayer submits it for them, so the user never needs to hold gas. * **Writer**: an address granted delegated write access to a prism, optionally with an expiration. ### Where to start * New to the project? Read the [Architecture](/architecture) overview, then the [Concepts](/concepts/prisms). * Building an integration? Start with the [SDK Overview](/sdk/overview). * Working on the chain? See the [Protocol contracts](/protocol/contracts) and [Deployments](/protocol/deployments). * Running infrastructure? See the [Relayer](/relayer/overview) and [Payment Relayer](/payment-relayer/overview). ## License The DataPrism smart contracts are released under the **Business Source License 1.1** (BUSL-1.1). | Parameter | Value | | -------------- | ----------------------------- | | Licensor | DataPrism Ltd | | Licensed Work | The DataPrism Smart Contracts | | Change Date | 2030-05-11 | | Change License | GPL-3.0-or-later | **Additional Use Grant:** Permission is granted to use, copy, and modify the Licensed Work solely for the purpose of integrating with the official deployments of the DataPrism Protocol, or for deploying on test networks (Testnets) for testing and development purposes. Deploying the Licensed Work on any main network (Mainnet) for commercial purposes or to operate a competing protocol is strictly prohibited prior to the Change Date. The Licensor grants the right to copy, modify, create derivative works, redistribute, and make non-production use of the Licensed Work. Effective on the Change Date (or the fourth anniversary of the first publicly available distribution of a given version, whichever comes first), the rights under the Change License apply. The full license text is available in the `LICENSE` file of the `dataprism-protocol` repository. ## Cryptography The crypto module is a **hybrid integrated-encryption scheme** (ephemeral key → ECDH → HKDF-SHA256 → AEAD), built directly on `@noble/curves`, `@noble/ciphers`, and `@noble/hashes`. It is the basis for sealing every encrypted prism slot. The central type is `DataPrismKey`. ### `DataPrismKey` The cryptographic identity of a prism. **The private key is never stored**: it is derived from a wallet signature and reconstructed on demand. #### Constructors (static) | Method | Description | | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | | `DataPrismKey.fromSignature(signature, config)` | derives the secret from a wallet signature, `keccak256(signature)`, reduced to a valid scalar. Deterministic. | | `DataPrismKey.fromSecret(secret, config)` | from a raw 32-byte secret (reduced to a valid scalar) | | `DataPrismKey.fromPublicKey(publicKey, config)` | a **public-only** key: can encrypt, cannot open | ```ts import { DataPrismKey } from "@dataprism/sdk"; const sig = await walletClient.signMessage({ message: `dataprism:key:${cloudId}` }); const key = DataPrismKey.fromSignature(sig, { curve: "secp256k1", algorithm: "aes-256-gcm" }); ``` Because the derivation is deterministic, signing the same message always produces the same key. Derive the message from the **`cloudId`** (default), it is public and reproducible from the prism's id, so nothing per-prism is stored. For extra security, mix in a **password** (`` `dataprism:key:${cloudId}:${password}` ``): without it, anyone who steals the wallet's private key could reproduce the signature and decrypt; with it, decryption also requires the password, which is never stored. #### Properties | Property | Type | Note | | -------------- | ------------------ | ----------------------------- | | `config` | `EncryptionConfig` | curve + algorithm | | `curve` | `Curve` | | | `canOpen` | `boolean` | `false` for a public-only key | | `publicKeyHex` | `Hex` | | | `secretHex` | `Hex` | throws if public-only | #### Methods | Method | Returns | Note | | ----------------------------------- | ------------ | ------------------------------------------------------------------ | | `seal(data, aad?)` | `Uint8Array` | encrypt for this prism | | `open(blob, aad?)` | `Uint8Array` | decrypt (requires the secret) | | `sealJson(value, aad?)` | `Hex` | JSON → sealed hex | | `openJson(blob, aad?)` | `T` | sealed hex → JSON | | `toKeySlot(includePrivate = false)` | `KeySlot` | `{ public, private }`; `private` empty unless explicit public mode | ```ts const sealed = key.sealJson({ secret: "value" }); // Hex, safe to put on-chain const data = key.openJson<{ secret: string }>(sealed); ``` `toKeySlot(true)` is the explicit, opt-in "public mode": it writes the private key into the slot so **anyone** can decrypt the prism's data. The default (`false`) keeps `private` empty. ### Low-level functions ```ts seal(config: EncryptionConfig, recipientPublicKey: Uint8Array, data: Uint8Array, aad?): Uint8Array open(config: EncryptionConfig, recipientSecret: Uint8Array, blob: Uint8Array, aad?): Uint8Array ``` The blob layout is `ephemeralKey ‖ nonce ‖ tag ‖ ciphertext`. Because it is AEAD, any tampering with a sealed payload is detected, `open` fails rather than returning corrupt data. ### Configuration ```ts type Curve = "secp256k1" | "x25519" | "ed25519"; type Algorithm = "aes-256-gcm" | "xchacha20"; interface EncryptionConfig { curve: Curve; algorithm: Algorithm } const DEFAULT_ENCRYPTION = { curve: "secp256k1", algorithm: "aes-256-gcm" }; assertEncryptionConfig(config): void // validates, or throws ``` All six curve × algorithm combinations are supported and round-trip tested. The chosen config is stored (in clear) in the prism's [`dataprism.encryption`](/concepts/prisms#dataprismencryption) slot so any reader knows how to decrypt the rest. ### Access control A **public-only** key (`fromPublicKey`) can `seal` / write but cannot `open`, attempting to decrypt a sealed slot throws. This is the access boundary used by the [`CloudReader`](/sdk/reading#decrypted-reads-cloudreader): give a collaborator the public key and they can write data for the prism, but only the secret holder can read the sealed slots. ## Execution Layer Every on-chain **write** goes through the execution layer, a single interface, `ExecutionLayer`, with interchangeable implementations. The same application code works whether the user pays gas (wallet) or a relayer fronts it (API). ```ts const r = await client.execution.writeData({ cloudId, dataId, payload }); await r.wait(); // identical on both layers ``` ### The two built-in layers | Layer | Who signs / pays gas | Contract function called | | ------------------------------ | ------------------------------------------------------------------- | -------------------------------- | | **Wallet** (`WalletExecution`) | the wallet (viem), user pays ETH | direct `DataPrismCloud` call | | **API** (`ApiExecution`) | EIP-712 request → `dataprism-api`; relayer pays (reimbursed in DPC) | `*For` meta-tx via the Forwarder | The client picks one from `options.mode` (`"wallet"` default | `"api"`), or you inject a custom layer via `options.execution`. ```ts import { walletExecution, apiExecution } from "@dataprism/sdk"; const wallet = walletExecution(network, walletClient); const api = apiExecution(network, walletClient, apiUrl); ``` ### The `ExecutionLayer` interface ```ts interface ExecutionLayer { readonly mode: ExecutionMode; // "wallet" | "api" | (custom string) readonly account: Address; // signer address createPrism(input): Promise; createPrismWithData(input): Promise; transferPrism(input): Promise; acceptTransfer(input): Promise; deletePrism(input): Promise; writeData(input): Promise; modifyData(input): Promise; deleteData(input): Promise; batchWriteData(input): Promise; batchModifyData(input): Promise; batchDeleteData(input): Promise; } ``` ### Inputs Every input extends `MetaOptions = { relayerFee?: bigint; timestamp?: bigint }`. These are **ignored in wallet mode**; in API mode they default to `timestamp = now`, `relayerFee = 0n`. | Input | Own fields | | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `CreatePrismInput` | `fundAmount: bigint`, `salt: Bytes32`, `relayerFundAmount?: bigint` | | `CreatePrismWithDataInput` | `fundAmount: bigint`, `salt: Bytes32`, `dataIds: Bytes32[]`, `payloads: Payload[]`, `relayerFundAmount?: bigint` | | `TransferPrismInput` | `cloudId`, `newOwner` | | `AcceptTransferInput` / `DeletePrismInput` | `cloudId` | | `WriteDataInput` / `ModifyDataInput` | `cloudId`, `dataId`, `payload` | | `DeleteDataInput` | `cloudId`, `dataId` | | `BatchWriteDataInput` / `BatchModifyDataInput` | `cloudId`, `dataIds[]`, `payloads[]` | | `BatchDeleteDataInput` | `cloudId`, `dataIds[]` | ### `ExecutionResult` & `.wait()` ```ts interface ExecutionResult { mode: ExecutionMode; txHash?: Hex; // wallet: available immediately requestId?: string; // api: available immediately wait(options?: { intervalMs?; timeoutMs? }): Promise; } ``` * **Wallet**: `txHash` is available right away; `wait()` awaits the transaction receipt. * **API**: `requestId` is available right away; `wait()` polls `dataprism-api` until the relayer succeeds, then returns the `txHash`. Only the API layer honors `intervalMs`. * **`createPrism`** returns a `CreatePrismExecutionResult` with a `cloudId`, immediate in wallet mode, available after `wait()` in API mode (computed locally via `computeCloudId`). Its `wait()` resolves to `CreatePrismReceipt { txHash, cloudId }`. ```ts const created = await client.execution.createPrism({ fundAmount: 10n, salt }); const { cloudId } = await created.wait(); ``` * **`createPrismWithData`** creates a prism **and** writes an initial batch of entries in a **single atomic transaction** (one signature, one nonce increment), equivalent to `createPrism` then `batchWriteData`. It also returns a `CreatePrismExecutionResult`; its `wait()` resolves `{ cloudId, txHash }`. Here `fundAmount` must cover the creation fee **plus** the per-entry data fees. This is what backs `PrismManager.create()`, which provisions the prism and writes the standard slots (key, encryption, metadata, providers) in one transaction, plus any extra `initialData` entries. ```ts const created = await client.execution.createPrismWithData({ fundAmount: 20n, salt, dataIds, payloads, }); const { cloudId, txHash } = await created.wait(); ``` > When `fundAmount > 0` the SDK signs an EIP-2612 `permit` on the DPC token and calls > `createPrismWithPermit` (wallet), funding **and** creation in a single transaction, with no > separate `approve`. In API mode the permit is sent as a `PermitParams` for the Forwarder's > `createPrismFor`. See > [single-transaction funding](/concepts/credits#single-transaction-funding-eip-2612-permit). ### Fees | Fee | Unit | Paid by | When | | ---------------- | ---- | ----------------------- | ----------------------------- | | **Gas** | ETH | the user | wallet mode | | **`relayerFee`** | DPC | prism balance → relayer | api mode | | **Protocol fee** | DPC | prism balance | every write / modify / delete | > ⚠️ In API mode the relayer fronts the gas, so you must set a `relayerFee` (in DPC) high > enough to compensate it. In wallet mode the user pays gas in ETH and `relayerFee` is ignored. ### Custom layers `ExecutionLayer` is the call standard. For a custom path (a different relayer, a multisig, specific constraints…), extend the abstract **`BaseExecutionLayer`**: it already implements all nine write methods by normalizing each into an `ExecutionOperation` (a discriminated union) and delegating to a single method you provide. ```ts class MyExecution extends BaseExecutionLayer { readonly mode = "my-custom-mode"; // ExecutionMode accepts any string get account() { return this.signerAddress; } async createPrism(input) { /* … */ } protected async execute(op /* ExecutionOperation */) { switch (op.kind) { case "writeData": /* … */ break; case "batchWriteData": /* … */ break; // transferPrism, acceptTransfer, deletePrism, modifyData, deleteData, batch* } } } const client = new DataPrismClient(network, { execution: new MyExecution() }); ``` The `ExecutionOperation` kinds are: `transferPrism`, `acceptTransfer`, `deletePrism`, `writeData`, `modifyData`, `deleteData`, `batchWriteData`, `batchModifyData`, `batchDeleteData` (`createPrism` is implemented directly, not via `execute`). ### Signing (EIP-712) The API path signs meta-transactions with `DataPrismSigner` (exposed as `client.signer`). Domain: `{ name: "DataPrismCloud", version: "1", chainId, verifyingContract: }`. The **nonce** is read on-chain (`nonces(account)`) on every signature. | Method | Type signed | | -------------------------------------------------------------------- | -------------------------------------------------- | | `signCreatePrism` | `CreatePrism` | | `signWriteData` / `signModifyData` / `signDeleteData` | `WriteData` / `ModifyData` / `DeleteData` | | `signTransferPrism` / `signAcceptTransfer` / `signDeletePrism` | `TransferPrism` / `AcceptTransfer` / `DeletePrism` | | `signBatchWriteData` / `signBatchModifyData` / `signBatchDeleteData` | batch variants (pre-hashed) | All return `EIP712Signature = { v, r, s }`. Batch types sign over pre-computed `dataIdsHash`/`payloadsHash`; the SDK computes these for you via the exported helpers: ```ts hashBytes32Array(values: Bytes32[]): Bytes32 // keccak256(concat(values)) hashPayloadsArray(payloads: Payload[]): Bytes32 // keccak256(concat(payloads.map(keccak256))) ``` ### Low-level writers (advanced) The building blocks under the execution layer, exposed for advanced use and reachable via `.raw` on each layer: * **`NetworkWriter`** (wallet), each write is `simulateContract` (dry-run) then `writeContract`. Beyond the interface it also offers wallet-only extras: `fundPrism`, `addWriter`, `updateWriterExpiration`, `removeWriter`, `claimVouchers`. Helper: `createNetworkWriterFromPrivateKey(network, privateKey)`. * **`ApiWriter`** (api), signs EIP-712 and POSTs to `/requests`, returning an `ApiRequest` (`requestId`, `status`). Also: `getRequest`, `listRequests`, `waitForRequest`, `batchMultiUserWriteData(items)`, and `extractCloudId(req)`. Statuses: `pending → processing → success | failed`. ## File Pipeline The file pipeline is the SDK's implementation of [File Storage & Resilience](/concepts/file-pipeline). It combines content-defined chunking, convergent encryption, Reed-Solomon erasure coding, content-addressing (dedup), and a Merkle manifest (a compact on-chain index). The orchestrator is `FilePipeline`. ### Chunkers (pluggable) ```ts interface Chunker { cut(data: Uint8Array): { offset: number; length: number }[] } new FastCDCChunker({ avg, min?, max? }) // content-defined (default), variable sizes new FixedChunker(size) // fixed size (fallback) ``` FastCDC uses a Gear rolling hash with normalized chunking, so inserting bytes shifts only **one** chunk, robust deduplication across file versions. Because chunk sizes vary, the index records each chunk's length. ### Convergent encryption ```ts contentHash(bytes): Hex // 0x + sha256 convergentSeal(algorithm, plaintext): { hash, ciphertext } // key = KDF(hash), deterministic convergentOpen(algorithm, hash, ciphertext): Uint8Array ``` Same content ⇒ same ciphertext ⇒ deduplication works **and** shards are encrypted at rest. The algorithm is taken from the prism's `dataprism.encryption` config. ### `ReedSolomon` Systematic erasure coding over GF(2⁸) (Vandermonde, MDS). Any `dataShards` of the `dataShards + parityShards` total reconstruct the data. ```ts const rs = new ReedSolomon(dataShards, parityShards); const all = rs.encode(dataShards); // k → n = k + m const k = rs.reconstruct(shardsWithNullsForLost); // n (with nulls) → k ``` ### `FilePipeline` ```ts new FilePipeline({ providers: StorageProvider[], // ManagedStorageProvider in browser applications standard: CloudStandard, // holds the key (algorithm + sealing) execution?: ExecutionLayer, // to write the index on-chain cloudId?: CloudId, source?: SlotSource, // to detect "file already exists" manifestThreshold?: number, // default 8192 bytes }) ``` #### Upload ```ts upload(name, data, options): Promise ``` `UploadOptions = { chunkSize, dataShards, parityShards, meta?, overwrite?, chunker? }`. The pipeline: chunk → convergent-encrypt → Reed-Solomon → address each shard by `contentHash(shard)`, **uploaded only if missing** (dedup) → disperse over `providers[pos % n]` → build a `ChunkDesc` list → roll it up into a Merkle tree (`putBlob`) while it exceeds `manifestThreshold` → produce a compact `BlobRef` root → seal the `FileIndex` → write it on-chain (if `execution` + `cloudId` were given). Up-front validation: the name must be non-empty, must not hash to a reserved slot, and the file must not already exist (unless `overwrite: true`, which routes to `modifyData`). It warns if `providers.length < dataShards + parityShards` (shards would share a provider, reducing real fault tolerance). ```ts const providers = [new ManagedStorageProvider({ apiUrl: "https://api.dataprism.co", projectId: cloudId, getAccessToken, })]; const pipeline = new FilePipeline({ providers, standard: std, execution: client.execution, cloudId, source: client.reader, }); const { index, slot, write } = await pipeline.upload("report.pdf", bytes, { chunkSize: 65536, dataShards: 1, parityShards: 0, }); await write?.wait(); // index committed on-chain ``` `UploadResult = { index: FileIndex; slot: SlotWrite; write?: ExecutionResult }`. If you omit `execution`/`cloudId`, `upload` returns the `index` and `slot` without writing, you can persist them yourself. #### Download ```ts download(index, options?): Promise // DownloadOptions = { providers? } downloadFromPayload(payload, options?): Promise ``` Descends the Merkle tree (`getBlob`): reconstructs manifests down to the data chunks → fetches shards in parallel (outages → `null`) → Reed-Solomon reconstruct → convergent-decrypt → reassemble. As long as a **quorum** of shards per chunk is reachable, the file is recovered. ```ts const view = std.reader(client.reader, cloudId); const index = await view.file("report.pdf"); // read + decrypt the index const bytes = await pipeline.download(index); // gather ≥ quorum → reconstruct ``` `downloadFromPayload(payload)` is `standard.readFile(payload)` followed by `download`. > **Limit:** safe deletion (reference counting / garbage collection) is not yet implemented, so > an `overwrite` or delete can leave orphan shards behind. ## Network & Client The SDK is built around two top-level objects: a **network** (which chain, which RPC, which contract addresses) and the **client** facade that hangs everything off it. ### `DataPrismNetwork` Encapsulates **where** you talk. It builds a viem `PublicClient` (with automatic fallback if you pass several RPC URLs) and a viem `Chain`. ```ts new DataPrismNetwork(definition: NetworkDefinition, rpcUrls: string | string[]) ``` ```ts const network = new DataPrismNetwork(DEMO, "https://rpc.example.com"); // or with fallback RPCs: const network = new DataPrismNetwork(DEMO, ["https://rpc-a", "https://rpc-b"]); ``` | Member | Type | Description | | -------------- | ------------------- | ------------------------------------------------------------- | | `definition` | `NetworkDefinition` | the preset it was built from | | `chain` | `Chain` | the concrete viem chain (handy for building a `walletClient`) | | `publicClient` | `PublicClient` | a ready-to-read viem client | | `chainId` | `number` | the chain id | | `name` | `string` | the chain name (sent as `chainName` to the API) | | `addresses` | `NetworkAddresses` | the contract addresses | | `rpcUrls` | `readonly string[]` | the RPC URLs provided | #### `NetworkDefinition` & `NetworkAddresses` ```ts interface NetworkAddresses { readonly dataPrismCloud: Address; readonly dataPrismCloudForwarder: Address; readonly token: Address; readonly feeController: Address; } interface NetworkDefinition { readonly chainId: number; readonly name: string; readonly addresses: NetworkAddresses; } ``` > The SDK references **only** `DataPrismCloud` and the `Forwarder` (plus `token` / > `feeController` for context). It never touches the core `DataPrism` contract directly, even > slot reads go through `DataPrismCloud`'s read-through getters. #### Presets * **`DEMO`**: `chainId 11155111` (Sepolia), `name: "demo"`, bundling the four protocol contract addresses. It is deliberately surfaced to integrators as `"demo"`, **not** `"sepolia"`, that string is the `chainName` the API and relayer filter on. * **`NETWORKS`**: `{ DEMO }`, with `NetworkName = keyof typeof NETWORKS`. ```ts import { DEMO, NETWORKS } from "@dataprism/sdk"; ``` For a different chain, pass your own `NetworkDefinition` (custom `chainId`, `name`, and addresses). For the canonical deployed addresses, see [Deployments](/protocol/deployments). ### `computeCloudId` Derives a prism's `cloudId` **offline**, deterministically, the same formula the contract uses. This is what lets the API (gasless) path know a new prism's ID immediately, without waiting for the relayer. ```ts computeCloudId(cloudAddress: Address, salt: Bytes32): CloudId // = slice(keccak256(concat([cloudAddress, salt])), 0, 16) ``` ### `DataPrismClient` A **lazy** assembler: it builds each sub-client on first access. ```ts new DataPrismClient(network: DataPrismNetwork, options?: DataPrismClientOptions) ``` #### `DataPrismClientOptions` | Field | Type | Role | | --------------- | ------------------- | ---------------------------------------------------- | | `walletClient?` | `WalletClient` | signs / pays gas (required to write or sign) | | `apiUrl?` | `string` | URL of `dataprism-api` (required when `mode: "api"`) | | `mode?` | `"wallet" \| "api"` | which execution layer to expose (default `"wallet"`) | | `execution?` | `ExecutionLayer` | inject an already-built (e.g. custom) layer | #### Getters & methods | Member | Type | Needs a wallet | | --------------------------- | -------------------------------------------- | -------------- | | `network` | `DataPrismNetwork` | - | | `reader` | [`NetworkReader`](/sdk/reading) | no | | `execution` | [`ExecutionLayer`](/sdk/execution) | yes | | `signer` | `DataPrismSigner` | yes | | `prism(standard, cloudId?)` | [`PrismManager`](/sdk/standard#prismmanager) | yes | Accessing `.execution` / `.signer` without a `walletClient` throws an explicit error; so does `.execution` in `api` mode without an `apiUrl`. `.reader` works with no wallet at all, useful for read-only integrations. ```ts // read-only const client = new DataPrismClient(network); const tokens = await client.reader.getPrismTokens(cloudId); // writes (wallet mode) const client = new DataPrismClient(network, { walletClient }); await (await client.execution.writeData({ cloudId, dataId, payload })).wait(); ``` ## SDK Overview `@dataprism/sdk` is the **client-side core** of DataPrism, in TypeScript, embeddable anywhere (browser, Node backend, or a script). It exposes a single API for three jobs: * **Drive the blockchain**: create prisms, read/write slots, whether or not the user pays gas. * **Encrypt / decrypt**: keys derived from a wallet signature, never stored. * **Store files**: chunking, encryption, content addressing, and managed object storage. It is ESM, strict TypeScript, and pulls in **no** heavy cloud SDK. Dependencies: `viem` (peer, ≥ 2.0), `@noble/curves`, `@noble/ciphers`, `@noble/hashes`. ### Install ```bash npm install @dataprism/sdk viem ``` ### Quick start ```ts import { DataPrismClient, DataPrismNetwork, DEMO } from "@dataprism/sdk"; const network = new DataPrismNetwork(DEMO, "https://rpc.example.com"); // wallet mode (default) - the user pays gas const client = new DataPrismClient(network, { walletClient }); // api mode - the relayer fronts the gas (reimbursed in DPC) const client = new DataPrismClient(network, { walletClient, apiUrl, mode: "api" }); // read-only - no wallet const client = new DataPrismClient(network); ``` The entry point is `DataPrismClient`, a lazy facade that builds its sub-clients on first access: | Getter | Type | Needs a wallet | | ---------------------- | -------------------------------------------- | -------------- | | `reader` | [`NetworkReader`](/sdk/reading) | no | | `execution` | [`ExecutionLayer`](/sdk/execution) | yes | | `signer` | `DataPrismSigner` | yes | | `prism(std, cloudId?)` | [`PrismManager`](/sdk/standard#prismmanager) | yes | ### Layered architecture ``` YOUR APPLICATION │ ┌───────────────────▼───────────────────────────────────────┐ │ STANDARD CloudStandard / CloudReader - prism slots │ standard/ ├────────────────────────────────────────────────────────────┤ │ CRYPTO DataPrismKey - keys + seal/open │ crypto/ ├────────────────────────────────────────────────────────────┤ │ FILES FilePipeline + ReedSolomon │ files/ │ STORAGE StorageProvider (Managed, Memory) │ storage/ ├────────────────────────────────────────────────────────────┤ │ EXECUTION ExecutionLayer (wallet | api) - writes │ execution/ │ READING NetworkReader SIGNING DataPrismSigner │ network/, signing/ ├────────────────────────────────────────────────────────────┤ │ NETWORK DataPrismNetwork (chainId, RPC, addresses) │ network/ └────────────────────────────────────────────────────────────┘ │ DataPrismCloud.sol · DataPrismCloudForwarder.sol (on-chain) │ DataPrism-managed object storage ← encrypted file shards ``` **Golden rule:** each layer only knows the one directly beneath it. ### Importable modules (subpaths) The SDK is published wagmi/viem-style: everything is re-exported from the root `@dataprism/sdk`, and each module is also importable on its own subpath. ```ts import { DataPrismClient } from "@dataprism/sdk"; // facade import { DataPrismNetwork, DEMO } from "@dataprism/sdk/network"; import { walletExecution } from "@dataprism/sdk/execution"; import { DataPrismSigner } from "@dataprism/sdk/signing"; import { ApiWriter } from "@dataprism/sdk/api"; import { DataPrismKey } from "@dataprism/sdk/crypto"; import { CloudStandard, CloudReader } from "@dataprism/sdk/standard"; import { PrismManager } from "@dataprism/sdk/prism"; import { ManagedStorageProvider } from "@dataprism/sdk/storage"; import { FilePipeline } from "@dataprism/sdk/files"; import { DataPrismCloudAbi } from "@dataprism/sdk/abi"; ``` | Subpath | Role | Main exports | Docs | | ------------ | ------------------------------------------------------------ | ------------------------------------------------------------------------------ | --------------------------------------------------------- | | `/network` | the "where" (chain, RPC, addresses) + low-level reads/writes | `DataPrismNetwork`, `DEMO`, `NetworkReader`, `NetworkWriter`, `computeCloudId` | [Network & Client](/sdk/network), [Reading](/sdk/reading) | | `/execution` | unified write layer | `ExecutionLayer`, `BaseExecutionLayer`, `WalletExecution`, `ApiExecution` | [Execution Layer](/sdk/execution) | | `/signing` | EIP-712 signatures | `DataPrismSigner`, `hashBytes32Array`, `hashPayloadsArray` | [Execution Layer](/sdk/execution) | | `/api` | `dataprism-api` REST client | `ApiWriter`, `extractCloudId` | [Execution Layer](/sdk/execution) | | `/crypto` | prism keys + encryption | `DataPrismKey`, `seal`, `open` | [Cryptography](/sdk/crypto) | | `/standard` | prism slots (codec + decrypting reads) | `CloudStandard`, `CloudReader`, `SLOT_IDS` | [Cloud Standard](/sdk/standard) | | `/prism` | high-level prism lifecycle | `PrismManager` | [Cloud Standard](/sdk/standard#prismmanager) | | `/storage` | storage connectors | `StorageProvider`, `ManagedStorageProvider`, `MemoryProvider` | [Storage Providers](/sdk/storage) | | `/files` | file pipeline | `FilePipeline`, `ReedSolomon`, `FastCDCChunker`, `convergentSeal` | [File Pipeline](/sdk/files) | | `/abi` | contract ABIs | `DataPrismCloudAbi`, `DataPrismCloudForwarderAbi` | - | ### End-to-end at a glance ```ts // 1. key derived from a wallet signature (message derived from the cloudId; add `:${password}` for extra security) const sig = await walletClient.signMessage({ message: `dataprism:key:${cloudId}` }); const key = DataPrismKey.fromSignature(sig, { curve: "secp256k1", algorithm: "aes-256-gcm" }); const std = new CloudStandard(key); // 2. create a prism + initialize its 4 reserved slots const manager = client.prism(std); const { cloudId } = await manager.create({ fundAmount, salt, metadata, providers }); // 3. upload a file to managed storage, index written on-chain const storage = [new ManagedStorageProvider({ apiUrl: "https://api.dataprism.co", projectId: cloudId, getAccessToken, })]; const pipeline = new FilePipeline({ providers: storage, standard: std, execution: client.execution, cloudId }); await pipeline.upload("report.pdf", bytes, { chunkSize: 65536, dataShards: 1, parityShards: 0 }); // 4. read it back const view = std.reader(client.reader, cloudId); const index = await view.file("report.pdf"); const bytes = await pipeline.download(index); ``` Each step is detailed in the pages that follow. ## Reading Data All reads go through one or more **RPC** endpoints via `client.reader` (a `NetworkReader`). They consume **no gas and no credits**. Reads target `DataPrismCloud` and the `Forwarder` only, never the core `DataPrism` contract. ### `NetworkReader` Build it via `client.reader`, or directly with `new NetworkReader(network)`. #### Contract metadata `getOwner()`, `getPendingOwner()`, `getForwardContract()`, `getDomainSeparator()`, `getNonce(account)`. #### Prism state | Method | Returns | | ------------------------------- | ---------------------------------------------------- | | `getPrismOwner(cloudId)` | `Address` | | `getPrismPendingOwner(cloudId)` | `Address` | | `getPrismTokens(cloudId)` | `bigint` (DPC balance) | | `getProtocolId(cloudId)` | `Bytes32` (the core prism ID) | | `getPrismInfo(cloudId)` | `PrismInfo`, all of the above in one call (parallel) | #### Slot payloads Raw (still-encrypted) slot values, read through `DataPrismCloud`: | Method | Returns | Note | | --------------------------------------- | ------------------------ | -------------------- | | `getData(cloudId, dataId)` | `Payload` | raw payload | | `getDataWithTimestamp(cloudId, dataId)` | `{ payload, timestamp }` | + on-chain timestamp | | `getDataFrom(cloudId, dataIds[])` | `Payload[]` | several in one call | | `dataExists(cloudId, dataId)` | `boolean` | | > To read **and decrypt** a slot in one step, use the [`CloudReader`](#decrypted-reads-cloudreader) > instead of raw `getData`. #### Forwarder / relayer `getForwarderOwner()`, `getForwarderPendingOwner()`, `getCloudAddress()`, `isOfficialRelayer(relayer)`, `getProtocolTreasury()`, `getPriorityWindow()`, `getRelayerShareBps()`, `getVouchers(relayer)`. ### Paginated enumerations For lists whose size is **unpredictable** (a prism can hold millions of files), never load everything at once. Each list comes in three forms: load-all (convenience), by-slice, and streaming iterator. | List | load all | by slice | streaming | | ------------- | ---------------------- | ------------------------------------------ | -------------------------------------- | | user's prisms | `getUserPrisms(user)` | `getUserPrismsSlice(user, offset, limit)` | `iterateUserPrisms(user, sliceSize?)` | | data keys | `getDataKeys(cloudId)` | `getDataKeysSlice(cloudId, offset, limit)` | `iterateDataKeys(cloudId, sliceSize?)` | | writers | `getWriters(cloudId)` | `getWritersSlice(cloudId, offset, limit)` | `iterateWriters(cloudId, sliceSize?)` | The raw `*Count` / `*ByIndex` getters (e.g. `getDataCount`, `getDataKeyByIndex`) are also available. A slice is: ```ts interface Slice { items: T[]; offset: number; limit: number; total: bigint; hasMore: boolean } ``` The iterators (`iterate*`) are **async generators** that walk the whole list across several requests (default `sliceSize = 200`), reading each window in parallel, without holding it all in memory. ```ts for await (const dataId of client.reader.iterateDataKeys(cloudId)) { // process one key at a time } ``` ### Decrypted reads (`CloudReader`) The `NetworkReader` returns ciphertext. To read a slot **and decrypt it** in one step, pair a [`CloudStandard`](/sdk/standard) (which holds the key) with a `SlotSource` and a cloudId. Build it via `std.reader(source, cloudId)`, `source` is anything with a `getData(cloudId, dataId)` method, and `NetworkReader` satisfies it. ```ts const view = std.reader(client.reader, cloudId); const keySlot = await view.key(); // clear const enc = await view.encryption(); // clear const metadata = await view.metadata(); // decrypted const providers = await view.providers(); // decrypted const fileIndex = await view.file("report.pdf"); // decrypted const ciphertext = await view.raw(dataId); // raw payload ``` The providers slot contains only the public `dataprism_managed` service marker. Storage access is authorized separately with the current user session, so this slot contains no credentials. | Method | Returns | Encrypted? | | -------------- | ------------------ | -------------- | | `key()` | `KeySlot` | clear | | `encryption()` | `EncryptionConfig` | clear | | `metadata()` | `MetadataSlot` | decrypted | | `providers()` | `ProvidersSlot` | decrypted | | `file(name)` | `FileIndex` | decrypted | | `raw(dataId)` | `Hex` | raw ciphertext | A **public-only** key reads the clear slots fine but **throws** on the sealed ones, this is the access-control boundary. The `SlotSource` interface also accepts optional `getDataFrom` and `dataExists` methods for richer sources. ## Cloud Standard The standard module turns the [Cloud Standard slots](/concepts/prisms#the-cloud-standard) into a typed codec. `CloudStandard` encodes and seals each slot; `CloudReader` reads and decrypts them ([see Reading](/sdk/reading#decrypted-reads-cloudreader)); `PrismManager` orchestrates the high-level prism lifecycle. ### Slot identifiers ```ts SLOT_NAMES = { key: "dataprism.key", encryption: "dataprism.encryption", metadata: "dataprism.metadata", providers: "dataprism.providers", } SLOT_IDS = { key, encryption, metadata, providers } // keccak256 of each name dataId(name: string): DataId // keccak256(stringToBytes(name)) fileDataId(filename: string): DataId // a file slot's dataId isReservedSlot(id: DataId): boolean // is it one of the dataprism.* slots? ``` ### `CloudStandard` A codec bound to a [`DataPrismKey`](/sdk/crypto). `new CloudStandard(key)`. | Slot | build | read | Encrypted? | | ---------------------- | -------------------------- | ------------------------- | ---------- | | `dataprism.key` | `keySlot(includePrivate?)` | `readKey(payload)` | **clear** | | `dataprism.encryption` | `encryptionSlot()` | `readEncryption(payload)` | **clear** | | `dataprism.metadata` | `metadataSlot(meta)` | `readMetadata(payload)` | **sealed** | | `dataprism.providers` | `providersSlot(providers)` | `readProviders(payload)` | **sealed** | | file | `fileSlot(index)` | `readFile(payload)` | **sealed** | Each `build` method returns a `SlotWrite = { dataId, payload }`, ready to hand to the [execution layer](/sdk/execution). The clear slots are `toHex(JSON)`; the sealed slots go through `key.sealJson`. The `read*` methods throw a clear "slot empty / does not exist" error on a `0x` payload instead of a cryptic crypto failure. Other members: * `initialSlots({ metadata?, providers?, includePrivate? }) → SlotWrite[]`: the provisioning slots. Always includes `key` + `encryption`; adds `metadata` / `providers` only if provided. * `fileId(filename) → DataId` * `reader(source, cloudId) → CloudReader`: build a decrypting reader. ```ts const std = new CloudStandard(key); const slots = std.initialSlots({ metadata, providers }); await (await client.execution.batchWriteData({ cloudId, dataIds: slots.map(s => s.dataId), payloads: slots.map(s => s.payload), })).wait(); ``` ### Slot schemas ```ts interface MetadataSlot { name: string; description: string; addresses: Record; // owner / writer addresses → readable name } interface ProvidersSlot { dataprism_managed?: { service: "dataprism" }; } interface Quorum { dataShards: number; parityShards: number } interface ChunkDesc { hash: Hex; len: number; shards: Hex[] } // shards = content hashes interface BlobRef { level: number; size: number; chunks: ChunkDesc[] } interface FileIndex { name: string; size: number; chunkSize: number; quorum: Quorum; providers: string[]; // ordered provider-ID list, frozen at upload root: BlobRef; // compact Merkle root (≈ a few hundred bytes) meta?: Record; } interface SlotWrite { dataId: Hex; payload: Hex } ``` See [File Storage & Resilience](/concepts/file-pipeline#the-index-structure) for how `FileIndex` / `BlobRef` / `ChunkDesc` describe a file. ### PrismManager High-level lifecycle orchestration, on top of the execution layer + the standard. Build via `client.prism(standard, cloudId?)` or `new PrismManager({ standard, execution, source?, cloudId? })`. #### Two-step creation ```ts create(args: CreatePrismArgs): Promise ``` `CreatePrismArgs = { fundAmount, salt, metadata?, providers?, includePrivate?, relayerFee?, timestamp? }`. 1. **Step 1**: `createPrism` on-chain → `cloudId`. 2. **Step 2**: initializes **all four** reserved slots (`key`, `encryption`, `metadata`, `providers`) in one batch (empty defaults if not supplied). Both steps are awaited; returns `{ cloudId, create, init }`. ```ts const manager = client.prism(new CloudStandard(key)); const { cloudId } = await manager.create({ fundAmount, salt, metadata, providers }); ``` #### Dedicated slot editors These write reserved slots intentionally (via `modifyData`): | Method | Effect | | -------------------------------- | ---------------------------------------- | | `setMetadata(metadata, opts?)` | replace `dataprism.metadata` | | `setProviders(providers, opts?)` | replace **all** of `dataprism.providers` | `readMetadata()` and `readProviders()` require a `source` (the `NetworkReader`). ```ts await manager.setProviders({ dataprism_managed: { service: "dataprism" } }); await manager.setMetadata({ name: "Acme", description: "v2", addresses }); ``` #### Guarded generic writes ```ts writeData(dataId, payload, opts?) // throws if dataId is reserved modifyData(dataId, payload, opts?) // throws if dataId is reserved deleteData(dataId, opts?) // throws if dataId is reserved batchWriteData(dataIds, payloads, opts?) // throws if ANY dataId is reserved ``` > **Guard:** any generic write that targets a reserved slot (`dataprism.*`) throws. The same > guard protects [`FilePipeline.upload`](/sdk/files) (it rejects a filename that would hash to > a reserved slot). To change a reserved slot you **must** use its dedicated editor. ## Storage Providers Storage providers are the connectors to the object store where encrypted file shards live. Applications use `ManagedStorageProvider`: DataPrism authorizes each operation with the current user session and returns a short-lived signed URL. The SDK does not accept cloud credentials. ### The `StorageProvider` interface ```ts interface StorageProvider { readonly id: string; put(key: string, data: Uint8Array): Promise; get(key: string): Promise; // throws if absent has(key: string): Promise; // existence check (used for dedup) delete(key: string): Promise; // idempotent } ``` `has(key)` is what powers **deduplication**: before uploading a shard, the pipeline checks whether it already exists at the provider, and skips the upload if so. Managed storage uses a short-lived signed `HEAD` request. ### `ManagedStorageProvider` The managed provider is the default for browser and dashboard integrations. Supply the API URL, the bytes16 project ID, and a callback that returns the current Privy access token. ```ts const storage = new ManagedStorageProvider({ apiUrl: "https://api.dataprism.co", projectId: cloudId, getAccessToken, }); ``` The provider requests a separate URL for `PUT`, `GET`, `HEAD`, or `DELETE`. URLs expire after 60 seconds by default, are scoped on the server to the authenticated tenant and project, and cannot be changed to target another object path. If Privy rejects an expired token, the SDK asks the callback for a fresh token once. A revoked session remains denied. ### `MemoryProvider` An in-process reference implementation for tests, demos, and the browser. ```ts const provider = new MemoryProvider("dataprism_managed"); provider.offline = true; // simulate an outage (get/has/put fail) ``` `MemoryProvider` is demo/test behavior only. It is process-local and is not persistent storage. `ManagedStorageProvider` and `MemoryProvider` are the only bundled implementations. Applications can implement `StorageProvider` for tests or specialized runtime adapters, but provider secrets are not part of the DataPrism slot schema or SDK API. ## Relayer `dataprism-relayer` is a continuously running Node.js service that submits user meta-transactions on-chain. It polls the [API](/api/overview) for pending requests, checks each one for profitability, submits it to `DataPrismCloudForwarder`, and reports the result back to the API. ### Tech stack * **Language:** TypeScript `5.8` · **Runtime:** Node.js * **Key dependencies:** [viem](https://viem.sh/) `2.28` (chain interaction), [nodemailer](https://nodemailer.com/) (email alerts), [tslog](https://tslog.js.org/) (logging), [pm2](https://pm2.keymetrics.io/) (process management) ### Project structure ``` src/ ├── relayer/ │ ├── runner.ts # Entry point: init executor + notifier, start the loop │ ├── poller.ts # Main polling loop │ ├── executor.ts # Simulates and submits transactions (retries, gas bumps) │ ├── profitability.ts # Profitability check via Uniswap V4 price │ └── gas.ts # Standalone ETH balance monitor ├── api/client.ts # API client (auth + request management) ├── config.ts # Environment variable parsing ├── notification.ts # Email + Telegram notifications └── abi/DataPrismCloudForwarder.ts ``` ### Polling loop (`poller.ts`) Every `POLLING_INTERVAL` ms (default 5 s): 1. Fetch pending requests via `getPendingRequests()` (`GET /requests?status=pending`). 2. For each request: * **Atomic claim** (`PATCH /requests/{id}` → `processing`), skip if already taken. * **Profitability check**: skip if not profitable. * **Execution** via `executor.execute(req)`. * **Report** success (with `txHash`) or failure (with the error message) to the API. Errors are caught per-request, so one failed request never halts the loop. ### Profitability check (`profitability.ts`) Optional, disabled when `UNISWAP_POOL_MANAGER` is empty. It compares the `relayerFee` (in DPC) against the estimated gas cost: ``` gasCostInFeeToken = estimateGas × maxFeePerGas × ETH_price_in_DPC profitable = relayerFee ≥ gasCostInFeeToken ``` The ETH/DPC price is read from a configurable Uniswap V4 pool (`getSlot0` → `sqrtPriceX96`), via a separate price RPC endpoint (`PRICE_RPC_URL_0`). ### On-chain execution (`executor.ts`) Calls the `*For` functions on `DataPrismCloudForwarder` via viem. For each request: 1. **Simulation** (`simulateContract`), a failure throws a non-retriable `SimulationError` (the contract would revert). 2. **Submission** (`writeContract`). 3. **Wait** for `CONFIRMATIONS` confirmations (with a timeout). On a network failure (not a simulation error), the relayer **retries** with progressively higher gas (`INCREASE_GAS_PERCENT × attempt`) up to `MAX_RETRIES`, pausing if fees exceed `MAX_FEE_PER_GAS`. Multiple RPCs are supported in fallback mode (`RPC_URL_0`, `RPC_URL_1`). Supported actions: `CREATE_PRISM_FOR`, `WRITE_DATA_FOR`, `MODIFY_DATA_FOR`, `DELETE_DATA_FOR`, `TRANSFER_PRISM_FOR`, `ACCEPT_TRANSFER_FOR`, `DELETE_PRISM_FOR`, `BATCH_WRITE_DATA_FOR`, `BATCH_MODIFY_DATA_FOR`, `BATCH_DELETE_DATA_FOR`, `BATCH_MULTI_USER_WRITE_DATA`. For `CREATE_PRISM_FOR` the executor forwards the request's `permit` field as the `PermitParams` tuple `(deadline, v, r, s)` argument of `createPrismFor` (a zero tuple when the prism is unfunded), so the Forwarder can run `token.permit(...)` and fund the prism with no prior `approve`. ### Gas monitor (`gas.ts`) A separate PM2 process checks the relayer's ETH balance every `GAS_POLLING_INTERVAL` ms (default 3 min) and sends a notification when it drops below `GAS_MIN_BALANCE` (default 1 ETH). ### Configuration The relayer reads a `.env` file. The account **must be whitelisted** in the API before it can relay. ```bash # Identity & contracts RELAYER_PRIVATE_KEY="" DATAPRISM_FORWARDER_ADDRESS="" # DataPrismCloudForwarder # API API_BASE_URL="" CHAIN_NAME="" # optional, filters requests by chain # RPC (with fallback) RPC_URL_0="" RPC_URL_1="" # Polling & gas POLLING_INTERVAL="5000" MAX_FEE_PER_GAS="100000000000" # 100 Gwei MAX_PRIORITY_FEE_PER_GAS="100000000000" INCREASE_GAS_PERCENT="20" # gas bump % per retry MAX_RETRIES="50" RETRY_DELAY="5000" CONFIRMATIONS="2" CONFIRMATIONS_TIMEOUT="30000" GAS_MIN_BALANCE="1000000000000000000" # 1 ETH (triggers alert) GAS_POLLING_INTERVAL="180000" # balance check every 3 min # Profitability via Uniswap V4 (leave empty to disable) UNISWAP_POOL_MANAGER="" UNISWAP_CURRENCY0="0x0000000000000000000000000000000000000000" # ETH UNISWAP_CURRENCY1="0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48" # USDC (mainnet) UNISWAP_FEE="500" UNISWAP_TICK_SPACING="10" UNISWAP_HOOKS="0x0000000000000000000000000000000000000000" PRICE_RPC_URL_0="" # mainnet RPC for price reading PRICE_RPC_URL_1="" # Notifications (optional) SMTP_HOST="" SMTP_PORT="465" SMTP_USER="" SMTP_PASS="" SMTP_FROM="" SMTP_TO="" TELEGRAM_BOT_TOKEN="" TELEGRAM_CHAT_ID="" ``` ### Install, run, deploy ```bash npm install # Install dependencies npm run typecheck # tsc --noEmit npm run dev # Build + run runner.js directly npm run build # Compile to dist/ npm run deploy # Build + start relayer & gas monitor via PM2 npm run list # pm2 list ``` PM2 (`ecosystem.config.js`) runs two processes: **DataPrism Relayer** (`runner.js`) and **DataPrism Gas Monitor** (`gas.js`). Deploy multiple instances (each with its own key) for redundancy, the atomic claim makes this safe. ### API authentication The relayer authenticates to the API with an ECDSA signature over the request headers (`X-Relayer-Address`, `X-Relayer-Signature`, `X-Relayer-Timestamp`). See [API → Authentication](/api/authentication) for the message format. ## Contracts Reference for every contract in `dataprism-protocol`. For deployed addresses, see [Deployments](/protocol/deployments). ### Token (DPC) `Token.sol`, the protocol's ERC-20 token (`name = "DataPrism Token"`, `symbol = "DPC"`, `decimals = 18`). It is consumed (burned) on data operations to pay protocol fees. Role-based access: * **Minters**: addresses authorized to mint, with a configurable min/max range per minter. * **Burners**: addresses authorized to burn from another account (e.g. `DataPrism`). * **Operators**: addresses authorized to `transferFrom` without an allowance (e.g. `DataPrism`). Key functions: | Function | Description | | ----------------------------------------------------------- | --------------------------------------------- | | `transfer` / `approve` / `transferFrom` | Standard ERC-20 (operators bypass allowance). | | `permit(owner, spender, value, deadline, v, r, s)` | EIP-2612 gasless approval. | | `mint(amount)` / `mintTo(to, amount)` | Mint (active minters only). | | `burn(account, amount)` / `burnFrom(account, amount)` | Burn (active burners only). | | `configureMinter` / `configureBurner` / `configureOperator` | Admin role configuration. | | `transferOwnership` / `acceptOwnership` | Two-step ownership transfer. | Events: `Transfer`, `Approval`, `OwnershipTransferStarted`, `OwnershipTransferred`. ### FeeController `FeeController.sol`, centralizes pricing. Each action has a default fee in DPC; custom fees can be negotiated per address (`DataPrismCloud` pays 1000 DPC / 1 USDC to create a prism, 100 DPC / 0.1 USDC per data operation, and 0 on other prism operations). ``` getProtocolFee(caller, action) → custom fee if configured, otherwise default fee ``` Action constants are `keccak256` of the action name. Default fees at deployment: | Action | Default fee | | ---------------- | ------------ | | `CREATE_PRISM` | 1 DPC | | `FUND_PRISM` | 0 DPC | | `TRANSFER_PRISM` | 0 DPC | | `DELETE_PRISM` | 0 DPC | | `ADD_DATA` | 1 DPC | | `MODIFY_DATA` | 1 DPC | | `DELETE_DATA` | 1 DPC | | `BUY_TOKENS` | 0% (no fee) | | `SELL_TOKENS` | 1% (100 bps) | Admin functions: `setDefaultProtocolFee`, `setCustomProtocolFee`, `removeCustomProtocolFee`, `setTreasury`, plus two-step ownership. ### DataPrism `DataPrism.sol`, the raw storage layer. Prisms are identified by `bytes32` IDs. Each prism has an `owner`, a DPC `tokens` balance, and a mapping of `dataId → (bytes payload, uint256 timestamp)`. Only the owner can write, modify, or delete; tokens are deducted on every operation. **Prism lifecycle:** `createPrism`, `createPrisms`, `fundPrism`, `fundPrisms`, `transferPrism`, `transferPrisms`, `deletePrism`, `deletePrisms`. **Single data ops:** `writeData`, `modifyData`, `deleteData`. **Batch data ops:** `writeDataTo`, `modifyDataIn`, `deleteDataFrom` (one prism); `writeDataAcross`, `modifyDataAcross`, `deleteDataAcross` (multiple prisms). **Views:** `getPrism`, `getPrismOwner`, `getPrismTokens`, `prismExists`, `getData`, `getDataFrom`, `getDataAcross`, `dataExists`, `getDataTimestamp`. Events: `PrismCreated`, `PrismFunded`, `PrismTransferred`, `PrismDeleted`, `DataWritten`, `DataUpdated`, `DataDeleted`. ### DataPrismCloud `DataPrismCloud.sol`, the application layer. It maps `bytes16` cloud IDs to the underlying `bytes32` storage IDs and adds: 1. **Cloud IDs** (`bytes16`): the first 16 bytes of the prism ID; cheaper in calldata. 2. **Writers**: per-prism delegated write access with optional expiration. 3. **Two-step transfers**: `transferPrism` → `acceptTransfer`. 4. **On-chain enumeration**: prisms per user, dataIds per prism, writers per prism. 5. **Internal token balance**: DPC managed at the cloud layer. **Prism management:** `createPrism`, `createPrismWithPermit`, `createPrismWithData`, `createPrismWithDataAndPermit`, `fundPrism`, `fundPrismWithPermit`, `transferPrism`, `acceptTransfer`, `deletePrism`. The `createPrismWithData` variants create a prism **and** write an initial batch of entries (`dataIds` / `payloads`) in a **single atomic transaction**, equivalent to `createPrism` followed by `batchWriteData` but with one signature and one nonce increment; `fundAmount` must cover the creation fee plus the per-entry data fees. The `*AndPermit` form additionally bundles an [EIP-2612 `permit`](#token-dpc) so a funded, populated prism is created with no separate `approve`. The `*WithPermit` variants bundle an [EIP-2612 `permit`](#token-dpc) on the DPC token into the same call, so a **funded** prism is created (or topped up) in a **single transaction** with no separate `approve`. They run `token.permit(msg.sender, address(this), amount, deadline, v, r, s)`, the cloud is **not** a token operator, so pulling DPC needs an allowance, and then the normal `createPrism` / `fundPrism` logic. With `amount == 0` the permit is skipped. **Writer management:** `addWriter`, `updateWriterExpiration`, `removeWriter`. **Data ops:** `writeData`, `modifyData`, `deleteData`, `batchWriteData`, `batchModifyData`, `batchDeleteData`. **Meta-transaction callbacks** (only callable by the Forwarder): the `*As` variants - `createPrismAs`, `createPrismWithDataAs`, `fundPrismAs`, `transferPrismAs`, `acceptTransferAs`, `deletePrismAs`, `writeDataAs`, `modifyDataAs`, `deleteDataAs`, `batchWriteDataAs`, `batchModifyDataAs`, `batchDeleteDataAs`. **Views:** `getPrismOwner`, `getPrismPendingOwner`, `getPrismTokens`, `getProtocolId`, `getUserPrismCount`, `getUserPrismByIndex`, `getDataCount`, `getDataKeyByIndex`, `getWritersCount`, `getWriterByIndex`, `getWriterExpiration`, `DOMAIN_SEPARATOR`. The Forwarder is wired in post-deployment via `setForwardContract`. ### DataPrismCloudForwarder `DataPrismCloudForwarder.sol`, EIP-712 meta-transaction relayer. Users sign off-chain and a relayer submits on their behalf in exchange for a `relayerFee`. #### Meta-transaction flow ``` 1. User signs EIP-712 (verifyingContract = DataPrismCloud address) 2. Relayer calls Forwarder.writeDataFor(...) 3. Forwarder verifies the signature and the priority window 4. Forwarder calls DataPrismCloud.writeDataAs(...) 5. DataPrismCloud executes and transfers relayerFee to the Forwarder 6. Forwarder credits the relayer's voucher balance ``` #### Priority window (anti-frontrunning) After a request is created, only **official relayers** can submit it during the first `priorityWindow` seconds. After that window, any relayer may relay it. #### Relayer economics * **Official relayer**: receives 100% of `relayerFee`. * **Non-official relayer**: receives `relayerShareBps`% (default 7000 = 70%); the remainder goes to `protocolTreasury`. * Fees accumulate as **vouchers**, claimed via `claimVouchers()`. #### `*For` dispatch functions | Function | Description | | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `createPrismFor` | Create a prism on behalf of the signer (optional `relayerFundAmount`). Carries an EIP-2612 `PermitParams` and runs `token.permit(...)` before `createPrismAs`, so funding needs no prior `approve`. | | `createPrismWithDataFor` | Create a prism **and** write an initial batch (`dataIds` / `payloads`) on behalf of the signer, in a single atomic transaction. Carries a `PermitParams` and calls `createPrismWithDataAs`; `fundAmount` must cover the creation fee plus the per-entry data fees. | | `fundPrismFor` | Fund a prism (also bundles a `permit` for the DPC pull). | | `transferPrismFor` | Initiate a prism ownership transfer. | | `acceptTransferFor` | Accept a pending transfer. | | `deletePrismFor` | Delete a prism. | | `writeDataFor` | Write a new data entry. | | `modifyDataFor` | Update an existing data entry. | | `deleteDataFor` | Delete a data entry. | | `batchWriteDataFor` | Write multiple entries into one prism. | | `batchModifyDataFor` | Update multiple entries in one prism. | | `batchDeleteDataFor` | Delete multiple entries from one prism. | | `batchMultiUserWriteData` | Write entries for multiple users in a single transaction. | Admin/relayer management: `claimVouchers`, `setOfficialRelayer`, `setProtocolTreasury`, `setPriorityWindow`, `setRelayerShareBps`. Batch operations (`BatchWriteData`, `BatchModifyData`, `BatchDeleteData`) sign over pre-computed `dataIdsHash` and `payloadsHash`. The combined create-and-write path (`createPrismWithDataFor`) uses a dedicated typehash `CreatePrismWithData(uint256 fundAmount,bytes32 salt,bytes32 dataIdsHash,bytes32 payloadsHash,uint256 relayerFee,uint256 timestamp,uint256 nonce)`, where `dataIdsHash` and `payloadsHash` follow the same convention as `BatchWriteData`. Security: nonce replay protection, s-value malleability check, and ERC-1271 smart-contract signer support. ### Exchange `Exchange.sol`, converts external ERC-20 tokens to/from DPC, with optional yield strategies. Each accepted token has a `TokenConfig { bool active; uint256 rate; }` where `rate` is DPC wei per payment-token wei (1 USDC = 1000 DPC → `rate = 1000 × 10^12`, since USDC has 6 decimals and DPC has 18). | Function | Description | | ------------------------------------------------------ | ----------------------------------------------- | | `buyTokens(paymentToken, paymentAmount, minTokensOut)` | Swap external token → DPC (mints DPC). | | `sellTokens(paymentToken, dpcAmount, minTokenOut)` | Swap DPC → external token (burns DPC). | | `setTokenConfig(token, active, rate)` | Admin: accept a token and set its rate. | | `setTokenStrategy(token, strategy)` | Admin: route reserves through a yield strategy. | | `harvestYield(token)` | Admin: harvest yield from the strategy. | | `recoverToken(token, amount)` | Admin: recover stuck tokens. | A protocol fee (in basis points) is deducted on both buy and sell, sent to `feeController.treasury()`. Events: `TokensBought`, `TokensSold`, `TokenConfigUpdated`, `TokenStrategySet`. ### MorphoStrategy `MorphoStrategy.sol`, an optional yield strategy that routes Exchange reserves into [Morpho Blue](https://morpho.org/) supply markets. Only the authorized `exchange` can call `deposit`, `withdraw`, and `withdrawAll`; the admin configures markets via `setMarket` and claims surplus yield via `harvest`. ### Factory `scripts/Factory.sol`, a CREATE2 factory for deterministic deployments. `deploy(salt, initCode)` deploys and verifies the predicted address; `getAddress(initCodeHash, salt)` and `initHash(initCode)` help compute addresses ahead of time. ## Deployments The protocol is deployed on the **Sepolia** testnet. Addresses are recorded in `scripts/deployments.json`. ### Sepolia addresses | Contract | Address | Role | | ------------------------- | -------------------------------------------- | ------------------------- | | `Factory` | `0xadeb683e2c28514b89e600a0cd50185254b2d7f3` | CREATE2 deployer | | `Token` (DPC) | `0x6d663c7c401a18208748dF6F909ffc0C68FA1B73` | Protocol-native token | | `FeeController` | `0xe091dD90A4a3e0ed43fEd2D85A0fD043823B4648` | Manages protocol fees | | `DataPrism` | `0x2614B6E7F2134FBD880c88BAB10412b046a11211` | Core storage | | `DataPrismCloud` | `0x7fEda6645CDdDd193d6bEFAB7367c98C30d62469` | Application layer | | `DataPrismCloudForwarder` | `0x6f01C6E81ae6Adac1471B3Ce4000a3CC812438dF` | EIP-712 meta-transactions | | `Exchange` | `0x0Cc50006445C7E0e08Fb46Bf5d8b1bECf5275Ba1` | DPC exchange | External token used by the Exchange (not deployed by the protocol): | Token | Address | | -------------- | -------------------------------------------- | | USDC (Sepolia) | `0xC6a054681bEBb1096BB03E3Fc433bD99BAe987d0` | The Exchange is configured with a USDC rate of `1000 × 10^12` (1 USDC → 1000 DPC). ### Deployment configuration Configured in `scripts/deploy.ts`: ```ts const RPC_URL = "https://ethereum-sepolia-rpc.publicnode.com"; const EXTERNAL = { usdc: "0xC6a054681bEBb1096BB03E3Fc433bD99BAe987d0" }; const USDC_RATE = 1000n * 10n ** 12n; // 1 USDC (6 decimals) → 1000 DPC (18 decimals) ``` Each contract is deployed via the `Factory` using a deterministic CREATE2 salt (salts `0`-`6` in `SALTS`). The deployer (`PRIVATE_KEY`) is passed as `initialOwner` to each contract. ### Deployment sequence ``` 0. Factory ← deployed via regular CREATE 1. Token(initialOwner) ← Factory CREATE2, salt 1 2. FeeController(initialOwner) ← Factory CREATE2, salt 2 3. DataPrism(Token, FeeController) ← Factory CREATE2, salt 3 4. Exchange(Token, FeeController, owner) ← Factory CREATE2, salt 4 5. DataPrismCloud(DataPrism, owner) ← Factory CREATE2, salt 5 6. DataPrismCloudForwarder(Cloud, owner) ← Factory CREATE2, salt 6 7. Wire: DataPrismCloud.setForwardContract(Forwarder) 8. Configure Token permissions (operator, minter, burner) 9. Configure Exchange (accept USDC) 10. Set DataPrismCloud custom fees in FeeController (creation → 1000 DPC / 1 USDC, data operations → 100 DPC / 0.1 USDC each, other prism operations → 0) ``` ### Post-deployment setup Run manually from the deployer account: ```solidity // 1. Set the protocol fee treasury (defaults to deployer) FeeController.setTreasury(treasuryAddress); // 2. Set the relayer commission treasury (non-official relayer share) DataPrismCloudForwarder.setProtocolTreasury(treasuryAddress); // 3. Whitelist at least one official relayer DataPrismCloudForwarder.setOfficialRelayer(relayerAddress, true); // 4. Set the priority window (seconds of exclusive official-relayer access) DataPrismCloudForwarder.setPriorityWindow(60); ``` ## Protocol Overview `dataprism-protocol` contains the Solidity smart contracts that form the on-chain core of DataPrism. Users own personal vaults (**prisms**) where they write, modify, and delete arbitrary byte payloads, paying for each operation in the native **DPC** token. ### Tech stack * **Build system:** [Foundry](https://book.getfoundry.sh/) * **Language:** Solidity `0.8.29` (`via_ir = true`, optimizer enabled, `200` runs) * **Deployment tooling:** TypeScript + [viem](https://viem.sh/) (`scripts/deploy.ts`), Node ≥ 18 * **External dependency:** [Morpho Blue](https://morpho.org/) for the yield strategy * **License:** BUSL-1.1 (Business Source License 1.1) ### Project structure ``` dataprism-protocol/ ├── src/ # Core smart contracts │ ├── Token.sol # ERC-20 DPC token │ ├── FeeController.sol # Protocol fee registry │ ├── DataPrism.sol # Storage layer (bytes32 prism IDs) │ ├── DataPrismCloud.sol # Application layer (bytes16 cloud IDs) │ ├── DataPrismCloudForwarder.sol # Meta-transaction relayer │ ├── exchange/ │ │ ├── Exchange.sol # ERC-20 ↔ DPC converter │ │ └── strategies/ │ │ └── MorphoStrategy.sol # Yield strategy via Morpho Blue │ └── interfaces/ # All contract interfaces ├── scripts/ │ ├── Factory.sol # CREATE2 factory for deterministic deploys │ ├── deploy.ts # Main deployment script │ ├── vanity.ts # Vanity salt finder │ └── deployments.json # Deployed contract addresses ├── test/ # Forge tests └── foundry.toml # Foundry configuration ``` ### The two storage layers DataPrism separates raw persistence from application logic: * **`DataPrism`**: the raw storage layer. Prisms are identified by `bytes32` IDs (`keccak256(owner, salt)`). Only the owner can write. Fees are deducted from the prism's DPC balance on every operation. * **`DataPrismCloud`**: the application layer built on top of `DataPrism`. It introduces **cloud IDs** (`bytes16`, the first 16 bytes of the prism ID, cheaper in calldata), **writers** (delegated write access with optional expiration), **two-step transfers**, and **on-chain enumeration** of prisms, data keys, and writers. `DataPrismCloud` is the `msg.sender` for all calls to `DataPrism`. Its `*As` functions (e.g. `writeDataAs`) are only callable by `DataPrismCloudForwarder`. ### Gasless meta-transactions `DataPrismCloudForwarder` lets users operate **without holding gas**: they sign an EIP-712 message off-chain and a relayer submits the transaction on their behalf in exchange for a `relayerFee` in DPC. See [Contracts → DataPrismCloudForwarder](/protocol/contracts#dataprismcloudforwarder) for the full flow and relayer economics. ### Compile, test, deploy ```bash # Compile (output in /out) forge build # Run tests forge test # Install deployment tooling pnpm install # (optional) Find vanity salts for deterministic addresses pnpm run vanity -- --factory --contract Token --args "" --prefix 0000 # Deploy (set PRIVATE_KEY first) export PRIVATE_KEY=0x... pnpm run deploy ``` The deploy script deploys all contracts via a CREATE2 `Factory`, wires the Forwarder into `DataPrismCloud`, configures token permissions, sets up the Exchange with USDC, and writes addresses to `scripts/deployments.json`. See [Deployments](/protocol/deployments) for the deployed Sepolia addresses and post-deployment steps. ### Security model * Solidity `0.8.29` built-in overflow/underflow checks * EIP-712 signed meta-transactions with per-user nonce replay protection * ERC-2612 `permit` for gasless approvals * ERC-1271 support for smart-contract signers * ECDSA malleability check (`s ≤ secp256k1_n / 2`) * Two-step ownership transfers on all owned contracts * Role-based access (minters, burners, operators) with per-address min/max bounds To report a vulnerability, send a transaction to the owner address with details encoded as UTF-8 calldata. ## Payment Relayer `dataprism-payment-relayer` bridges fiat card payments to on-chain DPC tokens. After a user completes a Stripe payment on the [Frontend](/frontend/overview), this service buys DPC from the [Exchange](/protocol/contracts#exchange) with USDC and transfers it to the user's wallet. > Not to be confused with the [Relayer](/relayer/overview): that service relays user data > meta-transactions, while the **payment** relayer handles the fiat → DPC on-ramp. ### Tech stack * **Language:** TypeScript (ES2022) · **Runtime:** Node.js * **Key dependencies:** [viem](https://viem.sh/) `2.28`, [tslog](https://tslog.js.org/), [nodemailer](https://nodemailer.com/), [dotenv](https://github.com/motdotla/dotenv), [pm2](https://pm2.keymetrics.io/) ### Project structure ``` src/ ├── relayer/ │ ├── runner.ts # Entry point: bootstrap executor + notifier, start the poller │ ├── poller.ts # Polling loop: fetch, claim, execute, report │ ├── executor.ts # buyAndTransfer(): buy DPC on the Exchange, transfer to user │ └── gas.ts # ETH balance monitor (separate process) ├── api/client.ts # HTTP client for the frontend's relayer endpoints ├── abi/ # ERC20, Exchange, DPCToken ABIs ├── config.ts # Environment variable parsing └── notification.ts # Email + Telegram alerts ``` ### Payment flow ``` User pays via Stripe → Frontend records a pending payment ↓ Payment relayer polls GET /api/relayer/pending ↓ Claims the payment POST /api/relayer/claim (atomic, dedupes across instances) ↓ Buys DPC on the Exchange with USDC (1% slippage), then transfers DPC to the user ↓ Reports completion POST /api/relayer/complete { paymentIntentId, txHash } ``` #### `buyAndTransfer()` (`executor.ts`) 1. **Acquire DPC**: check the relayer's DPC balance. If insufficient, verify gas is below `MAX_FEE_GWEI`, ensure (unlimited) USDC approval to the Exchange, then call `Exchange.buyTokens(USDC_ADDRESS, usdcAmount, minOut)` with 1% slippage tolerance and wait for 1 confirmation. 2. **Transfer DPC**: call `ERC20.transfer(toAddress, dpcAmount)` on the DPC token and wait for 2 confirmations; return the transfer tx hash. Both steps run inside a retry loop (`MAX_RETRIES`, `RETRY_DELAY_MS`). The buy step is **idempotent**: if the relayer already holds enough DPC from a prior failed attempt, it skips straight to the transfer. DPC has 18 decimals, USDC has 6; the USDC amount to spend is derived from the configured `EXCHANGE_RATE` (1 USDC → 1000 DPC by default). ### Frontend endpoints consumed The payment relayer is a **client**: it exposes no HTTP server. It calls three frontend endpoints, each with an `x-relayer-secret: ` header: | Endpoint | Purpose | | ---------------------------- | --------------------------------------------------- | | `GET /api/relayer/pending` | Fetch pending payments awaiting processing. | | `POST /api/relayer/claim` | Atomically claim a payment (`{ paymentIntentId }`). | | `POST /api/relayer/complete` | Report success (`{ paymentIntentId, txHash }`). | ### Configuration ```bash # Required RELAYER_PRIVATE_KEY="" # relayer wallet (needs MINTER_ROLE on DPC, holds USDC) RELAYER_SECRET="" # shared secret, must match the frontend FRONTEND_URL="http://localhost:3000" # Chain & RPC CHAIN_ID="11155111" # 11155111 = Sepolia, 1 = mainnet RPC_URLS="https://rpc.sepolia.org" # comma-separated, with fallback # Contracts (Sepolia defaults) DPC_TOKEN_ADDRESS="0x6d663c7c401a18208748dF6F909ffc0C68FA1B73" USDC_ADDRESS="0xC6a054681bEBb1096BB03E3Fc433bD99BAe987d0" EXCHANGE_ADDRESS="0x0Cc50006445C7E0e08Fb46Bf5d8b1bECf5275Ba1" # Tuning POLL_INTERVAL_MS="10000" MAX_FEE_GWEI="100" MAX_RETRIES="5" RETRY_DELAY_MS="5000" # Gas monitor GAS_WARN_BALANCE_ETH="0.1" GAS_POLL_INTERVAL_MS="180000" # Notifications (optional) SMTP_HOST="" SMTP_PORT="587" SMTP_USER="" SMTP_PASS="" SMTP_FROM="" SMTP_TO="" TELEGRAM_BOT_TOKEN="" TELEGRAM_CHAT_ID="" ``` A `.env.example` template is provided. ### Install, run, deploy ```bash npm install # Install dependencies npm run build # Compile to dist/ npm run dev # Build + run runner.js npm run deploy # Build + start via PM2 (production) ``` PM2 (`ecosystem.config.js`) runs two processes: **DataPrism Payment Relayer** (`runner.js`) and **DataPrism Gas Monitor** (`gas.js`), each with their own logs under `logs/`. ### Notes * The relayer wallet must have `MINTER_ROLE` on the DPC token and hold USDC (and ETH for gas). * Multiple RPCs are supported via viem's `fallback()` transport. * A failed payment stays unclaimed and can be retried by the same or another instance. * It grants the Exchange an **unlimited** USDC approval for efficiency, ensure the Exchange is trusted. ## Frontend `dataprism-frontend` is the **DataPrism Cloud** web dashboard. Users manage data projects, storage, API keys, team members, and buy DPC tokens with a card (Stripe) or on-chain (USDC). Authentication is wallet-based. ### Tech stack * **Framework:** [Next.js](https://nextjs.org/) 16 (App Router) · React 19 · TypeScript 5 * **Styling/UI:** TailwindCSS 4, Radix UI, shadcn, Framer Motion, Lucide & HugeIcons * **Web3:** [wagmi](https://wagmi.sh/) 3, [viem](https://viem.sh/) 2, MetaMask connector * **Data:** TanStack React Query 5, next-themes, react-dropzone * **Payments:** Stripe SDK ### Project structure ``` dataprism-frontend/ ├── app/ │ ├── api/ │ │ ├── relayer/ # /pending, /claim, /complete (consumed by the payment relayer) │ │ ├── stripe/ # /checkout, /history, /subscription │ │ └── exchange/ # /history (on-chain USDC→DPC swaps) │ ├── (auth)/login, (auth)/register │ ├── projects/[id] # Project detail (6 tabs) │ ├── tokens/ # Token dashboard + /payment checkout │ ├── settings/ │ ├── layout.tsx, page.tsx, providers.tsx ├── components/ # UI + app components (sidebar, topbar, dashboards, sheets) ├── contexts/ # auth-context, projects-context ├── lib/ # contracts.ts, chains.ts, wagmi.ts, key-storage.ts, … └── public/ ``` ### Features #### Authentication * **`/register`**: generates a private key (a "Prism Key") via viem, encrypts it with a password (`encryptPrivateKey`), and stores it in `localStorage`. The user can copy, show, or download it. * **`/login`**: unlock a stored Prism Key with its password / import from file, or connect a browser wallet (MetaMask, Coinbase, WalletConnect, …). * An `AuthGuard` component protects every page except `/login` and `/register`. Identity is the wallet address; private keys never leave the browser. #### Projects * **Home (`/`)**: a dashboard of project cards (status, storage usage, token balance, members, API calls). The **"New Project"** sheet is a 4-step wizard, **Identity** (name / description), **Encryption** (curve + algorithm), **Storage** (DataPrism managed) and **Deploy** (DPC funding + review). It creates a real on-chain prism through the SDK: it derives the project key from a wallet signature, then creates **and** funds the prism in a **single transaction** via an EIP-2612 permit, no separate `approve`. (A "project" in the UI is a prism.) * **Project detail (`/projects/[id]`)**: six tabs: **Overview**, **Storage** (upload/list files), **SDK**, **Storage access** (managed storage status), **Members** (owner / writer / reader roles), and **Tokens** (balance + transaction history). #### Tokens & payments * **`/tokens`**: shows the on-chain DPC balance, network, and a unified payment history (card + crypto). Buy preset or custom token packs. * **`/tokens/payment`**: two checkout paths: * **Card (Stripe):** redirects to Stripe Checkout; on success the [Payment Relayer](/payment-relayer/overview) mints/transfers the DPC. * **Crypto (on-chain):** approve USDC → call `Exchange.buyTokens()`, or a single atomic batch on EIP-5792-capable wallets. 1% slippage protection; a testnet faucet mints 1,000 test USDC. #### Settings Profile, security (password / 2FA), notification preferences, and account deletion. ### Backend & contract integration `lib/wagmi.ts` targets **Sepolia** with Coinbase, WalletConnect (if `NEXT_PUBLIC_WC_PROJECT_ID` is set), and a custom private-key connector; storage key `dataprism.store`. Contract addresses (`lib/contracts.ts`): | Contract | Sepolia address | | --------- | -------------------------------------------- | | Exchange | `0x0Cc50006445C7E0e08Fb46Bf5d8b1bECf5275Ba1` | | DPC Token | `0x6d663c7c401a18208748dF6F909ffc0C68FA1B73` | | USDC | `0xC6a054681bEBb1096BB03E3Fc433bD99BAe987d0` | The `app/api/*` routes back the dashboard: Stripe checkout/subscriptions, Stripe & on-chain payment history, and the relayer endpoints (`/pending`, `/claim`, `/complete`) consumed by the [Payment Relayer](/payment-relayer/overview). ### Configuration | Variable | Purpose | | --------------------------- | ------------------------------------------------- | | `STRIPE_SECRET_KEY` | Stripe payment processing (backend only). | | `RELAYER_SECRET` | Shared secret authenticating the payment relayer. | | `NEXT_PUBLIC_WC_PROJECT_ID` | Optional WalletConnect project ID. | ### Install, run, deploy ```bash npm install npm run dev # http://localhost:3000 npm run build npm run start # production server npm run lint ``` Deploy to Vercel (recommended) or any Node host. Before going to production: set a live `STRIPE_SECRET_KEY`, a secure `RELAYER_SECRET`, and verify the contract addresses and network in `lib/contracts.ts`. ## Credits & Payments Every on-chain operation in DataPrism is paid for in **DPC**, the protocol's credit token. This page explains what DPC is, how users acquire it, and how it funds a prism. ### The credit token (DPC) Credits are an ERC-20 token internal to the protocol, **DPC** (defined in [`dataprism-protocol`](/protocol/contracts#token-dpc), 18 decimals). DPC pays for: * **Protocol fees**: every on-chain write / modify / delete of data consumes DPC, priced by the [FeeController](/protocol/contracts#feecontroller). * **Relayer fees**: when the user does not pay gas themselves, a relayer fronts the transaction in exchange for a commission in DPC (the `relayerFee`). ### Topping up A user funds their DPC balance in one of two ways. #### By card (Stripe) ``` User ──card payment──► Stripe ──webhook──► dataprism-payment-relayer │ payment confirmed? │ yes ▼ buys DPC on the Exchange (with USDC) and transfers it on-chain to the user ``` The [`dataprism-payment-relayer`](/payment-relayer/overview) **listens** for confirmed Stripe payments. On confirmation, it acquires the matching amount of DPC (buying it on the [Exchange](/protocol/contracts#exchange) with USDC) and transfers it to the user's wallet, at the rate corresponding to the amount paid. #### By crypto (USDC) The user can also fund their balance directly in **USDC**, without Stripe, approving USDC and calling `Exchange.buyTokens()` on-chain (or a single atomic batch on EIP-5792-capable wallets). The default rate is 1 USDC → 1000 DPC. ### Assigning credits to a prism Once DPC is on the wallet, the user **assigns it to a prism**. A prism's DPC balance is what funds all of its on-chain operations (index writes, relayer fees). **A prism with no credits cannot write anything.** ``` wallet (DPC) ──fundPrism──► prism balance ──consumed by──► writes + relayer fees ``` #### Single-transaction funding (EIP-2612 permit) `DataPrismCloud` pulls DPC from the wallet with `token.transferFrom(owner, cloud, amount)`. The cloud is **not** a token [operator](/protocol/contracts#token-dpc) (only `DataPrism` is), so that pull needs an **allowance**. Rather than a separate `approve` transaction, the SDK has the user sign an **EIP-2612 `permit`** (a free, off-chain signature) and the contract runs the permit and the funding in **one** transaction: * **Wallet path**: `createPrismWithPermit(fundAmount, salt, deadline, v, r, s)` / `fundPrismWithPermit(...)`: a single tx, **no approve** (the user pays gas). * **API path**: the request carries a `PermitParams`; the Forwarder calls `token.permit(...)` before `createPrismAs(...)`: gasless and approve-free. The permit domain is the DPC token (`name = "DataPrism Token"`, `version = "1"`, nonce `token.nonces(owner)`); the spender is the `DataPrismCloud` address and the value is the funded amount. ### The fee model in practice | Fee | Unit | Paid by | When | | ---------------- | ---- | --------------------------- | ----------------------------- | | **Gas** | ETH | the user | wallet execution path | | **`relayerFee`** | DPC | the prism balance → relayer | API execution path | | **Protocol fee** | DPC | the prism balance | every write / modify / delete | In practice the protocol fee is 100 DPC (0.1 USDC) per data operation (write, modify, or delete) and 1000 DPC (1 USDC) to create a prism; funding, transferring, and deleting a prism are free. On the API (gasless) path the user pays no ETH; instead they set a `relayerFee` in DPC when signing, which compensates the relayer for fronting the gas. On the wallet path the user pays gas in ETH directly and `relayerFee` is irrelevant. See the [SDK execution layer](/sdk/execution) for how this is exposed in code, and the [Exchange contract](/protocol/contracts#exchange) for the on-chain swap mechanics. ## File Storage & Resilience DataPrism accepts **any file, any size**. To do so it combines content-defined chunking, convergent encryption, erasure coding, content-addressing, and managed object storage. The result: storage only sees encrypted fragments, identical data is stored once, and missing shards can be reconstructed when the configured quorum is available. This is the conceptual model. The [SDK File Pipeline](/sdk/files) page documents the concrete TypeScript API (`FilePipeline`, `ReedSolomon`, chunkers). ### Upload ``` file (any size) │ ① CHUNKING (content-defined, FastCDC by default) │ boundaries defined by content → variable sizes │ (inserting bytes shifts only one chunk → robust dedup) ▼ [chunk 1] [chunk 2] ... [chunk N] │ ② CONVERGENT ENCRYPTION │ chunk key = KDF( hash(plaintext chunk) ) (deterministic) │ → ciphertext (same content ⇒ same ciphertext) ▼ [ct 1] [ct 2] ... [ct N] │ ③ ERASURE CODING (Reed-Solomon) - on the CIPHERTEXT │ each chunk → k data shards + m parity shards │ (e.g. 6 total, quorum 4 → any 4 of 6 reconstruct) ▼ [shard 1] [shard 2] [shard 3] [shard 4] [shard 5] [shard 6] │ ④ CONTENT-ADDRESSING + DEDUP │ shard key = hash(shard content) │ if the shard already exists in storage → skip upload ▼ │ ⑤ MANAGED STORAGE │ each shard → its own content-addressed object ▼ managed storage ◄─ shard1, shard2, shard3, shard4, ... │ ⑥ MERKLE MANIFEST │ the chunk list (hash + len + shard hashes) forms a "manifest"; │ if it is large it is stored ITSELF as a blob (encrypted + RS + │ stored) → rolled up to a tiny ROOT ▼ ⑦ ON-CHAIN WRITE (the root only) dataId = keccak256(filename) value = encrypted root index (~few hundred bytes, CONSTANT size regardless of file size) transaction sent via the execution layer (wallet OR api/relayer) ``` #### Why each step matters * **Content-defined chunking (FastCDC).** Chunk boundaries are chosen by the content via a rolling hash, so inserting or removing bytes shifts only **one** chunk instead of re-aligning the whole file. This keeps deduplication robust across file versions. Chunk sizes are therefore variable, so the index records the **length of each chunk**. The chunker is pluggable, `FastCDCChunker` (default) or a fixed-size chunker. * **Convergent encryption.** Each chunk is encrypted with a key **derived from its own content** (`KDF(hash(chunk))`). The same plaintext always yields the same ciphertext, so storage only ever sees ciphertext **and** deduplication still works. (Caveat: like all convergent schemes, it is deterministic, vulnerable to a confirmation-of-file attack.) * **Erasure coding (Reed-Solomon).** Each chunk's ciphertext is split into `k` data shards + `m` parity shards. Any `k` of the `n = k + m` shards reconstruct the chunk. The **quorum** (e.g. 4 of 6) sets the fault tolerance. * **Content-addressing + dedup.** Each shard is stored under **the hash of its content**. Before uploading, the SDK checks whether the shard already exists with a signed `HEAD` → if so, it is **not re-uploaded**. Two files (or versions) sharing identical chunks share their shards: zero duplication. It also gives **integrity**: any tampering changes the hash and is detectable. * **Managed placement.** Each shard is a separate content-addressed object. The SDK requests a short-lived, project-scoped URL for every storage operation. * **Compact index (Merkle manifest).** The routes are **not** all written on-chain. The chunk list is treated as a blob (encrypted, erasure-coded, stored, content-addressed), and only a **root** (`BlobRef`) goes on-chain. For a 1 TB file as for a 1 KB file, the on-chain index is **\~a few hundred bytes**. The real chunk list lives in managed storage, and everything is verifiable from the root. ### The index structure The on-chain index has the shape: ```ts FileIndex = { name, size, chunkSize, quorum, providers[], root: BlobRef } BlobRef = { level, size, chunks: [ { hash, len, shards[] } ] } ``` * `level 0`: the chunks are file data (small files are inline). * `level > 0`: the chunks reconstruct a manifest; the tree is descended on download. * `providers[]`: the ordered storage adapter IDs used at upload. The managed path records `dataprism_managed`; object access is resolved by the API from the user, project, and content hash. ### Download / reconstruction ``` filename │ ① hash → dataId ▼ read the on-chain ROOT (via RPC) + decrypt it (prism key) │ ② DESCEND THE MERKLE TREE │ if root.level > 0: reconstruct the manifest(s) from the │ storage adapter, until the full data-chunk list is obtained ▼ data-chunk list: for each → hash + len + shard hashes │ ③ fetch the shards from managed storage (in parallel) ▼ gather at least `quorum` shards per chunk │ ④ RECONSTRUCT (Reed-Solomon) → the chunk CIPHERTEXT │ as long as quorum is reached, the chunk is recovered ▼ │ ⑤ CONVERGENT DECRYPTION │ key = KDF( chunk hash, read from the index ) → plaintext chunk ▼ reassemble the chunks ▼ original file reconstructed ``` The quorum describes how many shards are needed to reconstruct each encrypted chunk. With the default managed-storage setup, all shards are authorized through the same platform service. ### Current limits > Safe deletion of shared files (reference counting / garbage collection) is **not yet > implemented**. An `overwrite` or delete may leave orphan shards behind (planned work). ## Prisms & the Cloud Standard A **prism** is an on-chain, namespaced vault dedicated to the **index** of your files (the reconstruction "routes"), not to the files themselves. The heavy data lives at your cloud providers; the prism only holds lightweight, encrypted, tamper-proof metadata. ### Identity A user's DataPrism identity **is** an Ethereum wallet address. Two entry points: * **Existing wallet**: connect MetaMask (or any compatible wallet) through the frontend (viem / wagmi). * **PrismKey**: for users without a wallet, DataPrism generates and manages one for them. The user never has to understand the blockchain. That address serves as identity (prism ownership, request signatures), as the destination for purchased credits, and, crucially, as the **source of the encryption key** (see [`dataprism.key`](#dataprismkey) below). ### Identifiers: protocolId & cloudId Each prism has a `bytes32` identifier, the **protocolId**: derived deterministically as `keccak256(DataPrismCloud address, salt)`. On the application layer, a shortened **cloudId** (`bytes16`, the first 16 bytes of the protocolId) is used to reduce calldata cost. Because the derivation is deterministic, the SDK can compute the cloudId **offline** (`computeCloudId(address, salt)`). This is what lets a prism created through the gasless API path know its ID **immediately**, without waiting for the relayer's confirmation. ### Creating a prism Creation goes through the SDK, which calls the on-chain creation function (`DataPrismCloud.createPrism`). Two paths, depending on whether the user holds gas: ``` ┌──────── does the user hold ETH? ────────┐ │ YES NO │ ▼ ▼ direct contract call call to dataprism-api (DataPrismCloud.createPrism) with a signed request user pays the gas (EIP-712, gasless) │ │ │ dataprism-relayer executes │ the transaction on-chain ▼ ▼ prism created on-chain ◄───────────────────────┘ ``` The SDK's [execution layer](/sdk/execution) makes both paths transparent, the calling code is identical. At creation, a prism is provisioned with a set of **standardized data slots**. When a prism is created with an initial balance (`fundAmount > 0`), funding and creation happen in a **single transaction**: the SDK signs an EIP-2612 `permit` instead of a separate `approve` (`createPrismWithPermit` on the wallet path, `createPrismFor` with `PermitParams` on the API path). See [Single-transaction funding](/concepts/credits#single-transaction-funding-eip-2612-permit). Creation can also **write an initial batch of data in the same atomic transaction**, so a prism is created *and* populated in one signature and one nonce increment (equivalent to a `createPrism` immediately followed by `batchWriteData`, but atomic). The contract exposes: * `createPrismWithData(fundAmount, salt, dataIds, payloads)`, on the wallet path. * `createPrismWithDataAndPermit(fundAmount, salt, dataIds, payloads, deadline, v, r, s)`, which also bundles the EIP-2612 `permit` so a funded, populated prism is created with no separate `approve`. * `createPrismWithDataFor(...)` on the Forwarder for the gasless (API) path, and the `createPrismWithDataAs` callback that the Forwarder invokes. Here `fundAmount` must cover the creation fee **plus** the per-entry data fees. The `cloudId` derivation is unchanged and stays deterministic, so it is known offline immediately. In the SDK this is the default behaviour of `PrismManager.create()`: it provisions the prism and writes the standard slots (key, encryption, metadata, providers) in **one** transaction, and any extra `initialData` entries are written in the same call. See the [execution layer](/sdk/execution) for the `createPrismWithData` method. ### The Cloud Standard The Cloud Standard is a naming convention for the reserved slots of a prism, the same idea as Ethereum's EIP/ERC standards, applied to cloud storage. Each slot is identified by a `dataId` = `keccak256("dataprism.xxx")` (a `bytes32`). The stored value is always JSON serialized to bytes; it is **encrypted** unless noted otherwise. | Slot | `dataId` | Encrypted? | Holds | | ---------------------------------------------- | ----------------------------------- | ---------- | ----------------------------------- | | [`dataprism.key`](#dataprismkey) | `keccak256("dataprism.key")` | clear | the prism's public key | | [`dataprism.encryption`](#dataprismencryption) | `keccak256("dataprism.encryption")` | clear | curve + algorithm | | [`dataprism.metadata`](#dataprismmetadata) | `keccak256("dataprism.metadata")` | sealed | name, description, member addresses | | [`dataprism.providers`](#dataprismproviders) | `keccak256("dataprism.providers")` | sealed | managed-storage service marker | | file index | `keccak256(filename)` | sealed | per-file reconstruction root | #### `dataprism.key` ```json { "public": "", "private": "" } ``` * `public`: the prism's public key. Readable by anyone (it is used to encrypt *for* the prism), so this slot is stored **in clear**. * `private`: the private key. **Empty by default and never written on-chain.** The private key is not stored, it is **derived on demand**, client-side, from a **wallet signature**: ``` connected wallet │ the user signs a deterministic message: │ • dataprism:key:${cloudId} (default) │ • dataprism:key:${cloudId}:${password} (extra security) ▼ signature (deterministic) ▼ = private key used by the SDK ▼ public key derived → written into dataprism.key.public ``` **Why the cloudId?** It is public and known from the prism's id, so the key is **reproducible** from the prism alone (no per-prism secret to store; works when listing prisms by `cloudId`). **Why the optional password?** Deriving from the cloudId alone means anyone who **steals the wallet's private key** could reproduce the signature and decrypt. Adding a password (`dataprism:key:${cloudId}:${password}`) makes it a **second factor**: decryption then also requires the password. The password is never stored, only mixed into the signed message. The exact signed message is **`dataprism:key:${cloudId}`** (default) or **`dataprism:key:${cloudId}:${password}`** (with a password). The same message must be used at creation and on every read, or the derived key will not match. > Optional public mode: if the user accepts that **anyone** can read their data, they may > choose to write `private` on-chain. This is an explicit, opt-in "public" mode. #### `dataprism.encryption` ```json { "curve": "secp256k1", "algorithm": "aes-256-gcm" } ``` * `curve`: `"secp256k1"` | `"x25519"` | `"ed25519"` * `algorithm`: `"aes-256-gcm"` | `"xchacha20"` These parameters define how the key (above) encrypts and decrypts everything else (metadata, providers, file indexes). Stored **in clear** so a reader knows how to decrypt the rest. #### `dataprism.metadata` ```json { "name": "", "description": "", "addresses": { "0x...": "human-readable name" } } ``` * `name`: `description`, prism information. * `addresses`: the list of authorized owner / writer addresses, each mapped to a readable name. This JSON is **sealed** (encrypted) with the parameters from `dataprism.encryption`. #### `dataprism.providers` A sealed JSON keyed by provider ID. New projects store only a public marker for DataPrism managed storage: ```json { "dataprism_managed": { "service": "dataprism" } } ``` This marker contains no credentials. The signed-in user authorizes individual storage operations through the API, which returns short-lived URLs scoped to the project and object. #### File index slots In addition to the reserved slots above, each uploaded file creates a slot whose `dataId` is the **hashed filename**. Its value is the (sealed) JSON of the **root** reconstruction index: quorum, provider list, and a **Merkle root** (`BlobRef`) that points to the chunks/manifests stored at the providers. This root stays **tiny** (≈ a few hundred bytes) regardless of file size. See [File Storage & Resilience](/concepts/file-pipeline). ### Working with prisms in code The SDK turns these concepts into a typed API, `CloudStandard` (encode/seal each slot), `CloudReader` (read + decrypt in one step), and `PrismManager` (single-transaction creation that also writes the standard slots, plus dedicated slot editors with a guard against overwriting reserved slots). See the [SDK Cloud Standard](/sdk/standard) page. ## Authentication The API uses four access levels. ### Managed storage: Privy access token `POST /storage/sign` requires `Authorization: Bearer `. The API verifies the token with the configured Privy app and uses its user ID to derive a tenant-scoped object path. Missing, expired, or revoked tokens return `401 Unauthorized`. ### Users: no authentication `POST /requests`, `GET /requests?signer=…`, and `GET /requests/{id}` are public. User requests are self-authenticating: the EIP-712 signature embedded in `params` (`signer`, `v`, `r`, `s`) proves intent and is verified on-chain by the Forwarder, not by the API. ### Relayers: ECDSA signature Required for `PATCH /requests/{id}`, `GET /requests?status=…`, and `GET /requests?chainName=…`. Implemented in `src/auth/relayerAuth.ts`. The relayer builds and signs an auth message, then sends three headers: | Header | Value | | --------------------- | ------------------------------------ | | `X-Relayer-Address` | The relayer's Ethereum address. | | `X-Relayer-Signature` | ECDSA signature of the auth message. | | `X-Relayer-Timestamp` | Unix timestamp in **milliseconds**. | **Auth message format:** ``` DataPrism Relayer Auth Timestamp: {timestamp} Method: {HTTP method} Path: {request path} Body: {hash of the request body} ``` The server verifies that: 1. The timestamp is within **5 minutes** of the current time. 2. The signature is valid (`viem.verifyMessage`). 3. The recovered address is in the relayer whitelist (`dataprism-relayers-{stage}` table). Any failure returns `401 Unauthorized`. ### Admin: API key Required for all `/admin/relayers` routes. Send the `X-Admin-Key` header; its value must exactly match the `ADMIN_API_KEY` environment variable (verified by `isAdminAuthorized()` in `src/handlers/admin/relayers.ts`). The default development value is `dev-admin-key-change-me` and **must be changed in production**. A missing or incorrect key returns `401 Unauthorized`. ## Endpoints All routes are served via AWS HTTP API Gateway. Responses are JSON. See [Authentication](/api/authentication) for the storage, relayer, and admin schemes. ### Managed storage #### `POST /storage/sign` Return one short-lived signed object operation. Requires a valid Privy access token in the `Authorization: Bearer ` header. **Body:** ```json { "projectId": "0x… (bytes16)", "key": "0x… (SHA-256 content hash)", "operation": "put | get | head | delete" } ``` The API derives the object path from the authenticated user ID, project ID, and content hash; callers cannot provide a bucket or object path. **Response `200`:** ```json { "url": "https://…", "method": "PUT", "headers": { "Content-Type": "application/octet-stream" }, "expiresAt": "2026-…" } ``` Errors: `400` (strict validation), `401` (missing, expired, or revoked session), `500`. ### Requests #### `POST /requests` Submit a new request. **No authentication**: the request is self-authenticating via the embedded EIP-712 signature. **Body:** ```json { "chainName": "demo", "action": "WRITE_DATA_FOR", "params": { /* depends on action - see below */ } } ``` **Response `201`:** ```json { "requestId": "uuid", "chainName": "demo", "action": "WRITE_DATA_FOR", "signer": "0x...", "status": "pending", "params": { }, "createdAt": "2026-...", "updatedAt": "2026-..." } ``` Errors: `400` (validation), `500`. #### `GET /requests` List requests with filtering and cursor pagination. **Query parameters** (at least one filter required): | Param | Description | | ----------- | ------------------------------------------------------------------------------ | | `signer` | Filter by signer address (`0x…40 hex`). **Public.** | | `status` | `pending` \| `processing` \| `success` \| `failed`. **Requires relayer auth.** | | `chainName` | Filter by chain. **Requires relayer auth.** | | `limit` | 1-100, default 20. | | `lastKey` | Base64url pagination cursor from the previous page. | **Response `200`:** ```json { "items": [ { "requestId": "...", "status": "pending", "...": "..." } ], "count": 10, "lastKey": "base64url-or-null" } ``` To page, pass the returned `lastKey` on the next call. A `null` `lastKey` means the last page. Errors: `400`, `401` (status/chainName query without relayer auth), `500`. #### `GET /requests/{requestId}` Fetch a single request by its UUID. **Public.** Returns the full request object. Errors: `400`, `404`, `500`. #### `PATCH /requests/{requestId}` Update a request's status. **Requires relayer authentication.** Used to claim a pending request (`processing`) or finalize it (`success` / `failed`). **Body:** ```json { "status": "success", "txHash": "0x… (required when status=success)", "errorMessage": "string (required when status=failed, max 1000 chars)" } ``` The `pending → processing` claim is atomic; a `409 Conflict` means the request was already claimed or the state transition is invalid. Errors: `400`, `401`, `409`, `500`. ### Admin: relayer whitelist All admin routes require the `X-Admin-Key` header (see [Authentication](/api/authentication)). #### `POST /admin/relayers` Add a relayer to the whitelist. Body: `{ "address": "0x...", "label": "optional" }`. Response `201` with the created record. Errors: `400`, `401`, `500`. #### `DELETE /admin/relayers/{address}` Remove a relayer. Response `200` `{ "message": "Relayer removed", "address": "0x..." }`. Errors: `400`, `401`, `500`. #### `GET /admin/relayers` List whitelisted relayers. Response `200` `{ "relayers": [...], "count": N }`. Errors: `401`, `500`. ### Actions and parameters The `params` object of `POST /requests` is validated (via Zod) per `action`. Every action shares the meta-transaction fields `relayerFee`, `timestamp`, `signer`, `v`, `r`, `s`. | Action | Action-specific parameters | | ----------------------------- | ---------------------------------------------------------- | | `CREATE_PRISM_FOR` | `fundAmount`, `relayerFundAmount?`, `salt`, `permit?` | | `WRITE_DATA_FOR` | `cloudId`, `dataId`, `payload` | | `MODIFY_DATA_FOR` | `cloudId`, `dataId`, `payload` | | `DELETE_DATA_FOR` | `cloudId`, `dataId` | | `TRANSFER_PRISM_FOR` | `cloudId`, `newOwner` | | `ACCEPT_TRANSFER_FOR` | `cloudId` | | `DELETE_PRISM_FOR` | `cloudId` | | `BATCH_WRITE_DATA_FOR` | `cloudId`, `dataIds[]`, `payloads[]` | | `BATCH_MODIFY_DATA_FOR` | `cloudId`, `dataIds[]`, `payloads[]` | | `BATCH_DELETE_DATA_FOR` | `cloudId`, `dataIds[]` | | `BATCH_MULTI_USER_WRITE_DATA` | `requests[]` (array of individually signed write requests) | > `permit` is the EIP-2612 signature object `{ deadline, v, r, s }` that authorizes the DPC pull > for a funded prism (`fundAmount > 0`). The relayer passes it to the Forwarder's > `createPrismFor`, so funding and creation happen in one transaction with no prior `approve`. > It is omitted when `fundAmount` is `0`. ### Error format ```json { "error": "Human-readable message", "details": [ /* optional, for validation errors */ ] } ``` | Status | Meaning | | ------------- | ------------------------------------------------------------------- | | `200` / `201` | Success. | | `400` | Validation error or missing parameter. | | `401` | Authentication failed (signature, expiry, whitelist, or admin key). | | `404` | Resource not found. | | `409` | State conflict (e.g. request already claimed). | | `500` | Unexpected server error. | ## API Overview `dataprism-api` is a serverless REST API for managed-storage authorization and gasless request queueing. Signed-in users request short-lived storage URLs; users also submit EIP-712 requests that relayers claim, process on-chain, and complete. The API does not submit blockchain transactions directly. ### Tech stack * **Language:** TypeScript `5.8` * **Runtime:** Node.js `20.x` on AWS Lambda * **Framework:** [Serverless Framework](https://www.serverless.com/) `3.40` (HTTP API Gateway) * **Build:** esbuild (bundled, minified) * **Storage:** Amazon DynamoDB + a private, encrypted Amazon S3 bucket * **Validation:** [Zod](https://zod.dev/) `3.25` * **Crypto:** [viem](https://viem.sh/) `2.31` (signature verification) * **Session auth:** Privy access-token verification ### Project structure ``` dataprism-api/ ├── src/ │ ├── auth/relayerAuth.ts # Relayer signature authentication │ ├── auth/storageAuth.ts # Privy authentication for storage URLs │ ├── db/ │ │ ├── dynamo.ts # DynamoDB operations for requests │ │ └── relayers.ts # DynamoDB operations for the relayer whitelist │ ├── handlers/ │ │ ├── admin/relayers.ts # Admin endpoints (manage whitelist) │ │ ├── signStorageOperation.ts # POST /storage/sign │ │ ├── getRequest.ts # GET /requests/{id} │ │ ├── listRequests.ts # GET /requests │ │ ├── submitRequest.ts # POST /requests │ │ └── updateRequestStatus.ts # PATCH /requests/{id} │ ├── storage/signing.ts # Scoped object keys + signed operations │ ├── types/request.ts # Types and the RequestAction enum │ ├── utils/http.ts # HTTP response helpers + CORS │ └── validation/schemas.ts # Zod schemas per action └── serverless.yml # Serverless Framework configuration ``` ### Request lifecycle ``` pending → processing → success ↘ failed ``` * **pending**: created by the user via `POST /requests`. * **processing**: claimed by a relayer (`PATCH` with `status=processing`). The `pending → processing` transition is **atomic** via a DynamoDB conditional expression, preventing two relayers from claiming the same request. * **success**: completed; relayer supplied a `txHash`. * **failed**: relayer supplied an `errorMessage`. ### Request state (DynamoDB) Three tables (suffixed by stage, e.g. `-dev` / `-prod`): * **`dataprism-requests-{stage}`**: primary key `requestId` (UUID). Global secondary indexes: `signer-createdAt-index`, `status-createdAt-index`, and a chain index (queried by `chainName`). * **`dataprism-relayers-{stage}`**: the relayer whitelist, primary key `address`. * **`dataprism-ratelimit-{stage}`**: per-client API rate-limit counters. Both use on-demand (`PAY_PER_REQUEST`) billing. ### Configuration Environment variables (set in `serverless.yml`): | Variable | Description | Default | | -------------------------------- | ------------------------------------------------ | ---------------------------------------------- | | `STAGE` | Deployment stage (`dev` / `prod`). | from `--stage` | | `REQUESTS_TABLE` | Requests table name. | `dataprism-requests-{STAGE}` | | `RELAYERS_TABLE` | Relayers table name. | `dataprism-relayers-{STAGE}` | | `RATELIMIT_TABLE` | Rate-limit table name. | `dataprism-ratelimit-{STAGE}` | | `ADMIN_API_KEY` | Key for admin endpoints. | `dev-admin-key-change-me` (**change in prod**) | | `PRIVY_APP_ID` | Privy app used by the frontend. | required | | `PRIVY_APP_SECRET` | Secret for server-side Privy token verification. | required | | `STORAGE_BUCKET` | Managed bucket name. | provisioned by the stack | | `STORAGE_SIGNED_URL_TTL_SECONDS` | Signed operation lifetime (30-300 seconds). | `60` | Default region is `eu-west-1`; functions run with 256 MB memory and a 29 s timeout. ### Install, run, deploy ```bash npm install # Install dependencies npm run offline # Run locally with serverless-offline (port 3000) npm run build # Compile TypeScript npm run deploy # Deploy to AWS (dev stage, eu-west-1) npm run deploy:prod # Deploy to the prod stage ``` Deploying provisions the Lambda functions, HTTP API Gateway, DynamoDB tables, the private managed-storage bucket, and least-privilege IAM permissions for object operations. ### CORS All responses include: ``` Access-Control-Allow-Origin: * Access-Control-Allow-Headers: Content-Type, Authorization, X-Relayer-Address, X-Relayer-Signature, X-Relayer-Timestamp, X-Admin-Key Access-Control-Allow-Methods: GET, POST, PATCH, DELETE, OPTIONS ``` The managed bucket has a separate CORS policy: production permits `https://app.dataprism.co`, while development permits the local frontend origins. See [Endpoints](/api/endpoints) for the full route reference and [Authentication](/api/authentication) for storage, relayer, and admin auth.