DocumentsRunning it

Recovery time and recovery point

2 min readSource rto-rpo.mdsha256 a7411039453341a3…

What has to be recovered#

The operator is stateless compute plus a store:

  • The compute is a Cloudflare Worker and the EU-pinned Durable Object it forwards to, redeployed from the one compiled bundle, whose hash is pinned in the provenance. The object keeps no state of its own: everything durable is in the store.
  • The store holds the governance events, member public keys, the signed verdict ledger, pKYC triggers and the webhook outbox.
  • Nothing else. The operator holds no secret and no raw data, so there is nothing else to restore.

Recovery point: why it is better than the store's#

Two independent copies of everything that matters exist outside the operator:

  1. Every governance event is signed by the member that made it, and members keep them.
  2. Every verdict is signed by the operator and held by the members whose legs authenticated. They receive it in the response, by webhook and through the SIEM export.

server/recovery-drill.test.js proves that the verdict log rebuilds from the members' copies alone to the same root, with every copy verified before re-admission, so a rebuild cannot smuggle a changed verdict in. For verdicts, the effective recovery point is therefore zero for any member that keeps its copies, whatever happens to the operator's store.

The store itself follows its provider's backup regime (see sub-processors). Its point-in-time setting is not published here. Confirm it before quoting a store-level recovery point.

Recovery time: measured#

node server/bench-recovery.js restores a store holding n signed verdicts, boots a fresh operator, and asks it for the signed head of the log, which reads and hashes every row.

VerdictsRestore to a signed head
1,0000.25 s
10,0002.4 s

Measured on 28 September 2026, on an Intel Core i5-8265U laptop (Node 24.16.0, Windows, in-memory store). It excludes the time to read the rows back from a remote database, and the time to redeploy the Worker. Run the script on your own hardware; the numbers move with it.

StepTargetBasis
Redeploy the operator from the pinned bundleMinutesOne command (wrangler deploy), or a rollback to the previous version
Restore the store from backupThe provider's restore timeNot measured here
Operator ready on the restored storeSeconds per 10,000 verdictsMeasured above
Rebuild the log from members' copies, if the store is lostMinutes to collect; seconds to rebuildThe recovery drill

These are targets with their basis stated, not an SLA. No availability commitment is published until someone operates the service under one.

What is tested#

The recovery drill (server/recovery-drill.test.js) runs on every build:

  • back up, restore, and require an identical registry and an identical log root;
  • rebuild from members' copies, and require the same root;
  • refuse an altered copy.