Multi-agent setups

An org can register any number of agents. Each is an independent cryptographic identity; the blast radius of a compromise of one does not reach the others. Common shapes and the trade-offs behind them.

Shapes that actually work

Per environment

One agent each in production, staging, and the QA cluster. Each agent sees only its environment's databases. Each has its own signing key; receipts are legibly per-environment in the weekly digest.

This is the most common multi-agent shape. A compromise of the staging agent cannot forge production receipts because the signing keys are different.

Per region

One agent per primary region for databases that are region-local. The EU-based customer with a US replica, or the global shop with DR in a second provider. Each regional agent is on a host inside that region so it reaches its databases over the low-latency path.

Receipts tagged by their agent make it clear which region's databases were tested. Per-region agents also mean a regional outage (the agent's host, not the database) takes only one region's evidence offline.

Primary + hot standby

Two agents on two hosts, each configured with the same set of sources. One is active; the other is passive. If the primary host dies, you run setup on the standby against its own auth key (it already has its own keys) and the standby takes over.

Source ownership is explicit, not automatic failover. The standby is not silently taking backups; you promote it by reassigning sources in the dashboard. That is deliberate. Two agents on the same source produce duplicate receipts and confuse the timeline.

Shapes to avoid

Same source on two active agents

Do not do this. Two agents racing on the same source produce two parallel backup and restore-test streams, and the dashboard cannot sensibly collapse them. The "per-source has exactly one agent at a time" rule is enforced at source creation.

The legitimate case of "same data in two places for redundancy" is handled by the standby shape above: both hosts can run, only one owns a given source at a time.

One agent, dozens of sources across environments

Technically fine; the agent's concurrency caps keep the work bounded. Operationally painful: one compromised host exposes the signing key for every environment's evidence, and the weekly digest becomes a wall of activity across unrelated systems.

Split along whichever axis the blast radius should stop at (environment is usually it).

How the orchestrator treats N agents

• One org, N agents. Each with its own agent_id, signing pubkey, bearer token.
• Per-agent signing key. Never shared. Revoking one agent invalidates only that agent's future submissions.
• Source ownership. Each source belongs to exactly one agent at a time. Reassignment is an explicit dashboard operation; the new agent picks up the source on its next heartbeat.
• Receipts cross-reference. Every receipt carries its agent_id; the verifier looks up that agent's registered pubkey. An auditor with receipts from multiple agents verifies each against its own key.

Registering a second (or third) agent

Identical flow to the first. From the dashboard, Agents → Add machine, give it a distinct name that matches its role (prod-eu, staging-eu, dr-nl), generate a auth key, run setup on the second host.

The names land in dashboard URLs and in the runtime.type field of receipts. Pick them accordingly; they show up in auditor-facing artifacts.

Deciding sources per agent

Each source's config.yaml entry lives on exactly one agent. When you install a second agent, you choose which sources it owns by declaring them in its yaml and reassigning from the dashboard.

A good default: each agent owns the sources that live in the network it sits inside. The staging agent owns staging databases; the production agent owns production; cross-env ownership is an anti-pattern that usually means the network story needs fixing.

What happens if two agents try the same auth key

Auth keys are single-use. The second attempt gets auth key already used. Generate a fresh code for the second agent.

Dashboard view with N agents

The Agents screen lists them all with individual heartbeat state. The Sources screen tags each source with its owning agent. The receipt detail view shows the agent that signed it. Most navigation works without thinking about the agent dimension; the dimension is there when you need it.

One org, many customers' databases?

If you are running Restorable for clients (fractional CTO, managed-service provider), the right shape is one org per customer, not one org with agents per customer. Cross-org dashboards are a fractional-CTO-specific feature on the backlog; today each org is a hard boundary. One org per customer keeps the evidence cleanly attributable.