Sources and checks

Add a Postgres source

Point the agent at a Postgres database, take the first backup, see it appear in the dashboard.

The fast path: restorable source add

The wizard handles the common case in three prompts. Pick engine kind (postgres), paste the connection string, accept (or override) the suggested slug. The agent connects, probes the role's privileges, introspects the schema, and attaches a <slug>-rows check on the largest user table at 50% of the current count plus a <slug>-tables check at 80% of the introspected user-table count (deliberately conservative floors that catch truncations, half-restored snapshots, and missing schemas without false-failing on stat noise). Preview the YAML diff, confirm, done.

sudo -u restorable restorable source add

The wizard writes the source plus its env-var entry transactionally: if the env file write fails for any reason, the config rolls back to its prior state so the next run isn't looking at a half-applied source. Pass --no-default-check to skip the auto-attach when you plan to author every check by hand.

Want a domain-specific check (a referential-integrity sweep, a fresh-row floor, a business invariant)? Edit config.yaml after the wizard runs; see Writing a check. The rest of this page walks the same flow by hand for operators who prefer YAML to wizards, and covers the role-grant + env-file setup the wizard expects to already exist.

Before you start

Have these ready:

  • An agent installed and connected per the quickstart. Its dashboard entry shows a green dot.
  • A connection URI for the database. A read-capable role is enough; superuser is not required.
  • The matching postgresql-client major version on the agent host. Newer clients read older dumps; older clients may not read newer dumps.

1. Create a read-only role

A dedicated role for the agent scopes the blast radius if the agent host is ever compromised. Give it enough to run pg_dump and nothing more.

CREATE ROLE restorable_backup LOGIN PASSWORD 'generate-a-strong-one';
GRANT CONNECT ON DATABASE prod TO restorable_backup;

\c prod

GRANT USAGE ON SCHEMA public TO restorable_backup;
GRANT SELECT ON ALL TABLES IN SCHEMA public TO restorable_backup;
GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO restorable_backup;

-- Keep rights up to date for tables created in the future:
ALTER DEFAULT PRIVILEGES IN SCHEMA public
  GRANT SELECT ON TABLES TO restorable_backup;
ALTER DEFAULT PRIVILEGES IN SCHEMA public
  GRANT SELECT ON SEQUENCES TO restorable_backup;

Repeat for every schema you want backed up. pg_dump run as this role produces a complete logical dump without requiring superuser.

If your database uses row-level security, the role may also need BYPASSRLS, or the dumps will be incomplete. Add it explicitly and document why: ALTER ROLE restorable_backup BYPASSRLS;.

2. Put the connection URI in the env file

Secrets do not belong in config.yaml. They live in /var/lib/restorable/env, which is loaded by systemd before the agent starts.

sudo tee -a /var/lib/restorable/env >/dev/null <<'EOF'
RESTORABLE_DB_URI_PROD=postgres://restorable_backup:PASSWORD@db.internal:5432/prod?sslmode=require
EOF

sudo chmod 0600 /var/lib/restorable/env
sudo chown root:root /var/lib/restorable/env

The variable name is up to you. Pick something that makes sense if you later add a second database. RESTORABLE_DB_URI_PROD scales.

sslmode=require is strongly recommended, even on a private network. It costs nothing and catches a class of misconfiguration where the agent silently connects in plaintext.

3. Declare the source in config.yaml

Open /var/lib/restorable/config.yaml and add:

sources:
  - slug: prod
    kind: postgres
    connection:
      uri: env:RESTORABLE_DB_URI_PROD
    checks:
      - healthy

checks:
  - id: healthy
    summary_sql: |
      SELECT COUNT(*) AS tables
      FROM information_schema.tables
      WHERE table_schema = 'public'
    expect: "tables > 0"

The source id is URL-safe and appears in the dashboard URL for this source. prod is short and clear. Pick something stable; renaming later is treated as delete-and-recreate from the orchestrator's perspective.

The healthy check above is the simplest honest starting point. Replace it with a check that exercises a representative query for your data as soon as the round-trip is working. See Writing a check for how to choose.

4. Reload the agent

sudo systemctl reload restorable.service

The reload re-reads the config. Watch the logs to confirm:

sudo journalctl -u restorable.service -f

You should see a line confirming the new source was picked up on the next heartbeat. Failures at this stage are validation errors or the env var missing; see agent.yaml reference for the full list.

5. Take the first backup

In the dashboard, open the source and click Run backup now. The command lands on the next heartbeat. The agent:

  1. Resolves the connection URI from the env var.
  2. Runs pg_dump as the restorable_backup role.
  3. Pipes the dump through zstd and age encryption.
  4. Requests a 15-minute signed upload URL from the orchestrator.
  5. Uploads the ciphertext directly to Scaleway Object Storage.
  6. Registers the backup's metadata.

Backup duration tracks pg_dump's runtime on your database. A 10 GB database with decent disk typically finishes inside a few minutes.

6. Run the first restore test

Once the backup is registered, click Run restore test now. The agent:

  1. Fetches a 15-minute signed download URL for the ciphertext.
  2. Downloads the ciphertext, decrypts it with the age key.
  3. Spins up an ephemeral Postgres scratch container matching the source's major version.
  4. Runs pg_restore into the scratch.
  5. Runs every check listed on the source, in order, against the scratch.
  6. Signs a receipt with the Ed25519 key.
  7. Submits the receipt to the orchestrator.
  8. Destroys the scratch container.

Open Receipts in the dashboard. The new row is the one you just signed. Click through to see the payload, the per-check detail, and the summary row the check returned.

Common failures

pg_dump: permission denied for table X

The role lacks SELECT on that table. Grant it; the role needs SELECT on everything the dump touches.

pg_restore: extension "vector" not available

The scratch container does not have your source's extensions. Two paths:

  • Override restore_scratch.image in config.yaml to an image that has the extensions (e.g., pgvector/pgvector:pg16).
  • Use restore_scratch.kind: uri pointing at an external endpoint where you control the image.

Collation version mismatch warnings

Not a failure by itself; the dump still restores. The warning signals that the scratch's libc collation version differs from the source's. Unique indexes on collated columns can silently accept duplicates after a real collation change. The receipt carries these warnings; watch for them and rebuild indexes in production after collation upgrades.

Connection refused

The agent host cannot reach the database over the network. If the DB is on a private network, the agent needs to sit in that network. No workaround on the Restorable side.

Adding a second Postgres source

Duplicate the source block with a new id, add its URI to the env file, and reload. The orchestrator's scheduler picks it up on the next heartbeat. Multi-source is where the dashboard's per-source schedule and retention controls start paying off.

Sources are independent: a backup for one does not block a backup for another, up to the agent's concurrency cap (two simultaneous backups per agent, configurable). Restore tests are serialised per source but can run concurrently with backups on the same source.