Dockup is one container per Docker host. It watches the Docker API for volumes and containers that carry dockup.* labels and backs them up according to those labels. Destinations, schedules, encryption keys and notification settings live in Dockup's own configuration so they are not repeated in every stack.

flowchart LR
  S[(Docker socket)] --> D[Discovery<br/>volumes, containers, events]
  D --> P[Planner<br/>labels to jobs]
  P --> C[Scheduler<br/>cron, locks, concurrency]
  C --> R[Runner<br/>quiesce, archive, upload, prune, notify]
  R -->|create, mount :ro, attach| H[Helper container<br/>same image]
  H -->|tar stream| R
  R --> T1[(local)]
  R --> T2[(S3)]
  R --> ST[(state volume<br/>SQLite + instance lock)]
  R --> M[/healthz, /metrics/]

Components

Package	Responsibility
internal/config	YAML + environment configuration, secret files, validation, redaction.
internal/labels	Parses dockup.* labels into source policies, with warnings and errors.
internal/plan	Merges Docker state with configuration into resolved sources and effective policies.
internal/schedule	Named and inline cron schedules.
internal/retention	Grandfather-father-son selection of archives to keep.
internal/dockerx	Docker API wrapper: snapshots, events, helper containers, quiescing.
internal/archive	tar writer/reader, compression, age encryption.
internal/target	Destination adapters (local, s3) and key layout.
internal/state	SQLite run history (goose migrations, sqlc queries), instance lock.
internal/runner	Executes one backup or restore, persists the result, notifies.
internal/notify	Webhook, healthchecks.io pings and Telegram.
internal/daemon	Discovery loop, scheduler, HTTP endpoints.
internal/cli	The dockup command tree.

Why helper containers

A volume can only be read by mounting it, and the set of volumes is not known when Dockup starts. Three approaches were considered:

Approach	Works with	Cost
Read /var/lib/docker/volumes from the host	local driver only	Depends on the daemon's storage layout, impossible with plugin volumes, different under rootless Docker.
Require every volume to be mounted into Dockup	all	New volumes need a Dockup redeploy, which defeats label discovery.
Spawn a short-lived helper container that mounts the volume	all	~100 ms and one container per backup.

Dockup takes the third option. The helper is created from the same image with the volume mounted read-only, network=none, a read-only root filesystem, all capabilities dropped and no-new-privileges. It streams a tar of the mount to stdout; Dockup reads that stream through a Docker attach connection and pipes it through compression and encryption into a spool file. Restore is the mirror image: the archive is streamed into the helper's stdin and untarred into the volume.

Streaming through attach means no shared filesystem between Dockup and the helper and no dependence on where either container's files live.

Data flow of a backup

sequenceDiagram
  participant C as Scheduler
  participant R as Runner
  participant D as Docker
  participant H as Helper
  participant T as Target

  C->>R: run(source, schedule)
  R->>R: acquire per-source lock
  R->>D: exec pre hooks / stop opted-in containers
  R->>D: create helper (volume :ro, no network)
  R->>D: attach helper stdout
  H-->>R: tar stream
  R->>R: zstd -> age -> spool file + sha256
  R->>D: wait for helper, remove it
  R->>D: restart containers, run post hooks
  R->>T: upload spool file
  R->>T: verify (optional)
  R->>T: prune by retention policy
  R->>R: persist run, notify

A backup is spooled to <state-dir>/tmp before upload so that a single archive can be sent to several targets, verified by re-reading it, and retried without re-taring the volume. The spool directory must therefore hold the largest compressed archive.

Concurrency and safety

• Per-source lock: two schedules of the same source never overlap.
• Global limit: max-concurrent bounds parallel backups.
• One instance per host: an exclusive lock in the state directory makes a second Dockup exit with a clear error instead of backing everything up twice.
• Retention never deletes the newest archive, even when max-age would.
• Only opted-in containers are touched: dockup.stop=true to stop, dockup.hook=<name> to run commands.
• Labels are validated, never ignored silently: unknown or invalid labels are reported by dockup plan, dockup status, logs and a metric.

State

Run history lives in SQLite in the state volume:

• schema is managed with goose migrations embedded in the binary;
• queries are generated by sqlc from internal/state/queries;
• the driver is pure Go (modernc.org/sqlite), so the image stays static.

This is deliberately not a backup catalog: archives are the source of truth, and losing the database only loses history and metrics, never the ability to restore.

Health and metrics

listen (default :9779) serves:

Endpoint	Meaning
/healthz	Liveness. Always 200 while the process runs.
/readyz	200 once Docker has been read successfully; 503 with the reason otherwise.
/metrics	Prometheus exposition.

Metrics are read from the state database at scrape time rather than kept in memory, so they survive restarts and stay correct when a run is recorded by dockup backup while the daemon is running.

Metric	Labels	Meaning
dockup_build_info	version, commit, go_version	Build metadata.
dockup_sources	kind	Sources resolved from labels.
dockup_sources_invalid	kind	Sources blocked by an error issue.
dockup_plan_issues	level	Issues found in the current plan.
dockup_source_last_run_timestamp_seconds	kind, source, schedule, status	When the source last ran.
dockup_source_last_success_timestamp_seconds	kind, source, schedule	When the source last succeeded.
dockup_source_last_duration_seconds	kind, source, schedule, status	Duration of the last run.
dockup_source_last_archive_bytes	kind, source, schedule	Size of the last archive.
dockup_runs_total	status	Recorded runs by status.
dockup_state_read_errors_total	—	Failed reads of the state database.

Discovery is event driven: Docker volume and container events trigger an immediate reconcile, and a periodic resync (--interval, default one minute) is the safety net when the event stream drops.

Targets and retries

A target is any object store with Put, Get, List and Delete. v1 ships local (directory or bind-mounted network share) and s3 (any S3 compatible endpoint: MinIO, R2, B2, Wasabi, AWS). The runner always hands a target a seekable spool file, so a failed upload can be retried from the beginning instead of re-archiving the volume.

Transient failures are retried three times with exponential backoff and jitter: network errors, HTTP 5xx/429 and the usual S3 throttling codes. Permanent errors (bad credentials, missing bucket, invalid key) fail immediately. Each target is independent: one unreachable destination does not stop the others, but the run is recorded as failed so it shows up in status and alerts.

Encryption

Compression runs first, then age encryption, so the checksum the runner computes covers the bytes that are actually stored. Both modes are supported:

• X25519 recipients (encryption.age.recipients) for unattended backups;
• a passphrase (encryption.age.passphrase) for humans.

Encrypted archives get a .age suffix. Restoring needs encryption.age.identity (or the passphrase); without it Dockup refuses to write a backup that the policy marked as encrypted rather than silently storing plaintext.

Consistency

Dockup only touches containers that opt in, and only in a fixed order:

1. pre hooks run inside containers that carry dockup.hook=<name> — a pg_dump can write into the volume before it is archived.
2. Containers that carry dockup.stop=true and mount (or declare) the source are stopped with a grace period (stop-timeout, default 30s).
3. The helper archives the source.
4. Stopped containers are started again and post hooks run — also when the archive failed, so temporary dumps never linger.

A failing pre hook aborts the run before anything is stopped, and a failing restart is recorded as a failed run: leaving a database down must never be silent.

Notifications

Webhook, healthchecks.io style pings and Telegram are supported. Failures are always delivered; successes only when notify.on-success is set. Every channel is independent and a failing notification is logged but never changes the run outcome.

Catch-up

With dockup.catch-up=true a source is backed up shortly after Dockup starts if its last successful run is older than the upcoming schedule interval. Catch-up runs never overlap a scheduled or manual run: the per-source lock refuses the second one.

Restore

Restore is the mirror image of a backup: the archive is downloaded, decrypted and decompressed in the controller, and streamed into a helper container's stdin, which untars it into the read-write mount.

The helper refuses to write into a non-empty directory unless --force is passed (exit code 3, mapped to a readable error), so a typo cannot silently replace a live volume. The newest archive is selected by default; dockup list shows what is available. See operations.md.

Key layout

<prefix>/<host>/volume/<volume>/<schedule>/<filename>.tar.zst[.age]
<prefix>/<host>/bind/<host path>/<schedule>/<filename>.tar.zst[.age]

Retention lists the directory of one source and schedule and buckets archives by the timestamp in their name, so it works identically for local directories, S3 and any future target.

Milestones

#	Deliverable	State
M0	Scaffold, configuration, labels, planner, retention, CLI (plan, config), CI, image build	done
M1	Docker event discovery, SQLite run history, instance lock, /healthz + /readyz + /metrics, status	done
M2	Archive engine (helper containers, tar + zstd/gzip streaming), local target, scheduled and manual backups, retention pruning, backup/prune, integration tests against real Docker	done
M3	S3 target, age encryption, multi-target fan-out, retries, checksum verification	done
M4	Quiescing (stop + hooks), notifications (webhook, healthchecks, Telegram), catch-up	done
M5	Restore + list, documentation, v0.1.0 release and multi-arch images	done

Explicit non-goals for v1

Docker Swarm and multi-node orchestration, deduplicating repository formats (restic/kopia style), a web UI, and credentials in labels.