Deploy your first update

A self-contained operator path from nothing to a shipped, verified update on a device: install atc, bootstrap the on-device daemon, declare what to ship, push it, and roll back if you have to.

You do not need any of the .claude/skills/ material, Notion, or an external URL to follow this. Every command below was run against the real CLI; discover any subcommand’s full options with atc <cmd> --help.

Two pages cover this journey. This one is the short version, and it ends with a troubleshooting table. onboarding.md is the complete path: both device types, lifecycle hooks, container images, and the traps each step hides. Read one of them, not both.

Mental model (30 seconds)

atc is a controller CLI you run on your machine. It delivers artifacts to a device over SSH. A daemon, atcd, runs on each device and manages a content-addressed store at /var/atc. Every push is versioned and atomic: artifacts are staged, then the active version is flipped only when you ask (--switch). Rollback is just flipping back to an older version.

You declare what to ship in atc.toml and run atc from that directory.

What you need

• A controller machine with atc installed (Step 1). You do not need a checkout of this repo.
• A device you can SSH into (key auth preferred) with passwordless sudo for the one privileged bootstrap step. --non-interactive forces key-only SSH.
• The device’s CPU arch must be aarch64 or x86_64 (ssh user@device uname -m). Bootstrap probes it and picks the matching atcd; anything else needs a binary supplied with --from.
• The device’s init system (ssh user@device 'ps -p 1 -o comm='). If PID 1 is not systemd, skip Step 2 and use the no-systemd recipe instead. Check PID 1, not for a systemctl binary — systemctl can be present (and even answer) on devices where systemd is not running.

Step 1 — Install atc

atc is the controller CLI you run on your own machine:

curl -fsSL https://atc.dev/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"   # the installer drops atc here
atc --version

The installer verifies what it downloads (sha256 against a signed release index) before writing it. See verifying-the-binary.md to check that yourself.

You do not need to install atcd, the device daemon, by hand. Step 2 downloads the arch-matched, signature-verified atcd and installs it for you.

atc reads atc.toml from the directory you run it in.

Building from source instead? If you have a checkout of this repo and want the binaries you built, follow building-from-source.md. It covers the ./agency build path, then rejoins this guide at Step 2.

Step 2 — Bootstrap atcd onto the device (one-time)

Bootstrap probes the device’s arch, downloads the matching atcd from the same signed release origin atc self-updates from, installs it into the store at /var/atc, and enables a systemd service:

atc bootstrap atcd install --yes --non-interactive user@device
# → bootstrap: atcd started on user@device (socket /run/atcd/atcd.sock)
# → ✓ atcd <version> installed and running on user@device

The downloaded atcd goes through the same verification as a self-update: the ed25519 signature over the release index is checked against the pinned trust keys, and the binary’s sha256 against the signed digest, before anything is written or copied to the device.

There is exactly one atcd on the device: the store-managed binary the systemd unit execs, at /var/atc/channel/agency/atcd/delivery/current/atcd/atcd. /usr/local/bin/atcd is a symlink to it.

So atcd --version reports the version that symlink resolves to, which is not always the version the daemon RUNS. atcd upgrade re-points current, and the symlink follows it, but the daemon keeps serving the old process until a restart. In that window atcd --version reports the STAGED version. The reply from atcd upgrade names the running version, which it reads from the daemon on its socket.

Any target form works: a hostname, an IP address or localhost. You do not name the device. atcd names itself from a hash of its hardware identity, or from its device id when the hardware has none.

To ship a specific binary instead (one you built, or a patched build), pass --from ./path/to/atcd — it must match the device’s arch. --from is explicit intent, so a bad path is an error rather than a silent fall back to the download.

Verify the daemon is up and the controller can reach it:

ssh user@device 'systemctl is-active atcd'   # → active
atc ota status user@device                    # controller ↔ daemon; lists the (empty) store

A status (or any other command) can occasionally fail with ATC-E0308: ping ... timed out after 5s — a transient SSH-connection timeout, common on the first contact after starting atcd but possible on any contact. Retry once; it succeeds immediately (see Troubleshooting).

Want the device to install itself, with no atc? For provisioning scripts and Ansible, run atcd install on the device: curl -fsSL https://atc.dev/install-atcd.sh | sudo sh -s -- --socket-owner user. It is safe to run again, and --check changes nothing. Flags, exit codes and an Ansible task: onboarding.md, Path A on the device.

No systemd on the device? (a bare container, a minimal image) bootstrap aborts up front with ATC-E0419. Skip to No-systemd / container devices for the foreground recipe, then come back to Step 3.

Step 3 — Declare what to ship (atc.toml)

atc ota init scaffolds a minimal manifest; or write it by hand. The smallest valid atc.toml ships one bundle (a directory tree — your built app plus any service files / hooks):

[[bundle]]
name   = "app"             # on-device identity; unique across artifacts
source = "dist"            # directory to ship — must exist at push time

Put your built application under source (here, ./dist/). There is no version field — the version is a label assigned at push time (Step 4). A manifest is validated when you push; a missing source directory fails with ATC-E0504: bundle source not found.

Bundles can also carry lifecycle hooks — scripts run on the device at specific lifecycle states — see atc-toml.md for the hook syntax and the full manifest reference.

(For container images use a [[oci]] block with a source registry ref and atc oci import/oci build first — see atc oci --help.)

Step 4 — Push and activate

A bare push sends every declared artifact (the default); narrow to a subset with --only <name> (repeatable). A push stages by default; add --switch to make it the active version. The version label comes from --version (or -t name@version):

atc ota push --version v0.1.0 --switch user@device

atc ota push --dry-run user@device uploads and mutates nothing, but its report is terse: it previews only peripheral reconciliation — printing peripheral reconcile: no removals, or a table of peripherals it would remove. It does not summarize the bundles, versions, or bytes a real push would send.

Step 5 — Verify

atc ota status user@device     # current + prior version and daemon health
atc ota ls     user@device     # all on-device versions
atc ota netcheck user@device   # connectivity / daemon-reachability preflight

atc ota status showing your version as current means the update shipped and is live. The device table at the top reads:

• STORE — the on-device store root (/var/atc).
• DAEMON — the version of the atcd running on the device.
• MODE — how versions are assigned: delivery-pinned (the default and the normal mode for push-based delivery — the device tracks one pinned delivery across all artifacts) or per-artifact (each artifact resolved independently). Change it with atc ota assignment-mode <target> per_artifact, which takes an optional --for <DURATION> so the opt-in expires on its own.
• CHANNEL (in the artifact tables) — the channel each artifact tracks, e.g. local/default.

Where your files land on the device: the active version’s files are materialized at /var/atc/channel/local/default/artifacts/<name>/versions/<sha256>/. In the default delivery-pinned mode the stable pointer is not a per-artifact current symlink — it is the delivery pin: /var/atc/channel/local/default/delivery/current/<name>, a symlink that resolves (through the active delivery dir) to that artifact’s active versions/<sha256>/ dir and is flipped atomically on every switch/rollback. Point on-device services at delivery/current/<name> for a stable path. (A per-artifact artifacts/<name>/current symlink is written only in per-artifact MODE, never in the delivery-pinned default — see the MODE row above.)

Emergency rollback

Flip the active version back to an older one. Push output does not tell you the previous version — find it with atc ota ls, then switch. The signature is atc ota switch <TARGET> <VERSION> — host first, version second:

atc ota ls     user@device            # find the previous version to revert to
atc ota switch user@device v0.1.0     # re-activate it on user@device

(Order matters: a version-first form like atc ota switch v0.1.0 user@device is parsed as “host v0.1.0” and fails ATC-E0303: could not resolve hostname.)

Anti-rollback floor (opt-in)

A device can be told to refuse a delivery older than the one it already accepted. That is the anti-rollback floor. It is OFF by default. Turn it on per segment, with policy in the deployment doc:

{
  "root": {
    "policy": { "anti_rollback": true },
    "children": [
      { "kind": "rule", "name": "canary",
        "when": { "all": [ { "attr": "custom.tier", "op": "eq", "value": "canary" } ] },
        "policy": { "anti_rollback": false } }
    ]
  }
}

The root value is the deployment default. A segment that sets nothing inherits it. A segment that sets anti_rollback overrides it for itself and for every segment below. Read the resolved value back before you roll out:

atc fleet device resolve <DEVICE_ID> --channel stable
# anti-rollback   armed

The server cannot turn it off

The device reads the bit as an arm-only ratchet. The bit can arm the floor. It can never disarm one. A server that sends false to a device that already armed does nothing. This is deliberate: the floor defends against a compromised server, so the server must not hold its off switch.

To disarm a TEST robot, delete the floor on the device itself:

ssh user@device 'rm -rf /var/atc/.atc/delivery-floors/local/default'

The last two path components are the channel’s <namespace>/<name>. The device arms again on the next stamped delivery it accepts.

Limit: legacy devices never arm

The bit rides the desired-state object the server publishes for each device. An atcd that predates that object reads its desired state from POST /v1/checkin, and the server no longer serves that route. Such a device receives no channel policy and never arms. The deployment cannot change that. Upgrade the device to arm it.

Health timeout of a multi-host device

A device with several hosts switches to a new delivery on every host at once. After the switch, each host must confirm the new delivery within the health timeout. If a host does not confirm in time, the server rolls back every host. The default is 15 minutes. If your devices take longer to reboot, set health_timeout_secs in the segment policy:

{ "root": { "policy": { "health_timeout_secs": 2400 } } }

The value is from 1 to 86400 seconds. The server refuses a publish with any other value. It inherits like anti_rollback. The server also raises the value to twice the longest check-in interval of the hosts, so that one late check-in cannot cause a rollback.

No-systemd / container devices

atc bootstrap installs atcd as a systemd service. On a device whose PID 1 is not systemd (a bare container, a minimal image), bootstrap stops up front with ATC-E0419 and prints a foreground recipe. Here is the full end-to-end version of that recipe.

Critical: the binary must live at /usr/local/bin/atcd on the device. In this no-systemd recipe that path is a real binary you install, not the symlink-into-the-store the systemd bootstrap creates — there is no store-managed current/atcd to point at, because nothing seeds or manages a store here. The controller’s liveness check and auto-bootstrap reach the daemon by SSH-exec’ing /usr/local/bin/atcd attach (the [daemon] remote_path default). A daemon started from any other path (e.g. /tmp/atcd) is running and healthy, yet atc ota status / netcheck report the daemon as not present — because the controller probes the fixed path, not the running process. Install it there.

Equally critical: run atcd as the same user you SSH in as. The controller SSH-execs atcd attach as your SSH login user (the user in user@device), and the socket defaults to 0660 owned by the user that started the daemon. So if you start atcd as root (sudo atcd run) but SSH in as a non-root user, every atc ota status / push fails with connect to atcd at /run/atcd/atcd.sock: Permission denied (os error 13) — the daemon is up and healthy but unreachable. (Worse, that permission error is currently masked as ATC-E0401 atc.toml is malformed or an ATC-E0415 release-download 404 — see Troubleshooting.) The recipe below starts atcd without sudo so the socket is owned by your login user. Only steps 1–2 (install the binary, create+chown the dirs) need sudo.

On the device (the user you SSH in as, with passwordless sudo for steps 1–2):

# 1. Install the device-arch atcd at the path the controller probes. Bootstrap
#    needs systemd. `atcd install --no-service` can seed a store instead (see
#    onboarding.md, Path B); this recipe stages the binary by hand. Fetch it
#    with install-atcd.sh. It checks the signature on the release index and
#    the binary's sha256, and ATC_DOWNLOAD_ONLY makes it install nothing.
#    Set TRACK to the track that `atc --version` prints on your machine.
#      curl -fsSL https://atc.dev/install-atcd.sh | ATC_TRACK=$TRACK ATC_DOWNLOAD_ONLY=/tmp/atcd sh
#    It needs curl, OpenSSL 3, awk, and sha256sum or shasum on the device.
#    Without them, use the fetch
#    block in onboarding.md (Path B), or scp up a binary you built yourself.
sudo install -m 0755 /tmp/atcd /usr/local/bin/atcd
/usr/local/bin/atcd version   # arch check — 'exec format error' means the
                              # binary is for another CPU

# 2. Create the store root and the socket's parent dir (atcd does NOT create
#    them — on the systemd path the unit's RuntimeDirectory= does), and hand
#    both to your login user so the un-sudo'd daemon in step 3 can write them
#    and owns the socket the controller attaches through. Use chown -R:
#    if atcd ever ran as root it left root-owned subdirs a shallow chown misses.
sudo mkdir -p /var/atc /run/atcd
sudo chown -R "$(id -un):$(id -gn)" /var/atc /run/atcd

# 3. Start the daemon headless (NO sudo) so the SSH session returns AND the
#    socket is owned by your login user. setsid + redirecting all three streams
#    detaches it; a plain `nohup ... &` leaves the SSH channel hanging open.
setsid /usr/local/bin/atcd run \
  --store-root /var/atc --socket /run/atcd/atcd.sock \
  </dev/null >/tmp/atcd.log 2>&1 &
# Fresh-device sanity check: if the ssh gets wrapped in a `timeout` (so it exits
# 124/143 rather than returning), `DAEMON_UP` on stdout is a quick signal the
# daemon came up. This is only trustworthy on a FIRST bootstrap — on a re-run an
# OLD atcd may still hold the socket, so both the socket test and `pgrep -x atcd`
# can pass before (or even without) the new daemon binding. If you're re-running,
# stop the old daemon first (`sudo pkill -x atcd; sudo rm -f /run/atcd/atcd.sock`)
# before the start line above. The AUTHORITATIVE proof either way is the
# controller handshake in the next step (`atc ota status`), which only succeeds
# when the controller can actually attach to the live socket.
for i in $(seq 1 10); do
  [ -S /run/atcd/atcd.sock ] && pgrep -x atcd >/dev/null && { echo DAEMON_UP; break; }
  sleep 1
done

Release/opt builds log nothing by default, so an empty /tmp/atcd.log is normal — use pgrep -x atcd as the liveness check. To debug startup, prefix the run line with RUST_LOG=info.

Running atcd as your login user makes the socket owned by the identity the controller attaches as — the only setup that works out of the box. If you must run the daemon as a different user than you SSH in as (e.g. a dedicated atcd service account), grant that login user the socket with --socket-owner <login-user>, or add the login user to the daemon user’s group. Do not reach for --socket-mode 0666. That mode opens an admin control socket to every local process.

Started this way, lifecycle hooks run as atcd’s own uid — your SSH login user (the atc-hooks default in atc-toml.md applies to the systemd/bootstrap install, not this recipe). --hook-run-as-user <user> cannot work here: switching the hook uid needs a root (or CAP_SETUID/SETGID) daemon, and atcd refuses a hook user that resolves to its own uid. On this recipe, omit the flag.

A hook on this recipe cannot drive systemctl --user either, because the device has no user session. For a hook that must reach a root-owned file, use a supplementary group for the daemon’s user, then restart atcd. See path 1 in Root work on a device with no user session. Path 2 in that section needs systemd as PID 1, so it does not apply here.

/run is tmpfs — it does not survive a container restart. /run/atcd and the running daemon are both gone after the container restarts, so steps 2 and 3 (mkdir -p /run/atcd + the atcd run start) must run on every container start — bake them into your image’s entrypoint / init script. For a durable socket dir that survives restarts, put it under the (persistent) store root, e.g. --socket /var/atc/run/atcd.sock (and set the matching [daemon] socket_path in your atc.toml so the controller probes the same path).

The controller then talks to it exactly as a bootstrapped device — no bootstrap needed, plain SSH:

atc ota status user@device      # controller SSH-execs /usr/local/bin/atcd attach
atc ota push --version v0.1.0 --switch user@device

The first contact after starting atcd commonly exceeds the 5s ping window and fails with ATC-E0308: ping ... timed out — the recipe worked; retry once and it succeeds immediately.

If atc ota status reports the daemon as absent here, check pgrep -x atcd on the device (and /tmp/atcd.log, if you started it with RUST_LOG) and confirm the binary is at /usr/local/bin/atcd (step 1).

Troubleshooting

Next steps

• atc-toml.md. The full manifest schema and the lifecycle-hook reference.
• examples/heartbeat-module/. A repo that ships a module: an executable the device runs after the delivery commits, rather than a payload it only unpacks. Its README says what the example proves and what it cannot do. modules-today.md carries the whole map.