Operations

Backup and restore

Stop the daemon before restoring. Interactive restores require typing RESTORE; unattended automation additionally requires the explicit CG_RESTORE_CONFIRM=YES environment flag.

npm run backup creates a transactionally consistent SQLite snapshot with VACUUM INTO, packages the durable config/channel metadata, and encrypts it with AES-256-CBC/PBKDF2. Store the blob and key separately off-host. Set CG_BACKUP_PASSPHRASE or CG_BACKUP_KEY_FILE in unattended environments. Run npm run restore:drill after every material upgrade and at least monthly; it restores into a disposable directory and runs SQLite quick_check. Stop the service before a real npm run restore.

Both scripts read the database from CLAUDE_GATEWAY_DB when it is set (the same override src/config/paths.js honors), falling back to $CHANNELGATE_DIR/gateway.db. A restore replaces the managed config/ and channels/ directories rather than merging into them: each is staged beside its destination, swapped in with mv, and the previous copy is removed only after the swap succeeds — so a restored gateway never carries files the backup does not contain. If the archive step fails, npm run backup aborts instead of writing a partial blob.

Backup scope: npm run backup covers the gateway’s operational state — gateway.db, config, and the gated ~/.channelgate/channels/ folders. It deliberately does NOT include the visible channel workspaces (~/ChannelGate/<platform>/<slug>/ or per-channel custom folders): those are working directories, usually git repositories with their own remotes. Back them up separately if their contents aren’t pushed anywhere.

Service identities

Linux production installs use sudo bash scripts/install-systemd.sh, which creates the non-login channelgate account and a mode-0700 /var/lib/channelgate (an install upgraded from the pre-rename release keeps its existing claude-gateway account and /var/lib/claude-gateway, so no data moves underneath it; only a fresh install gets the new names). It also chowns the checkout to that account and verifies the account can write it — self-update runs git and npm ci as the service user, which fails on a root-owned checkout. A single-user box may instead run a user-scope unit (~/.config/systemd/user/channelgate.service, with loginctl enable-linger so it survives logout); the updater and the migration find both scopes. sudo bash scripts/uninstall-systemd.sh (npm run service:uninstall) removes either unit — current and pre-rename names — and leaves the service account and runtime root in place. The service must never share credentials or a runtime root with interactive development. ChannelGate runs on Linux only; there is no other service packaging.

Engine credentials — Claude uses the host user’s own login. The gateway authenticates Claude with the claude sign-in of the user the daemon runs as ($CLAUDE_CONFIG_DIR, else ~/.claude) — the same one that user maintains in their own shell. Keep that signed in and there is nothing else to configure: the daemon reads the login where it lives, relays its current access token to each container run, and refreshes it on the host when it gets close to expiry. The credentials file is never copied, linked or mounted, because Claude Code rotates the refresh token on every refresh and a second copy that refreshes logs the first one out.

A login session expires every few weeks and only a new interactive claude login moves that date. The daemon prints the source and the date at boot ([gateway] claude login: operator (…), session expires …), warns three days ahead, and shows the same line in /status and /api/health. Because a daemon that has been up for weeks would otherwise never say it again, an hourly watch re-checks the login while it runs: the gateway DMs every admin daily from three days before the login expires, and daily while it is missing (once per UTC day per message class, remembered in the database so a restart cannot turn it into a spam loop). The DM names the login, its expiry in UTC and in the gateway’s local timezone, and the remedy — never any token material. When it lapses, run claude as the daemon’s user and sign in again — no restart needed, and the next hourly check goes quiet on its own.

Two alternatives remain, in this precedence: a claude setup-token value pasted into Settings → Container runtime (long-lived, never rotates), and the daemon’s own ANTHROPIC_API_KEY. A login signed in to the gateway’s own engine home (<runtime root>/engine-state/claude/home/.claude) also still counts, behind the host user’s.

Measured, not assumed: the relayed token rides the child’s environment, but Claude Code strips CLAUDE_CODE_OAUTH_TOKEN (and ANTHROPIC_API_KEY) from the environment of every Bash tool subprocess — verified on 2026-09-02 on this host: a tool call running sh -c 'echo ${#CLAUDE_CODE_OAUTH_TOKEN}' printed 0 while HOME was visible — so an agent cannot printenv it from inside its container. What still inherits the process environment is a stdio MCP server Claude Code itself spawns; only the gateway’s own bridges and admin-curated catalog servers are ever injected, and they run as the same user in any case. The token is the ACCESS half only (hours), never the refresh token.

Engine credentials on Linux: the service account cannot run claude login (no login shell, its own empty home), so the engines authenticate via API keys. The installer creates a 0600 <runtime root>/service.env template read by the unit (EnvironmentFile=): fill in ANTHROPIC_API_KEY (and OPENAI_API_KEY for Codex), then systemctl restart channelgate. The installer also resolves the claude/codex binaries at install time and bakes their directories into the unit’s PATH — prefer system-wide CLI installs (e.g. npm prefix /usr/local); a CLI inside a user home is exposed to the service read-only with a warning. Without keys the daemon starts and serves the admin UI, but every engine turn fails authentication — this file is the fix, not claude login. The containers honor the same key: with no gateway login to relay, ANTHROPIC_API_KEY in the daemon’s environment authenticates Claude inside every container (it crosses through the reviewed passthrough list in src/engines/child-env.js), so a keyed install never sees “no Claude login to relay”.

Reverse proxies and tunnels

The daemon binds 127.0.0.1. The supported way to reach it from outside the host — for the admin UI over the internet, and for approval links a phone can open — is a tunnel or reverse proxy in front of it on the SAME machine (cloudflared, nginx). Set Settings → Connection → Public URL to the public address: it is what approval links are built from, and its hostname is automatically allowed past the DNS-rebinding guard (extra names go in CG_ALLOWED_HOSTS).

Client addresses behind that hop. Every request then arrives on a loopback socket, so req.ip — and Express’s idea of the client — is 127.0.0.1 for every caller on earth. Two things count failures per address: the admin login backoff and the approval-link token backoff. Keyed on the socket they would share ONE bucket, and a handful of bad tokens or passwords from anywhere would lock out every legitimate admin and every valid approval link. So both read the real client through one helper (clientKey in src/web/security.js): CF-Connecting-IP first, then the first hop of X-Forwarded-For, and only when the socket itself is loopback — from a non-loopback socket those headers are whatever the client typed and are ignored. A value that is not a valid IP address is ignored too.

Notes for operators:

• Express’s app-wide trust proxy is deliberately NOT enabled: it would also re-point req.secure / req.protocol / req.hostname at client-supplied headers (the Host/Origin guard and the session cookie’s Secure flag read those headers themselves, deliberately), and it knows nothing about CF-Connecting-IP.
• If your proxy runs on ANOTHER host, the socket is not loopback and the headers are ignored by default. Set CG_TRUST_PROXY=1 in .env to honour X-Forwarded-For there — only do this when a proxy you control rewrites that header, or clients can pick their own limiter bucket.
• Make the proxy set X-Forwarded-For rather than append to a client-supplied one. Where it appends (Cloudflare does), CF-Connecting-IP is the unforgeable value and is preferred here; a caller who spoofs the first XFF hop can only shed its own backoff, never push someone else into one.

The ChannelGate rename migration

scripts/migrate-channelgate.mjs moves a pre-rename install onto the current layout: ~/.claude-gateway/ → ~/.channelgate/, ~/Slack Agent/<slug>/ → ~/ChannelGate/<platform>/<slug>/, and, inside the runtime root, channels/<slug> → channels/<platform>/<slug> plus the same for clean-workspaces/. It runs automatically at boot (before the database opens and before Slack connects) and as a post-update step, and it is idempotent.

What counts as “already migrated” is STATE, not existence. The migration runs when the old root holds a gateway.db or a config/ directory and the new root holds neither. A ~/.channelgate/ that exists but is empty of state — a stray mkdir, an engine probe, a test that forgot to pin its environment — is treated as a leftover and the old install is MERGED into it, entry by entry; nothing at the destination is ever overwritten, and a colliding name is logged with its old copy left in place for you to reconcile. If the new root already holds gateway state it is never touched: the run says so and leaves the old root alone. The same rule applies to the workspace: an empty ~/ChannelGate/ is not a reason to skip the per-channel moves.

Before it moves anything it refuses if the pre-rename deployment is still working: a gateway.lock held by a live pid (the daemon), an update.lock held by a live pid (a self-update transaction), or any bg_jobs row whose recorded pid is still alive (a detached background shell whose cwd is inside the old workspace). A refusal moves nothing.

What gets repathed

Moving the folders is the easy half. Every store that RECORDS an absolute path has to follow, or the daemon comes back pointing at directories that no longer exist:

Store	What moves
Database JSON blobs	every data column in the schema — channel workDir, background-job cwd/logFile, approval and run records, anything a later migration adds
Database typed columns	every non-blob TEXT column holding a path (usage_repair_batches.backup_path today)
Config JSON	config/settings.json, config/mcp-catalog.json (a local MCP server’s command/args can name a path), the pre-SQLite users.json/channels.json/schedules.json backups, update-state.json, each update backup’s manifest.json, and the per-channel legacy meta.json/sessions.json/thread-*.json
Claude Code sessions	<engineHome>/.claude/projects/<encoded-cwd>/ is renamed to the encoding of the channel’s NEW cwd, and the cwd field inside every transcript (including per-session subagents/ and tool-results/) follows — only cwd, see “state vs. content” below. This is what keeps /resume working: claude -r <id> finds the project from the cwd it is launched in, so without the rename every thread’s history is invisible. projects/ is normally a symlink into the operator’s real ~/.claude, and is resolved before anything under it is touched
Claude engine home	.claude.json (both copies) keys a projects map by absolute cwd
Codex state	the threads index’s rollout_path column, each rollout’s session_meta/turn_context cwd + workspace_roots, config.toml’s [projects."…"] sections, and shell_snapshots
Work folder text	MEMORY.md, memory/<topic>.md, CLAUDE.md/AGENTS.md — the agent’s own prose can quote a path it was told to use
Service definitions	the installed systemd user unit (its log paths name the runtime root)
Per-channel lockdown	.claude/settings.json and settings-admin.json are REGENERATED, not rewritten
Per-channel runtime/	the content-addressed run caches (claude-settings/<digest>.json, claude-plugins/<digest>/) are DELETED. Their filename is a digest of their contents, so rewriting one would leave a file whose name no longer describes it; the next run recreates what it needs

Two things are deliberately left alone: the historical events log (rewriting an audit trail would make it say something that did not happen) and any path outside the roots that moved — a custom per-channel workDir, or the repo checkout named in a unit’s WorkingDirectory.

State vs. content: what may be rewritten at all

An engine transcript is two things in one file. A few fields are state — the engine reads them back to restore a session — and everything else is content: a record of what was said and done. A message that quoted ~/Slack Agent/ops, a cat whose stdout printed it, a shell command that ran there. Rewriting content would edit history to say something that never happened, so the migration never touches it and --verify never bills it.

The decision is made per JSON key path, never by substring, and the lists are deliberately tiny. Anything not on one is content.

Store	State keys	Everything else
Claude transcripts (projects/**/*.jsonl, subagents/*.json)	cwd — the session’s working directory, what claude -r and the migration’s own project-dir resolution read back	message.content[], toolUseResult.* (including stdout, filePath), attachment.*, and snapshot.trackedFileBackups, whose object KEYS are paths — object keys are never rewritten by anything
Codex rollouts (sessions/**/*.jsonl, format v0.143)	payload.cwd (the session_meta header and each turn_context) and payload.workspace_roots[]	payload.state.* under a world_state item, plus every message, argument and tool output

The filesystem key. world_state.payload.state.environments.filesystem is a rendered XML blob (<filesystem><workspace_roots><root>…</root></workspace_roots><permission_profile …>) that Codex emits once per turn, next to the turn_context it was derived from, as the environment context shown to the model. It is content, not state: Codex does not read it back to locate anything — on resume it re-derives the world state from the restored session_meta/turn_context cwd, which is repathed. The same goes for its neighbours state.agents_md.directory (the label on an inlined copy of an AGENTS.md) and state.environments.environments.local.cwd (the same cwd, already covered authoritatively). Repathing the two authoritative fields is what actually moves a session; rewriting the snapshot would only rewrite what the model was once told.

--verify

node scripts/migrate-channelgate.mjs --verify

A read-only sweep of every store above, grouped and counted, exiting 1 if any state still points at a pre-rename root. It opens the database read-only and writes nothing, so it is safe while the daemon runs — use it as a pre-flight and again as the acceptance check afterwards, where it must print 0. --dry-run ends with the same audit, so completeness can be judged BEFORE anything moves.

Three rules make the number mean something:

1. Per key, not per substring — the split above. Content is reported in its own historical content — a record of what was said and done, left by design section, next to the events log, and never counts toward the exit code. On a busy install it is normally the far larger number, and that is fine.
2. A name is not state, a cwd is. A Claude project directory is stale when its name is no longer the encoding of its own cwd, or when that cwd no longer exists — not because the name still spells a pre-rename path. A channel whose custom workDir really is ~/Slack Agent/<slug> was deliberately left in place, so the directory encoding that cwd is exactly right.
3. After the move, a path that still resolves is correct. A state field naming a pre-rename path that still exists on disk points at a real directory that did not move. Those are listed under still on disk — deliberately not moved, so the recorded path is correct with the stores that reference them, so the reason the count is zero is visible rather than assumed. Before the move this exoneration is switched off — nothing has moved yet, so everything is billed, which is what makes the same command work as a pre-flight.

So a non-zero exit always means there is something to repair, and --repath is the repair.

--repath

node scripts/migrate-channelgate.mjs --dry-run --repath   # preview
node scripts/migrate-channelgate.mjs --repath             # apply

Every rewrite pass the migration runs — database blobs and typed columns, config JSON, Claude projects (including directory renames) and engine home, Codex index/rollouts/config.toml, work-folder text, service definitions, and the per-channel lockdown regeneration — applied against the current roots. It moves nothing: not the runtime root, not the workspace, not a single channel folder. Use it when the one-time migration could not cover everything: a host somebody moved by hand, a store that was locked when the boot migration went through, a channel folder an operator relocated themselves.

Unlike the one-time migration, which drops every content-addressed run cache under channels/**/runtime/ (at migration time all of them name the old roots), a repath drops only a cache that actually still mentions one — otherwise every run would cold-start a healthy install.

It prints the same plan and the same summary as a migration, and ends with the audit (audit (post-repath)), so a run is its own acceptance check. It is idempotent — the rules are keyed on the pre-rename roots and on the flat channels/<slug> layout, so a store already on the new paths matches nothing and a second run reports zero and rewrites no byte. It refuses while the daemon is running for exactly the reason the migration does (it rewrites the same live stores); --dry-run reports the blocker and previews anyway, writing nothing.

--from <old> --to <new> — a folder you moved by hand

node scripts/migrate-channelgate.mjs --dry-run --repath --from /old/place --to /new/place   # preview
node scripts/migrate-channelgate.mjs --repath --from /old/place --to /new/place             # apply
node scripts/migrate-channelgate.mjs --verify --from /old/place --to /new/place             # count

The rename rules only know the roots the product renamed. A folder you moved — the daemon’s own checkout (~/Code/claude-gateway → ~/Code/channelgate), a channel’s custom workDir, a project directory — is invisible to them, so hand the pair over and it becomes one more rewrite rule for every pass above: the channel record, Claude project directories (renamed to the new cwd’s encoding, which is what keeps -r resume working) and their transcripts, the engine home’s .claude.json, Codex config.toml/index/rollouts, the moved folder’s own MEMORY.md/memory/*.md, the installed service definition (WorkingDirectory), and the per-channel settings regeneration. Repeat the two flags for several folders. Move the folder FIRST, then repath: the rule says where the folder is, and the prose is rewritten there. Both sides must be absolute and distinct, and the new path may not lie under the old one (that rule would rewrite its own output on every run). A malformed or misplaced pair (without --repath/--verify) exits 2 with usage before anything is read. --verify handed the same pairs counts what still names the old side, so the acceptance check is the same command.

Reloading the service after the migration

The migration rewrites the installed service definition, but the running service holds a CACHED copy: systemctl --user daemon-reload is required before the next start reads the new log paths. The migration cannot do this itself (it runs inside the service it would tear down), so it leaves <runtime root>/service-reload-required.json and scripts/update-runner.mjs consumes it on the next restart: daemon-reload (in the unit’s own scope) before the restart signal. Restarting by hand? Run the command the migration logged first.

--dry-run deliberately bypasses these checks and reports them instead — previewing the plan while the daemon is up is exactly when an operator wants it. The finding is printed as the first plan line (busy: … — a real run would refuse right now), the old database is opened read-only, and neither new root is created.

On refusal or failure the boot continues on the OLD paths. The process pins CHANNELGATE_DIR and CG_WORKSPACE_DIR back to the pre-rename roots for its own lifetime — an env override, not a symlink, so nothing on disk pretends the move happened. Clear the blocker and restart, or run the script by hand.

Preview with node scripts/migrate-channelgate.mjs --dry-run, which prints the same plan and writes nothing. Channels with a custom workDir are never moved, an existing destination is never clobbered (it is skipped and reported), a cross-device move copies and verifies before removing the source, and each old location keeps a MOVED.md breadcrumb. Every channel’s .claude/settings.json is regenerated afterwards so nothing in it names a pre-rename path.

After migrating, re-run the service installer so the systemd unit matches the new name (channelgate.service); the installer removes the pre-rename one first so an upgraded host never runs two daemons against one runtime root.

Container runtime

This is the runtime — required, Linux-only. Every channel’s engine processes run inside a long-lived container of that channel’s own: its own HOME volume, its own CLI logins, its own process namespace, and none of the host’s files reachable beyond the channel’s work folder. There is no host fallback: the daemon refuses to boot without a usable container CLI, and a channel that cannot get its container (no built image, no engine login) fails the turn closed with the remedy named rather than running anywhere else. The daemon itself still runs on the host as an ordinary service; only the engines are containerized.

Prerequisites. Rootless Podman is the supported runtime (Docker works and is probed as a fallback, but rootless podman is what the uid story is built on). On Debian/Ubuntu:

sudo apt install podman uidmap
grep "^$USER:" /etc/subuid /etc/subgid    # must return a range for the DAEMON user
podman info --format '{{.Host.Security.Rootless}}'   # must be true

uidmap (the newuidmap/newgidmap helpers) and the /etc/subuid + /etc/subgid ranges are what make --userns=keep-id work; without them files written inside a container come back owned by the wrong uid on the host. Add the ranges with sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 <daemon-user> and re-run podman system migrate if podman had already been used. Everything below is run as the user the daemon runs as — a rootless store belongs to one user, and two gateways on one host (each under its own account) get entirely separate stores.

Build the image. Nothing runs until the channel image exists; the daemon never builds one inside a turn, because a build takes minutes and would look like a hung answer. npm run setup builds it as part of a fresh install (skip with --skip-image / CG_BUILD_IMAGE=no); by hand:

npm run build:image                        # channelgate/runtime:<spec version> + :latest
npm run build:image -- --cli docker        # force a CLI
npm run build:image -- --no-cache

The build bakes the daemon user’s real uid/gid into the image’s agent user, so build it as that user — building as someone else produces an image whose channels cannot write their own workdir, and building as root is refused outright. CLI versions come from containers/versions.json; bump a pin, rebuild, and each channel picks the new image up on its next turn (the container fingerprint follows the resolved image ID, not the moving tag).

A self-update rebuilds it for you. The transactional updater runs the build itself, after dependencies and before the restart, whenever the candidate changed anything under containers/, bumped imageSpecVersion, or no image is built at all — so a /update checks the image against the checkout’s expected version and source digest. Build or executable-check failures leave an explicit image warning while the host daemon can still restart. New container runs fail closed when the image is missing or stale; an existing image is not a promise that channels remain usable. Run npm run build:image to repair it, or retry Update / Repair (the build is retried even when Git is already current). Active containers retain their current image until idle. A manual git pull still needs a matching image rebuild.

Settings → Container runtime (admin UI):

Setting	Meaning
Container CLI	auto (podman, then docker), or force one
Image reference	default channelgate/runtime:latest
Stop an idle channel container after	minutes, default 10
Max containers running at once	default 8; past it the least-recently-used idle container is stopped
Process / memory / CPU limit	--pids-limit (default 1024), --memory (e.g. 2g), --cpus (e.g. 1.5); blank = no limit
Claude token for container runs	the output of claude setup-token on the gateway host — write-only
Full-access channels see the gateway home	off by default; on = current admin authors in Admin channels mount the gateway user’s home; members keep a separate project container (see below)
Legacy open network (no egress proxy)	off by default (containerEgressMode = "proxy"); on ("bridge") = the pre-proxy behavior: the open bridge network and raw secret values (see Network below)
Withhold readable variables	containerEgressSecretsStrict: ON for a new install, OFF for an install that existed before this default (stored once at boot; your choice is never changed); on = a secret with no egress rule is not given to proxy-mode containers at all, off = it is injected raw and listed as unprotected

Values that would reach the container CLI’s argv are validated at the boundary: a flag, a space or a shell metacharacter in the image/memory/cpu fields is rejected with an error, not silently cleaned.

Engine logins. By default a container Claude run is handed a RELAY of the host user’s own Claude login — its current access token, in CLAUDE_CODE_OAUTH_TOKEN — so keeping claude signed in on the host is all a channel needs. Nothing is copied or mounted. Optionally run claude setup-token on the gateway host and paste the value into Claude token for container runs: that token is then used instead and never needs refreshing. Codex is RELAYED the same way behind the egress proxy (see Codex login relay below): keep the host signed in with codex login; nothing is mounted. Only the legacy open-network mode still bind-mounts the selected auth.json read-write into a container (Codex rewrites it in place, so a copy would fork the refresh chain). Codex sessions and history are per channel either way.

Network (the egress proxy). Every channel container runs with --network none: its only interface is lo. The one way out is the daemon’s egress proxy. cg-init starts a small forwarder (/opt/channelgate/bin/cg-egress.mjs) on 127.0.0.1:3128 inside the container, every HTTP(S) client there is pointed at it (HTTP_PROXY/HTTPS_PROXY and the lowercase twins, NO_PROXY for loopback only, NODE_USE_ENV_PROXY=1), and the forwarder pipes each connection to the channel’s own unix socket — ~/.channelgate/eg/<12 hex>/egress.sock, mounted read-only at /run/channelgate/egress/. The socket path IS the channel’s identity: the proxy never trusts what the container says about itself, and no container can see another channel’s socket. The proxy terminates TLS with the deployment’s egress CA (created once under ~/.channelgate/config/egress-ca/, ca.key 0600 never leaves the daemon) and applies the channel’s policy to every request:

• Allow network off: only the engine endpoints (api.anthropic.com, claude.ai, statsig.anthropic.com, api.openai.com, chatgpt.com, auth.openai.com, a configured Qwen endpoint) and the remote MCP servers this channel’s runs were handed are reachable; anything else answers 403 {"error":"network-off"}.
• Allow network on: any PUBLIC destination. Loopback, private, link-local (the cloud metadata address included), CGNAT and reserved ranges are always refused, checked on every resolved address and pinned for the connect, so DNS rebinding does not help.
• The switch is read on every request: a flip applies to the next connection, no recreate.
• Raw sockets (ssh, psql) get a plain CONNECT tunnel only while the network is on and only to github.com:22 plus the hosts an admin lists in the channel’s egressRawHosts (admin API: PUT /api/channels/<id>/meta with {"egressRawHosts":["db.example.com"]}), on ports 22, 5432 and 6543. A client has to be pointed at the proxy for that (a ProxyCommand); nothing else leaves the container.
• A channel that genuinely needs arbitrary raw sockets can be given the open bridge BESIDE the proxy: {"rawNetwork": true} on the same admin API. The proxy env stays set, so proxy-aware tools still use placeholders, but the switch is then advisory for that channel and every surface says so. Changing it recreates that channel’s container at its next idle moment.

The CA trust bundle. At boot the daemon writes ~/.channelgate/run/egress-ca.pem (0644): the host’s own /etc/ssl/certs/ca-certificates.crt followed by the egress CA, rewritten in place only when its bytes change. It is mounted at /run/channelgate/egress-ca.pem and named by NODE_EXTRA_CA_CERTS, SSL_CERT_FILE, REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, GIT_SSL_CAINFO, PIP_CERT, NPM_CONFIG_CAFILE, CARGO_HTTP_CAINFO, AWS_CA_BUNDLE and DENO_CERT; Chromium (agent-browser) gets --proxy-server plus the CA’s SPKI pin through AGENT_BROWSER_ARGS. A tool that reads none of these fails TLS with an unknown-issuer error — point it at the bundle. Rotating the CA means deleting config/egress-ca/ and restarting; every container picks the new bundle up at its next start.

Secrets through the proxy. A proxy-mode container never holds a channel, organization or personal secret that has an egress rule, nor the relayed Claude login: it holds a placeholder (cgph_…; the Claude token keeps its sk-ant-oat01- shape). The proxy swaps the placeholder for the real value only on that secret’s declared hosts and headers — GitHub, Vercel, Supabase, Make and Composio token names have built-in rules; any other secret is protected once an admin fills Used on hosts in the secrets editor (or passes hosts to set_secret) — and only while the channel has live work (a turn, a background job, a memory review or an SSH session); a personal secret additionally only while its owner is the one working there and NO other person has live work there — another author’s turn, background job or SSH session pauses it (403 another-author-active / another-person-ssh-session) until that work ends. Everything else about a placeholder is forwarded unchanged: a placeholder sent to a host it was not declared for arrives as the useless string it is. When declaring Used on hosts, avoid multi-tenant suffixes such as *.vercel.app, *.herokuapp.com or *.github.io: a wildcard there covers every other customer’s deployment too, and a container could have the real value swapped into a request to a site it controls. Rotation is live (the proxy resolves the current value per request); removing a secret revokes its placeholder, and re-adding mints a new one. A secret with no rule is withheld entirely while Withhold readable variables is on (the default on a new install; the run’s credential note names it as withheld), or injected raw and listed as unprotected in the admin UI and the run’s own credential note while it is off. Either way list_secrets ends with a Finding naming each such secret; declare its Used on hosts (or hosts with set_secret) to give containers a placeholder instead. A secret with no rule is decided by its kind. One that looks like a web API token is hidden: containers get a placeholder the proxy swaps in a credential-like header or query parameter (Authorization, x-api-key, api_key=, token=…), but only on servers an admin approved for it. The first time a program sends it to a new server, the proxy refuses that request (403, nothing sent) and posts an approval card in the conversation’s active thread: Use secret X on api.example.com? Who may click follows the secret’s scope — an admin for the organization’s, anyone working in the conversation for its own, the owner for a personal one (anyone eligible may Deny). The card never expires. Approving remembers the server on the secret, for every later run, schedule and SSH session; it takes effect within five seconds, so the agent just retries. No card is posted when there is no active thread (an SSH session or a background job alone) — approve with allow_secret_host instead. The engines’ own APIs are never approvable. A secret whose name or value shows it is used outside an HTTPS request — a mail or database password, a connection string, a signing key — or that is configuration (an …_ID, …_REPO, …_REGION) stays readable (raw, listed unprotected). set_secret_mode overrides the decision either way. SUPABASE_DB_PASSWORD and other raw-protocol passwords can never be swapped: they stay raw (withheld under strict).

What the proxy refuses on the way. Inside a tunnel the request target must be a path (an absolute-form GET https://other/… line is 400 absolute-form-in-tunnel); a Host header naming another host than the one the connection was approved for is 403 host-mismatch (no domain fronting through an allowed CDN host); a request with no Host gets the approved host and no secret. A request that carried a swapped secret asks the upstream for an uncompressed answer, and the real value is scrubbed back to the placeholder in response headers and in any readable body (text, JSON, or no declared type) — a declared binary body, or one the upstream compresses anyway, passes through unscrubbed. Deadlines: DNS 5 s (504 dns-timeout), at most 16 destination lookups in flight per channel (503 too-many-lookups), 256 open connections per channel socket, connect 15 s, response headers 60 s after the request was sent (504 upstream-timeout). Upstream TLS is always verified, against Node’s bundled roots plus the host’s /etc/ssl/certs/ca-certificates.crt.

After a daemon restart the running proxy-mode containers get their channel listeners back at boot ([egress] restored N running container listener(s)), so a recovered background job, an attached editor or a process left running from an SSH session keeps its network without waiting for the channel’s next turn.

Audit. Every swap of a channel/organization/personal secret, every refusal, every blocked destination and every raw tunnel is an egress event (names, hosts, reasons and byte counts — never a value, a header or a placeholder). Ordinary requests, including the relayed Claude login’s swap on every API call, are only counted; /api/health → containerRuntime.egress shows whether the proxy is up and how many channel listeners it holds.

When the proxy is down. A boot that cannot load the CA or write the bundle logs one line ([egress] proxy unavailable (…)) and carries on; every proxy-mode run, background job and SSH container then fails before spawning with egress proxy unavailable: … Restart the gateway to retry, or … set Settings → Container runtime → Egress to "bridge". A down proxy never silently opens the bridge.

The legacy escape. Legacy open network (containerEgressMode = "bridge") restores the pre-proxy behavior for the whole gateway: containers on the default bridge network, raw secret values in their environment, the switch advisory again. It exists for a host that cannot run the proxy; leave it off otherwise. Switching it recreates each container at its next idle moment.

Admin channels run in containers too. An admin author’s live turn adds the engine’s bypass flag; the channel’s work folder is bind-mounted read-write like any other’s. An admin channel whose work folder is a host directory — the gateway’s own checkout, say — hands that directory, and only that directory, to its container: everything in it is visible there (the checkout’s .env included), nothing beside it is. That is the intended trust model for admin channels; put nothing in such a folder that the channel must not see.

Admin channels can access the host home (Settings → Container runtime, off by default) widens only current organization admins’ runs in Admin/Full-access channels. Those runs get read-write access to the gateway user’s home at its identical path, including repositories, other channels, logs, metadata and credential stores. Container-engine storage stays masked; the grant does not provide host root access or the host’s process namespace.

Non-admin members and guests use a separate project container even when this switch is on. They keep Worker with the channel’s Auto/Lean options and cannot read outside the declared project mounts. The daemon checks the author’s current admin role at each spawn, including background work; revocation removes the grant on the next resolved run. Memory reviews always use the project container. Shared project files and channel memory remain visible to admitted members, so do not put admin-only data in that working folder.

Admin authors retain the legacy container name, HOME and artifact directory, even in Worker channels or with the switch off. Project authors use a fresh cgp2- container and separate HOME, /tmp, /var/tmp and .runtime-project-v2/<platform>/<slug> artifact directory. This prevents old admin transcripts, logins and scratch files from becoming visible to project authors. Switching between lanes starts a fresh engine session without importing admin history. Container recreation preserves each lane’s own volumes. SSH access remains refused for a channel with the optional Admin home grant enabled; this change does not extend that broker’s authority.

Where things live. Containers and the per-channel HOME volumes live in the rootless podman store under the daemon user’s home — ~/.local/share/containers by default; podman info --format '{{.Store.VolumePath}}' prints the exact path. The channel’s work directory is never a volume: it is the same ~/ChannelGate/<platform>/<slug> (or the channel’s custom workdir) bind-mounted at the identical absolute path, so host tooling and VS Code see the agent’s files instantly. Per-run engine-facing files (the settings copy, the MCP config, plugin dirs, job logs) live in ~/ChannelGate/.runtime/<platform>/<slug> for admins and ~/ChannelGate/.runtime-project-v2/<platform>/<slug> for project authors, each bind-mounted at the identical path. Nothing under ~/.channelgate/ is mounted except the channel’s clean workspace and the read-only control-socket directory.

What persists where. The short version: everything a channel accumulates survives, and only running processes do not. A channel container is stopped routinely — after containerIdleMinutes of quiet, and to make room under containerMaxRunning — and recreated whenever its create-time configuration changes (an image rebuild, a limit change). Neither loses data.

Where	Holds	Survives a stop/start	Survives a rm + recreate
Per-channel, per-author-lane HOME volume (/home/agent)	engine sessions and transcripts, CLI logins (gh, vercel, supabase, MCP auth), npm -g, pip --user/pipx/uv/cargo installs, caches, dotfiles	yes	yes — the daemon removes a HOME volume only when the CHANNEL is deleted, never on a rollback, a reconfiguration or an image bump
/tmp and /var/tmp (per-lane volumes <container>-tmp / <container>-vtmp)	scratch files, Claude Code’s per-session scratchpad, anything an agent parks between turns	yes	yes
Channel work directory (~/ChannelGate/<platform>/<slug>, bind mount)	the project itself	yes — it is a host directory	yes
Per-run artifacts (~/ChannelGate/.runtime/<platform>/<slug> or ~/ChannelGate/.runtime-project-v2/<platform>/<slug>, bind mount)	this run’s settings copy, MCP config, job logs	yes	yes
Engine session history (Claude transcripts, Codex rollouts, subagent transcripts)	inside the HOME volume	yes	yes
/run (tmpfs, 64m, noexec)	run-helper pid files, the read-only control socket	no — fresh on every start, deliberately	no
/tmp/codex-daemon-<uid> (tmpfs, 1m, 0700, owned by the agent user)	Codex’s app-server socket, which its sandbox requires on its own user-owned mount	no — a socket needs nothing kept	no
Image layers (/usr, /opt/channelgate, the pinned engines)	read-only and root-owned	yes	replaced by an image rebuild
Foreground/warm engine processes	—	no — a stop kills them; the turn replays	no
Detached background jobs	—	no — a stop kills them, and the next turn says so	no

/tmp and /var/tmp were tmpfs until image spec 1.1.0, which meant the idle reaper’s routine stop emptied them ten minutes after every turn. They persist now, so they keep their contents — and the size cap that came with tmpfs is gone with it: they grow against the disk, just like the channel’s work directory. Since 2026-09-27 they are per-channel podman volumes (<container>-tmp, <container>-vtmp; inspect with podman volume inspect or podman exec <container> ls /tmp), not host folders under ~/ChannelGate/.runtime/<platform>/<slug>/ — those were also visible inside the container through the artifact mount, a mount alias Codex’s sandbox refuses. The daemon deletes them only with the channel, like the HOME volume.

Optional VPN database service. npm run vpn provisions an operator-managed OpenVPN service and isolated database extractor without granting tunnel privileges to ordinary channel containers. See CHANNEL-VPN.md for protected profiles, channel Secrets, systemd persistence, database-only routing and live acceptance requirements.

Installing tools in a channel. An agent can install whatever it needs, and it stays installed — everything below writes inside the per-channel HOME volume:

How	Lands in	Notes
npm install -g <pkg>	~/.npm-global/bin	NPM_CONFIG_PREFIX; first on PATH, so a channel’s own version wins over the image’s
pip install <pkg>	~/.local/bin	PIP_USER=1 + PIP_BREAK_SYSTEM_PACKAGES=1, so Debian’s externally-managed-environment refusal never appears
pipx install <app>	~/.local/pipx, shims in ~/.local/bin	the right choice for a Python CLI with its own dependencies
shell installers piped from curl (uv, rustup, bun, deno, …)	~/.local/bin, ~/.cargo/bin, ~/.bun/bin, ~/.deno/bin	all four are on the image PATH
anything hand-dropped in ~/bin	~/bin	on the image PATH

Codex runs its shell commands through a login shell (bash -lc), and Debian’s /etc/profile resets PATH on the way in; the image re-asserts its PATH from /etc/profile.d/channelgate-path.sh (spec 1.1.1), so both engines see the same install dirs. In a Codex shell ~/.local/bin and ~/bin end up ahead of ~/.npm-global/bin (Debian’s ~/.profile prepends them once more) — a pip-installed CLI with the same name as an npm one wins there, and the npm one wins in a Claude shell. | apt install | — | not available: there is no sudo and the toolchain is root-owned by design, so a channel cannot replace its own engines. Ask the operator to add the package to containers/Containerfile and rebuild |

Inside a hand-made virtualenv, pip install needs PIP_USER=0 (a --user install is not possible in a venv); pipx handles this for itself. If a CLI installs but the shell cannot find it, the container is running an image built before spec 1.1.0 widened the PATH — rebuild with npm run build:image; the daemon logs that mismatch at boot.

Thread /sudo: deliberate direct-host execution. An organization admin may send the typed message @agent /sudo or @agent /sudo on in a Slack channel thread (/sudo as message text in a DM). This is an in-thread bot control, not the conversation-top-level Slack slash-command surface. Subsequent messages in that thread run the selected engine directly as the daemon OS user, with its host filesystem, process namespace, native HOME, installed commands and network—not inside the channel container. Only current organization admins can change or use the thread while sudo is enabled; other senders receive an explicit rejection before work starts. Use /sudo status to inspect it and /sudo off to return to the normal container. Boundary changes are refused while the thread has running or queued work. This is intentionally more powerful than Admin channel mode or the operator-home mount: it removes the container process/filesystem boundary. Treat it as root-equivalent to the daemon account and use it only in trusted operational threads.

The stored sudo flag is not authority. Slack ingress, the run orchestrator, and background launches recheck current admin status; the run API and channel metadata cannot select the host runtime. Host child environments still use ChannelGate’s reviewed allowlist rather than blindly copying all daemon secrets, but a process running as the daemon user can access anything that OS account can.

A thread’s history follows /sudo boundary changes. Claude history (projects/<cwd-key>/<id>.jsonl plus its <id>/ subagent directory) and Codex rollouts (sessions/YYYY/MM/DD/rollout-*-<id>.jsonl) normally live in the channel HOME volume. When sudo is enabled, ChannelGate carries the active session container→host; when disabled, it carries it back. If a copy is unavailable or fails, the existing transcript-healing path starts a fresh native session while replaying the visible conversation. Stopping, starting or recreating a container still preserves its HOME volume.

Inspect and debug.

Attach VS Code to the same channel container. Install VS Code’s Dev Containers extension; when the gateway uses Podman, set dev.containers.dockerPath to podman. From the gateway checkout as the daemon’s Linux user, run:

npm run vscode -- <channel-id-or-slug-or-exact-name>

The helper starts the managed container if necessary and opens the channel’s real mounted workdir, not a newly created dev container. Keep the helper terminal open: code --wait holds a signed editor lease until that VS Code window closes, preventing idle or capacity eviction. The attached terminal is user agent with the same persistent /home/agent, so its installed tools, GitHub and provider CLI logins, Claude history, and Codex history are the channel’s own existing state.

The VS Code server itself is not downloaded into the channel. The image carries one read-only copy of each pinned version (containers/versions.json → vscodeServers, currently 1.139.0 and 1.138.0) under /opt/channelgate/vscode-server, and every container start links it into ~/.vscode-server — the attach layout (bin/<commit>) and the Remote-SSH layout (cli/servers/Stable-<commit>/server, code-<commit>) alike — so an editor attach costs the volume a few kilobytes instead of ~650 MB. A client on a version that is not pinned downloads its own server into the volume exactly as before. To bump: add the new release’s commit and VS Code’s published sha256hash values from https://update.code.visualstudio.com/api/versions/commit:<commit>/server-linux-x64/stable and …/cli-alpine-x64/stable, drop the oldest, and rebuild; a mismatched download fails the build. A channel opts out with touch ~/.vscode-server/.cg-no-shared-server inside its container. Servers a client downloaded into a volume before this are left where they are — they show up in the storage report as reclaimable, and nothing deletes them automatically.

Container storage

Nothing reclaims container storage automatically, on purpose: every image build keeps the previous runtime image, the idle reaper stops a channel’s container without removing it, and a stopped container keeps the image it was created from. Check it with the report, which changes nothing:

npm run runtime:storage              # what could be reclaimed, and why each item is kept or not
npm run runtime:storage -- --json    # the same report as JSON
npm run runtime:storage -- --apply   # remove exactly the lines the report marks `remove`

Run it as the gateway’s own OS user (rootless container storage is per user). It removes a stopped channel container only when it was created from a superseded image (the channel gets a fresh one on the current image next time), never a running one; runtime images other than the current one, the previous spec (--keep-previous <n>, default 1) and any still in use; untagged leftovers; and anything named for another install only when every folder that install’s containers mounted is gone — a finished test run, not a second live gateway sharing this account. A channel’s home volume is never removed by it. The space estimate counts image layers once each. To make it routine, schedule the report and read it; schedule --apply only if you have decided to.

Codex uses the channel’s selected login, relayed in proxy mode. Claude’s rotating credential file is still never copied or mounted: the helper refreshes the gateway’s normal subscription access-token relay every 20 minutes and exposes only that access token to interactive claude commands. Closing VS Code removes the live token and releases the lease; an interrupted helper is detected by PID start identity and its stale lease is discarded automatically. A daemon configured only with ANTHROPIC_API_KEY cannot currently export that key to an interactive editor terminal; use the normal operator subscription login or a configured claude setup-token for this workflow. The editor lease is held in daemon-owned metadata, outside the writable container artifacts.

This is deliberately an operator command, not a remotely callable channel tool: VS Code provides a full shell inside the container and bypasses chat tool presets. The container boundary remains the same—only this channel’s mounts exist, there is no sudo, and no gateway database, host home, or other channel is exposed. Per-channel environment secrets remain write-only and are not exported to the editor terminal.

Let developers SSH into a channel container (docs/SSH-ACCESS.md). Unlike the operator’s local VS Code attach, this is for people WITHOUT a host account: a one-time root installer (scripts/install-ssh-access.sh) creates a no-shell login account whose forced command hands each connection to the daemon, which authorizes the registered key against the channel’s SSH grant list and runs an unprivileged sshd -i inside the container on that stream. Keys and grants are managed from chat (add_my_ssh_key, grant_channel_ssh, show_channel_ssh); a container with a live session is never idle-stopped; sessions and refusals are in the Audit feed.

podman ps --filter label=channelgate=1                 # every ChannelGate container on this host
podman ps -a --filter label=cg.install=<install id>    # only THIS gateway's (see /api/health)
podman exec -it <container> bash                       # a shell in a channel, as the agent user
podman logs <container>                                # cg-init output
podman volume ls | grep -- -home                       # the per-channel HOME volumes

/api/health (authenticated) carries a containerRuntime block: the CLI kind, version and rootlessness, whether the image is present, the control socket, and every running container with its lease count and idle time. /status in a channel names its backend, container, image, state and uptime. An attached podman exec terminal inherits the channel’s environment, not the daemon’s.

Rollback. One lever: roll the release back through the transactional updater. It deletes nothing — HOME volumes are removed only when a channel is deleted, never on a rollback or a reconfiguration, and the two temp trees are host directories the daemon never touches, so the restored release finds every CLI login and every scratch file where it was left. There is no per-channel or gateway-wide “back to the host” switch: the container is the only place a turn runs.

Known caveats (v0.8 P1).

• Per-user Codex skill grants are not delivered in containers. The per-run Codex skill overlay was built for a synthetic host HOME that a container does not have; a Codex run gets the channel’s skills through the mounted workdir, but not that overlay.
• Codex sessions are per channel; the sign-in is mounted only outside the proxy. Under the legacy open-network mode each container mounts its selected host auth.json. A codex login on the host that replaces the file leaves a running container holding the old inode — /status and /api/health report the drift; restart the channel’s container (or let the reaper stop it) to pick the new one up. Behind the proxy the login is relayed and a new codex login takes effect on the next turn.
• The relayed Claude login is a placeholder in proxy mode. A proxy-mode container’s CLAUDE_CODE_OAUTH_TOKEN is the channel’s relay placeholder, swapped for the current access token only on api.anthropic.com while the channel has live work. Under the legacy bridge mode it is the real access token again, readable by the channel’s own agent (it cannot rotate anything and dies within hours). A daemon that authenticates Claude with its own ANTHROPIC_API_KEY still passes that key through raw — the proxy does not rewrite it.
• Engine keys that still reach a proxy-mode container raw. A Qwen harness’s provider key (ANTHROPIC_AUTH_TOKEN pointed at the provider) still reaches that engine as a real value. Codex logins stored in auth.json, including API-key logins, are relayed behind the proxy.
• A self-hosted Qwen endpoint on a private address is refused in proxy mode. The proxy never connects to loopback, private, link-local or CGNAT addresses, and a configured Qwen base URL is no exception: point the harness at a public endpoint, or run that channel under the legacy bridge mode.
• Egress is policed per channel by the proxy; the switch is advisory only where the proxy is not the network. Under containerEgressMode = "bridge" or a channel’s rawNetwork escape the container has the open bridge and Allow network only tells the engines the channel’s intent; every surface then says so — /mode and /status add “advisory — not enforced for this container”, the gateway-managed block at the top of each CLAUDE.md and the per-attempt note tell the engine, and run_config records networkEnforced: false with egress: "bridge" or "proxy+raw". In proxy mode run_config records networkEnforced: true, egress: "proxy", and the names of any unprotected (egressUnprotected) or withheld (egressWithheld) secrets. networkEnforcedFor(target) in src/engines/network-policy.js is the one question every surface asks.
• Codex’s sign-in is a placeholder in proxy mode. See Codex login relay below. The legacy bridge mode still mounts the selected auth.json file because it has no proxy swap.
• SSH and VS Code sessions run in the same --network none container with the proxy env and hold placeholders like a turn (container-secrets P3, docs/SSH-ACCESS.md); SSH -L forwards to external hosts do not work without the network.

Codex login relay (container-secrets P4). Behind the egress proxy a channel container never sees the host’s Codex login. Before each Codex run (and when an SSH session is prepared) the gateway writes /home/agent/.codex/auth.json into the channel’s own HOME volume: the access token is the channel’s relay placeholder shaped like a JWT (the real token’s claims, cgph_r… in place of the signature — the CLI reads the claims, the provider checks the signature), the refresh token is empty, last_refresh is now. The proxy replaces that whole token with the host’s current access token in the Authorization header on api.openai.com, chatgpt.com and auth.openai.com only, while the channel has live work. The container can use the login but cannot renew, rotate or take it anywhere: sent elsewhere, or from another channel, it is worthless; a refresh attempt from inside is rejected by OpenAI (empty refresh token). The host’s login is renewed by the gateway itself: when its access token (10-day lifetime) has less than 48 hours left, the next Codex run first spends one cheap codex exec --ephemeral --ignore-user-config turn (gpt-5.6-luna, low effort) in the login’s own CODEX_HOME — the engine home when it holds a login, else ~/.codex — exactly what your own shell does. If the CLI declines to renew a still-valid token the gateway waits 6 hours before trying again; a failed refresh logs [codex-relay] host refresh turn exited …. What to watch: a Codex turn that fails with “This channel runs in a container, but … Run codex login” means the host is signed out or its token expired and could not be renewed — run codex login on the host. The first Codex turn after upgrading recreates each channel’s container once (the mount is gone from its fingerprint), and cg-init deletes an old copied Codex login carrying a refresh token from the volume. The legacy open-network mode keeps the shared read-write mount (and says so in /status).

A separate Codex login for one channel. In the admin channel editor, Runtime → Codex login source → This channel’s own login. The choice comes before the engine and model settings. Default gateway login shows the shared Slack, Claude and Codex status, with links to the gateway Slack/Claude settings and direct ChatGPT or API-key sign-in for shared Codex. Slack and Claude currently use the gateway connection regardless of the Codex choice. Choose This channel’s own login to show the dedicated Codex controls; channel connector overrides remain in MCP Connections and Environment tokens.

Use Sign in with ChatGPT to start a device code flow in the admin page; open the displayed link, enter its code, and wait for the status to show the completed ChatGPT login. Device code sign-in may first need enabling in ChatGPT security settings or workspace permissions. Alternatively, enter a Platform key in the private password field and choose Use API key. A successful sign-in selects and saves This channel’s login. The page shows only the method and sign-in status after completion; it never returns the key.

For a terminal fallback, the editor shows that channel’s Codex home directory on the gateway host. Create it under the gateway service account and sign in there. For ChatGPT subscription access, use CODEX_HOME=<displayed directory> codex login --device-auth and complete the browser code flow. For an OpenAI Platform API key, use CODEX_HOME=<displayed directory> codex login --with-api-key, supplying the key on standard input as prompted by the CLI. Never put the key on the command line, in Slack, or in the channel folder. CODEX_HOME=<displayed directory> codex login status checks the selected method. OpenAI bills API-key runs separately from a ChatGPT subscription. The channel login does not fall back to the gateway login if it is missing or expired.

For a laptop login, OpenAI documents copying ~/.codex/auth.json to a headless host as a fallback when device login is unavailable. Use an encrypted SSH transfer into the displayed host directory with restrictive permissions and run Codex as the gateway service account so the one host-side file can refresh. The file contains full credentials, including a refresh token: do not upload it through Slack or the admin UI. Channel containers receive only a channel-bound placeholder. A ChatGPT login is relayed as the JWT-shaped access token described above; an API-key login is relayed as a separate placeholder swapped only in the Authorization header at api.openai.com. The channel setting affects new Codex turns; it does not change Claude authentication. An Admin channel with the optional full operator-home mount can read host files under that mount, including the gateway’s runtime root; use ordinary project channels for credential isolation.

Remote MCP relay (container-secrets P1). Composio (composio-user, composio-agent in token mode), the MakeItFuture toolbox and the Make toolbox never reach a container with their token. The engine’s MCP entry is the image’s socket bridge naming the server (CG_MCP_SERVICE=remote-mcp); the run’s signed capability lists those names, and the daemon keeps the real URL and header in memory (never on disk, never logged) for no longer than the capability’s lifetime — 6 hours for a turn, 12 for an SSH session — and normally only while something uses it: a cold turn’s grant ends with the turn, a warm Claude process’s when the process retires, an SSH session’s at its end, and a person clearing their Composio or Toolbox token drops every grant minted for their runs at once. On a remote-mcp hello the daemon checks the claim and its own registration, dials the service (https only; Streamable HTTP, falling back to HTTP+SSE on a 4xx) and relays the tools. What an operator sees: the artifact dir’s cg-mcp-*.json and Codex’s run/cg-codex-secrets-*.json hold no Composio or toolbox token (Codex’s bundle holds only the capability), and a relay that fails shows in the engine’s MCP log as one fixed line — remote MCP is not authorized for this run (the grant expired or was revoked, or the daemon restarted since the run started: in-memory registrations do not survive a restart, so the next turn simply registers again), too many remote MCP connections for this run (more than 8 at once for one server on one grant) or remote MCP unavailable (the service could not be reached; the upstream error is deliberately not repeated, and a failed tool call reads remote MCP request failed). the gateway service takes no arguments means the channel image predates spec 1.6.0. A COMPOSIO_MCP_URL/TOOLBOX_MCP_URL override pointing at plain http cannot be relayed: container runs skip that server and say so in the thread. The relay removes the credential from the box; the egress proxy (above) is what polices where the box can connect. It needs image spec 1.6.0 (npm run build:image), which also ships the egress forwarder. Direct-host /sudo threads keep the old shape: the engine dials the service itself.

Claude login in containers: with no containerClaudeOauthToken, each container Claude run receives a RELAY of the gateway’s resolved login — normally the host user’s own ~/.claude — as a current OAuth access token in CLAUDE_CODE_OAUTH_TOKEN (refreshed on the host first by a cheap haiku turn, in that login’s own config dir, when under 30 minutes remain). The login file is never copied: Claude Code rotates refresh tokens, and a copy that refreshes logs the original out. A relayed access token cannot rotate anything. A claude setup-token value, when configured, is used instead and needs no refresh.

Service unit: the installer sets KillMode=mixed. systemd then sends SIGTERM only to the daemon, which drains, marks the shutdown and sweeps its own engine children (the container run groups) so interrupted execution is recorded for reconciliation on the next boot. With the default control-group mode systemd signals the engine (or the podman exec client) directly and the turn may be recorded as a plain error before orderly state persistence. Existing installs: add KillMode=mixed under [Service] and systemctl --user daemon-reload.

System health

Open System health, the last admin navigation item (/system-health). The host daemon reads Linux /proc and /sys without elevated privileges or subprocesses. It samples every five seconds even when no browser is open; browsers poll while visible and active. Pausing the page does not stop background collection. Refresh also rescans hardware. CPU uses counter deltas (the first sample is unknown), RAM uses MemAvailable, and load is the Linux 1/5/15-minute load average. Storage measures the filesystem containing the configured gateway runtime root, not the sum of disks in the hardware inventory. Used bytes exclude free blocks; available bytes exclude filesystem-reserved blocks; the usage percentage is used / (used + available), as for an unprivileged process. References: Linux proc documentation and Node filesystem statistics.

Resource minute averages and peaks stay for 30 days. Storage stays for 186 days (at least six calendar months); samples older than 30 days are reduced to hourly observations with preserved peaks. Responses reduce long resource series to about 240 buckets and storage to daily points. Empty history and missing metrics are reported without simulated values. Forecasts require at least seven days of measured history and use only the segment since the last observed capacity change. The estimate assumes continued growth at the fitted rate, not a guarantee of free space.

Hardware is read at startup, every five minutes and on refresh. The current snapshot and at most 100 change events are retained; no temperatures, serial numbers, MAC addresses or machine IDs are collected. DIMM information and some device details may be unavailable without host interfaces; the page reports unknowns. RAM/disks recognized by Linux appear at the next scan. Distribution changes appear on rescan; a newly installed kernel appears only after booting it. Hardware details remain behind the admin session and never appear on the public liveness endpoint.

Metrics use tables in the existing SQLite database. Retention applies only to these tables, not audit/usage records. Do not impose a 100 MiB limit on the entire gateway database or force global WAL truncation: other gateway records and readers share it. Measure incremental metrics storage using an isolated canary, with 100 MiB for its database plus WAL as a safety budget.

Before enabling on a served host, run the canary on that Linux host from the tested checkout:

node scripts/system-health-soak.mjs --output /absolute/new-health-soak-report.json --storage-path /path/to/gateway-runtime-root

It runs in the foreground for 24 hours, reads the target filesystem, and writes metrics only to a fresh temporary database. The JSON report updates every 30 seconds and records actual elapsed time, errors, gaps, CPU, peak RSS, database plus WAL bytes, hardware visibility and persistence after reopening. The report path must be new; the scratch database is removed at completion. Use the service manager or a daemon-owned job to keep it alive when a session cannot remain open. --duration-seconds 65 is a quick smoke; it explicitly cannot pass the 24-hour gate. A report with containerDetected: true describes the container’s visible interfaces and is not evidence of a host deployment. Even when false, independently verify where the process ran.

Review the measured overhead rather than treating the prototype estimates as guarantees. After the host canary passes, use the normal serialized beta landing and safe restart procedure, log in, and verify current samples, hardware visibility and history persistence on the actual daemon.

Retention and log rotation

Run npm run maintenance daily from the service manager. CG_RETENTION_DAYS defaults to 30 and removes expired backup files; CG_MAX_LOG_BYTES defaults to 10 MiB and retains one rotated copy. Active logs are rotated by copy-truncate, not rename: systemd holds the daemon’s stdout/stderr file descriptor open, so a renamed file would keep growing under its new name and the size cap would never apply to the live log again. Console output is centrally redacted for Slack/OpenAI-style tokens, bearer values, and secret query parameters before systemd (or the journal) receives it. Database audit/usage retention is deliberately not automated: legal and operational owners must define it before enabling destructive row pruning.

Upgrade and rollback canary

The transactional updater’s phases are: preflight → snapshot → checkout → install → audit → test → provision → channel image → restart → verify, with an automatic rollback (restore → install → restart → verify) on any failure up to the restart. The channel-image phase is the single exception to that rollback: it only runs when the image is provably behind this revision, and a failure there is reported and stepped over rather than rolled back (see Container runtime).

Before promotion: run static checks, the complete suite, npm run backup, npm run restore:drill, then npm run release:artifacts. Deploy one canary service, verify /api/health, Slack mention/DM, both enabled engines, approval gating, and backup restore. The transactional updater automatically returns Git/dependencies to the prior revision after a failed health check; runtime data snapshots are never restored automatically. Keep the previous release artifact and backup until the canary has run for 24 hours.

The admin Restart button picks its exit code from the detected service manager (nonzero under the unit’s Restart=on-failure, clean 0 for an unmanaged foreground run). Nothing in the test suite can prove a real systemd relaunch — that stays a manual canary step, and a follow-up if these paths ever get an automated harness: press Restart on the canary and confirm the daemon comes back with a new instance id.

License keys

The daemon verifies its license key at boot and about every 24 hours, and reports usage counts daily and once at shutdown. Both are fire-and-forget: neither can delay a boot, a turn, or an exit, and neither can kill a run. Full model in LICENSE-KEYS.md; exact payloads in PRIVACY-AND-DATA-FLOW.md.

Day-to-day. Settings → License shows the tier, the key’s last four characters, the last verification, the next scheduled check, a banner for any non-quiet state, and this month’s per-conversation usage against the limit. Verify now forces a check. In chat, get_license_status answers the same question for anyone allowed in the channel.

What each state means for an operator.

Banner	What happened	What to do
(none)	valid, or no_key on a deliberately unlicensed install	nothing
offline grace	the platform has been unreachable for under 14 days; the last verified tier still applies	check egress to CHANNELGATE_PLATFORM_URL; it will heal itself when the network does
grace expired	unreachable for over 14 days	the tier is kept until the start of the next UTC month, then the no-key limits apply — fix connectivity before then, or ask for an offline payload
key rejected / revoked	the platform returned 401 / 403	the no-key limits are already in force; install a valid key

Egress. The only host the daemon itself contacts is CHANNELGATE_PLATFORM_URL (https://channelgate.dev by default) over HTTPS, with a 10-second timeout. A deployment that must originate no outbound traffic at all runs on CHANNELGATE_LICENSE_PAYLOAD (an offline signed license) and makes no request.

Backups. The cached verification and the installation id live in _meta in gateway.db, and this month’s counters in license_usage — both are covered by the normal encrypted config backup. Restoring an older database restores an older cached verification; the next check refreshes it. Deleting the _meta installation-id row simply mints a new random one.

Troubleshooting a refused conversation. A refusal is always a visible reply in the thread, and it is also logged: license_run_refused in the event log carries the conversation, the reason (conversation_limit or monthly_cap), the UTC month, and the run count. Cross-check it against Settings → License → Usage this month.

Verifying and recovering a container toolchain update

The image carries a digest of its container build sources and its pinned CLI versions. Update compares this with the desired checkout even when no Git revision changed. A failed image build is retried by the next Update; a zero exit with stale labels is reported as a failure. Custom image references are operator-managed and are never silently replaced with the default image.

Container runtime status displays built/desired Claude and Codex versions, whether a rebuild is needed, and how many existing containers await adoption. Active turns keep their image until idle. Update results retain image-build warnings even when the daemon update succeeds.

Update runs directly on the gateway host through bash scripts/update.sh. The managed button starts it in an independent systemd user service so it survives the daemon restart. Provider logins, token relays, quotas and engine responses are not installation prerequisites. Git safety, recovery snapshots, exact dependency installation, security audit, regression checks, provisioning, image drift/build checks, graceful restart and authenticated daemon readiness still apply. The same complete repair runs when the checkout is already current. Node, Claude and Codex executables are checked using --version in a temporary network-off, read-only container without host mounts or credentials. Image build failures remain explicit warnings and are retried next time. The restart signal drains active work: SIGUSR2 exits with the restart code for the shipped Restart=on-failure unit; SIGTERM supports Restart=always and on-success. Readiness allows ten minutes for large skill catalogs to reconcile. The isolated engine response probe remains available as a separate diagnostic.