Celln Backend (Hermetic Execution)¶

Sympozium optionally integrates with Celln to run a single bounded, high-risk, or sensitive computation in a hardware-isolated microVM instead of a Kubernetes Job. It is selected per run with spec.backend: "celln" on an AgentRun.

Celln is opt-in: celln.enabled=false by default in the Helm chart, because the installer DaemonSet it deploys runs privileged, with hostPID, and a read-write mount of the host root filesystem — necessary to set up KVM on the host, but a materially different trust boundary than the rest of Sympozium's pods. Enable it explicitly with sympozium install --enable-hermetic-workloads (or helm upgrade --set celln.enabled=true). This page covers what that turns on, and the one requirement that's easy to miss: Celln needs its own AI provider access on the host, separate from whatever provider your Agent/AgentRun is configured with.

The web UI's Runs page shows a live banner if any listed run uses backend: celln while the router is unreachable, and the New Run dialog checks live reachability when you select the Celln backend — both backed by GET /api/v1/capabilities.

What Celln is (and isn't)¶

Celln runs one task in a sealed KVM cell and returns a bounded text result. It deliberately does not support ensembles, delegation, shared memory, IPC, NATS, streaming, or sub-agent spawns — anything that needs those capabilities must use the standard job backend (or agentSandbox; see Agent Sandboxing). Use it for individual computations you'd rather not run un-sandboxed: parsing untrusted input, running generated code, one-off risky operations.

apiVersion: sympozium.ai/v1alpha1
kind: AgentRun
metadata:
  name: risky-computation
spec:
  agentRef: my-agent
  task: "Parse this untrusted file and summarize its structure"
  backend: celln
  timeout: "5m"

How It Works¶

AgentRun (backend: celln)
  │
  ├─ Controller POSTs {id, task, timeout} to CELLN_ROUTER_URL
  │   (celln-router.celln-system.svc.cluster.local:8787)
  │
  ├─ Router (one pod per KVM node) forwards to the host-level
  │   celln dispatcher — a systemd service on that node, installed
  │   by the celln-installer DaemonSet
  │
  └─ Dispatcher asks its configured AI provider to write a program,
     attests and seals it into a real KVM cell, runs it, and returns
     the bounded output. status.cellnActionId tracks the poll.

The installer and router only schedule onto nodes labeled celln.dev/kvm: "true" — Celln needs /dev/kvm and is not a container-level isolation mechanism, so it can't run on arbitrary nodes the way the job backend can.

Trust model: your task always runs in the agent lane¶

Celln draws a hard line between two authority levels for code running inside a cell: the tool lane (an already-attested host binary, sealed in read-only — full but narrow authority) and the agent lane (agent-authored code, generated at run time — gets only what's explicitly loaned to it, permanently, no matter how well it's built). Full model: tool lane / agent lane.

A Sympozium backend: celln AgentRun always executes in the agent lane. The controller always sends the task as a forge request — a model writes a program from the task string, it gets rebuilt twice and hash-compared — and that program is author=agent by construction, which is agent-lane authority regardless of how cleanly it reproduces. There is currently no AgentRun field for naming a pre-declared, hash-pinned tool instead of a task string, so tool-lane execution isn't reachable from Sympozium today — only from the celln CLI directly (celln spec / celln run). Practically, that means every backend: celln run gets: no ambient host tools or filesystem beyond its own generated program and workspace, no persisted workspace between runs (workspace: "none"), a fixed 256MiB/64KiB memory/output envelope (not yet configurable per run), and hardware isolation with no softer fallback. See Sympozium × Celln actions for the full breakdown of what the dispatched request actually contains.

Enabling / Disabling Celln¶

sympozium install --enable-hermetic-workloads
# or: make install ENABLE_HERMETIC_WORKLOADS=true
# or: helm upgrade --install sympozium charts/sympozium/ --set celln.enabled=true ...

# values.yaml — the master switch, false by default
celln:
  enabled: false

When false (the default), no celln-system namespace, installer, or router is deployed, and the controller/apiserver aren't given a router URL — zero footprint.

Disabling it does not remove "celln" as a valid backend value on the CRD. A run submitted with backend: celln while disabled will still be admitted; the controller will attempt to reach the router at the default in-cluster DNS name, fail to resolve it, and the run transitions to Failed with a router-unreachable error. If you disable Celln after enabling it, communicate that to whoever authors AgentRuns or agent defaults that set backend: celln — or watch for the live banner on the Runs page, which flags exactly this case.

Enabling Celln: the AI provider requirement¶

This is the part that's easy to miss: Celln's AI provider is configured independently of your Agent/AgentRun's model: field. A celln-backed run's task string goes to whatever provider is configured on the KVM host — the run's own model.provider/model.name are not passed through and are ignored for this backend.

The host-level dispatcher (not the Sympozium controller) needs one of:

An API key, set via Helm — mounted into the celln-installer DaemonSet as a Secret, and written to /etc/celln/agent-key on the host for the dispatcher to read:
```
celln:
  anthropicApiKey: "sk-ant-..."   # needs the `claude` CLI on the host
  # openaiApiKey: "sk-..."        # needs the `codex` CLI on the host
  # deepseekApiKey: "sk-..."      # no CLI needed, plain API calls
  # openaiBaseUrl: ""             # optional, for an OpenAI-compatible proxy
```
Set one of these. anthropicApiKey/openaiApiKey still require the corresponding CLI (claude/codex) to actually be installed and authenticated on the KVM node — the key alone isn't sufficient if the CLI is missing.
A locally running ollama with a model already pulled, and no key set at all. The dispatcher auto-discovers it.
Nothing set — the dispatcher searches the host for any authenticated CLI (codex, claude, deepseek-api, ollama, in that order) at startup and uses the first one it finds.

If none of the above is true on a given KVM node, that node's dispatcher is still installed and healthy from Kubernetes' point of view (the router's health check only verifies /dev/kvm and non-empty tool/mote stores, not provider availability) — the failure only surfaces when a task is actually dispatched, as an AgentRun Failed status with the provider's own auth error (e.g. "claude has no saved login and ANTHROPIC_API_KEY is not set — authenticate it or set a key").

Graceful Degradation¶

Scenario	Behavior
`celln.enabled=false`	No `celln-system` namespace or resources. Runs with `backend: celln` fail at dispatch with a router-unreachable error, not at admission.
`celln.enabled=true`, no node labeled `celln.dev/kvm=true`	Installer/router DaemonSets deploy with zero pods scheduled. Runs fail the same way as above — nothing is listening at the router URL.
`celln.enabled=true`, KVM node(s) present, no AI provider reachable on the host	Router and dispatcher report healthy. The run reaches `Running`, then fails once the dispatcher's own provider check fails — see above.
Everything configured	Run dispatches, executes in a real sealed cell, and returns a bounded result.