Skip to content

Helm Chart

For production and GitOps workflows, deploy the control plane using Helm.

Prerequisites

cert-manager is required for webhook TLS:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.17.1/cert-manager.yaml

Install

Sympozium ships as two charts: sympozium-crds (the CRDs) and sympozium (the control plane). The CRDs live in their own chart so helm upgrade can roll schema changes forward — Helm 3 never touches files under a chart's crds/ directory on upgrade. Install the CRDs first, then the control plane:

helm upgrade --install sympozium-crds ./charts/sympozium-crds \
  --namespace sympozium-system --create-namespace

helm upgrade --install sympozium ./charts/sympozium \
  --namespace sympozium-system \
  --skip-crds --set createNamespace=false

Both charts are kept in lockstep. Always upgrade sympozium-crds before sympozium.

--skip-crds on the second command assumes the sympozium-crds release is installed. If you choose to use only the sympozium chart, omit --skip-crds so the CRDs bundled in that chart are applied — but you will then forfeit the ability to roll CRD schema changes forward via helm upgrade.

Uninstall ordering. Removing sympozium-crds cascade-deletes every Agent, AgentRun, SkillPack, Ensemble, SympoziumPolicy, etc. across all namespaces. Always helm uninstall sympozium first, then helm uninstall sympozium-crds.

The legacy single-chart install (helm install sympozium ./charts/sympozium) still works for fresh clusters that will never need a CRD upgrade.

See charts/sympozium/values.yaml for all configuration options. The sympozium-crds chart has no configurable values.

Celln (hardware-isolated execution)

The raw chart default is celln.enabled: false; sympozium install turns it on (and --no-celln leaves it off). Every Celln workload schedules only on nodes labelled celln.dev/kvm=true (except the single in-cluster dispatcher, below); the node probe applies that label to nodes with /dev/kvm and a kernel under /boot (nodeProbe.labelKVMNodes). See Celln Backend.

  • celln.fleet.* (the default when sympozium install finds a model backend) runs one owner per labeled node: a celln-node DaemonSet prepares each node from a digest-pinned, signed starter package and serves one-shot and enduring parents; a celln-node-configure DaemonSet configures each model backend; the router discovers owners through a headless Service; and the controller keeps its parent journal on a claim instead of a node. Its values (scope, package, publisher, backends, capacity, HTTPS hosts) are set by the installer; driving them by hand is possible but see Celln Fleet Installation for what must exist first.
  • celln.dispatcher.* is the single in-cluster dispatcher used by the one-shot router when no fleet is configured. It carries no node selector but mounts the node's /dev/kvm, so the node it lands on must provide KVM. celln.dispatcher.tolerations lets it run on a tainted KVM node (for example a single-node cluster's control plane).
  • celln.installer.enabled deploys the legacy privileged host-installer DaemonSet (bare-metal systemd dispatcher, labelled nodes only); it is mutually exclusive with the in-cluster dispatcher. Only this path reads celln.anthropicApiKey, openaiApiKey, deepseekApiKey and openaiBaseUrl.

Disable all Celln resources with --set celln.enabled=false.

Energy collector (ergoz)

energyCollector:
  enabled: true
  discovery:
    namespaces: [sympozium-system, ergoz-system]

Any Service labelled sympozium.ai/collector=energy in one of the listed namespaces is used for accelerator power draw (GET /api/v1/power, shown on the density and topology views). The namespace list is a security allowlist: only list namespaces an administrator controls. sympozium install deploys the pinned ergoz release (config/ergoz/release.json) into ergoz-system as a separate Helm release; --no-ergoz skips it, and a failed ergoz install never fails the Sympozium install. With plain Helm, install ergoz yourself if you want it.

Pricing and memory

pricing:
  enabled: true        # ship the sympozium-model-pricing ConfigMap; cost estimates on AgentRuns
  extraEntries: []     # extra/overriding price entries (provider, match, inputPerMTokMicro, outputPerMTokMicro)
memory:
  adminDelete:
    enabled: true      # admin-only DELETE /delete on memory servers, bearer token never given to agents

Cost estimates are display-only and never gate runs; local providers are always exempt.

Model gateway

modelGateway.enabled (default false) packages the separate Celln model gateway for controlled qualification. It requires an immutable image digest, a configuration claim and explicit egress peers, and does not by itself enable mediated Celln admission. See Model gateway component installation.

AgentHarness examples

The chart installs only the maintained, experimental persistent Pi and Hermes adapter catalog entries by default, alongside the built-in Ensemble catalog. They appear in Agents → Harnesses when the UI is set to the chart namespace (normally sympozium-system). Each is pinned to an immutable digest and carries its conformance reference.

The chart also creates a harness-examples policy that permits only these two images. It does not create or modify an Agent, so no external harness runs until an operator explicitly selects both that policy and a runtime on an Agent. The session adapters persist their own conversation state on a per-session PVC and are currently unmetered; they do not support MCP/SkillPack tools, native tool filtering, personas, or subagents. One-shot Pi and Hermes runtimes remain available in values as explicit opt-in examples and are hidden from the default interactive catalog.

Disable the catalog, or select only one adapter, with values such as:

harnessExamples:
  enabled: false
  # Or retain the persistent catalog but install Pi only:
  piSession:
    enabled: true
  hermesSession:
    enabled: false

Observability

The Helm chart deploys a built-in OpenTelemetry collector by default:

observability:
  enabled: true
  collector:
    service:
      otlpGrpcPort: 4317
      otlpHttpPort: 4318
      metricsPort: 8889

Disable it if you already run a shared collector:

observability:
  enabled: false

Web UI

apiserver:
  webUI:
    enabled: true       # Serve the embedded web dashboard (default: true)
    token: ""           # Explicit token; leave blank to auto-generate a Secret

If token is left empty, Helm creates a <release>-ui-token Secret with a random 32-character token.

Node Probe (inference provider discovery)

The node-probe DaemonSet discovers inference providers (Ollama, vLLM, llama-cpp, LM Studio) installed directly on cluster nodes. It probes localhost ports and annotates nodes so the web wizard can offer model selection and node pinning. It also labels KVM-capable nodes celln.dev/kvm=true for Celln; a label an operator set (either value) is never changed.

nodeProbe:
  enabled: true           # Deployed by default on all nodes
  labelKVMNodes: true     # Label nodes with /dev/kvm and a /boot kernel celln.dev/kvm=true
  config:
    probeInterval: 30s
    targets:
      - name: ollama
        port: 11434
        healthPath: /api/tags
        modelsPath: /api/tags
      - name: vllm
        port: 8000
        healthPath: /health
        modelsPath: /v1/models
      - name: llama-cpp
        port: 8080
        healthPath: /health
        modelsPath: /v1/models
      - name: lm-studio
        port: 1234
        healthPath: /v1/models
        modelsPath: /v1/models
  resources:
    requests:
      cpu: 50m
      memory: 32Mi
    limits:
      cpu: 100m
      memory: 64Mi
  tolerations:
    - operator: Exists    # Run on all nodes including GPU-tainted nodes
  nodeSelector: {}        # Limit which nodes run the probe

The DaemonSet uses hostNetwork: true to probe localhost on the host. It runs as non-root with a read-only filesystem. Its RBAC is minimal: nodes: [get, patch].

See the Ollama guide for full details on node-based inference.

llmfit DaemonSet (hardware density telemetry)

The llmfit DaemonSet runs on every node, continuously reporting hardware specs and model density scores. The controller and API server poll each pod to build a cluster-wide FitnessCache that powers instant model placement, the Model Density UI, Prometheus metrics, and the density API.

llmfit:
  daemonset:
    enabled: true           # Deployed by default
    eventInterval: 60       # Seconds between fitness polls
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 200m
        memory: 256Mi
    tolerations:
      - operator: Exists    # Run on all nodes including GPU-tainted
    nodeSelector: {}
  liveEviction:
    enabled: false          # Re-place models when fitness degrades
    checkInterval: 30s
    degradeThreshold: 0.3
  webhook:
    preflightValidation: false  # Reject Models that won't fit

The DaemonSet mounts /proc, /sys, /dev, /run/udev read-only for hardware detection. It runs as non-root with readOnlyRootFilesystem: true and SYS_PTRACE for /proc access. RBAC is minimal: nodes: [get].

When enabled: true (default), Model CRs with placement.mode: auto use cached density data for instant placement instead of spawning probe pods.

See the llmfit skill docs for full details on the density API, web UI, and Prometheus metrics.

Agent Sandbox (Kubernetes CRD)

Integrates with kubernetes-sigs/agent-sandbox for kernel-level isolation (gVisor/Kata), warm pools, and suspend/resume lifecycle. See the Agent Sandbox concept doc for details.

agentSandbox:
  enabled: false                   # Master switch — requires agent-sandbox CRDs installed
  defaultRuntimeClass: "gvisor"    # Default runtimeClassName for Sandbox CRs
  rbac: true                       # Grant controller RBAC for Sandbox/SandboxClaim/SandboxWarmPool

When enabled: true, the controller: - Creates RBAC rules for agents.x-k8s.io resources (Sandbox, SandboxClaim, SandboxWarmPool) - Checks for agent-sandbox CRDs at startup and enables the feature if found - Routes AgentRuns with agentSandbox.enabled: true to Sandbox CRs instead of Jobs

Network Policies

networkPolicies:
  enabled: true
  extraEgressPorts: []    # add non-standard API server ports here (e.g. [6444] for k3s)