Catalogue-backed local host issuance¶
Historical record
This page records a Celln development milestone (September 2026). Interfaces and limits it describes may since have changed or been replaced by the Celln fleet. For current behaviour see Celln Backend and Celln Fleet Installation.
sympozium celln-tool issue AGENT is an operator-only bridge from live
Kubernetes approvals to the local Celln host issuer. It does not submit an
execution, distribute artifacts to a fleet, advertise readiness or implement
the end-user UI/controller journey.
It requires the existing three independent tool-grant source flags, --run,
--model-policy, actual --execution-mote and --execution-closure hashes,
plus operator-configured --celln-binary, --policy-root and
--composer-publisher. Runtime/tool members and schemas must already exist in
the host stores, and the mote must already be independently admitted.
Issuance now defaults to --profile-lifetime 5m, requiring Celln's
harness-profile-clock and model-issuer-profile-v2 support (Celln #93).
Lifetimes must be whole milliseconds in 1ms..5m. Missing support refuses;
there is no automatic fallback. --profile-lifetime 0 explicitly retains the
legacy non-expiring operator path and must not be used for unattended issuance.
Library callers select IssueOptions.ProfileLifetime; its zero value remains
legacy-compatible, so future automated callers must explicitly require it.
The operator maintains POLICY_ROOT/model-credentials.json:
{
"apiVersion": "sympozium.ai/celln-host-credentials-v1",
"profiles": {"approved-deepseek": "/operator-owned/deepseek-token"}
}
The selected name comes from the independently reviewed model policy; the credential path comes only from this host mapping. The bridge never reads the credential contents, puts them in Kubernetes, or sends them to the guest. Tenant access to artifact stores must not permit writes to host credentials, issuer profiles, grant files or the operator-configured approval sources.
Issuance and withdrawal¶
The bridge builds the exact request from its revalidated frozen selection and live run. It compares the composed descriptor's ordered signed source identities with the selected sources and asks Celln to authenticate a private captured copy of those descriptor bytes. It then obtains the normalized request binding from the real Celln parser and atomically creates a private, content-derived host profile without overwriting different existing bytes.
Celln performs sealed guest member verification and issues a v3 grant. The bridge rechecks the returned request binding, current Kubernetes approval, host mapping revision and composed descriptor before returning. Local issuance/withdrawal operations share a nonblocking OS file lock; unsupported platforms refuse.
Failure after profile publication removes exactly that profile, including a matching profile retained by an earlier identical retry. Changed/unrelated bytes are not deleted, and a cleanup failure is returned alongside the original error. Grant/audit files remain intact; a v3 grant cannot pass its host profile check after that profile is removed. This is local refusal at a new resolution, not a fleet-wide active-cell withdrawal guarantee.
Persist the complete command output as an issuance report. To withdraw it:
sympozium celln-tool withdraw-grant issuance-report.json \
--policy-root /absolute/operator/host-root
This removes only the profile with the matching saved name and SHA256. Already absent profiles are an idempotent success. It does not remove credentials, grants, execution journals or audit records. Withdrawal bypasses Kubernetes client initialization, so local recovery remains available when the API server is unreachable.
Recovery limits and remaining controller work¶
This is not an autonomous approval watcher. Approval withdrawal after a successful invocation requires explicit withdrawal, or the future controller reconciler. A failed initial approval check cannot discover and withdraw all profiles from older invocations. Keep saved issuance identities for recovery.
Before profile publication, issuance durably writes a pending record under
POLICY_ROOT/sympozium-issuer-journal. After all final checks, it atomically
replaces that record with issued, including the frozen selection, candidate
and full issued result, before returning success. Withdrawal first records
withdrawing, then removes the exact profile and records withdrawn. Records
are private, bounded to 1 MiB, fsynced and retained for reconciliation.
If the process dies, the OS lock releases. Recovery withdraws exact profiles
for pending or withdrawing records and durably marks them withdrawn; it
never reissues a grant or executes a task. Every new local bridge issuance runs
this recovery step under the lock first. It can also be invoked without a
Kubernetes connection:
Committed issued outcomes survive loss of stdout/the caller and can be read
through ReadIssuance or the retained journal. They are history, not current
permission: revalidate approvals and reconcile existing execution identity
before dispatch/retry. Recovery does not delete grant/audit records, replay
side effects, or withdraw every committed grant automatically.
Recovery scans at most 1024 directory entries and refuses corrupt records, identity mismatches or a larger journal; operators must reconcile/archive history before exhausting that bound. Automatic retention is not implemented. Changed profile bytes are not removed, and any recovery error blocks new bridge issuance. Legacy profiles without journal records still require an explicit saved-report withdrawal or operator audit.
While the bridge is down, an unfinished legacy profile can remain effective until recovery runs; bounded profiles instead refuse at their original expiry. The host dispatcher does not consult this journal. Deployment startup/dispatch must be gated on recovery. Do not expose this local operator primitive as an unattended production service until controller reconciliation, ongoing approval withdrawal/expiry and startup recovery gates are integrated.
Durable bounded admission windows¶
Bounded issuance reads Linux boot identity and elapsed boot milliseconds from
the independently configured local Celln binary. It never trusts tenant or wall
clock timestamps. Before publishing any model profile it persists an immutable,
private window under POLICY_ROOT/sympozium-issuer-windows. The key binds the
complete execution candidate and independently mapped base profile; the record
pins boot identity, issuance time and expiry. Creation is fsynced and does not
overwrite different bytes. Window access is serialized by the existing issuer
lock.
An identical retry reuses the original window and hence the same profile and grant identity. It cannot renew the window by restarting the bridge, changing the configured lifetime, or waiting for expiry. A different boot, expiry, malformed window or withdrawn bounded issuance refuses. Recovery of an interrupted pending issuance withdraws it and retains its window; retries cannot restore it. Independently changed approval creates a different candidate, but never bypasses the host execution ownership/replay rules. A final host-clock check after issuance catches an observed expiry during provisioning and removes the exact profile before returning an error.
Retain windows with issuance history: deleting a window could allow an operator to create a new start time. Automatic retention/compaction is not implemented; tenant access must never permit modifying this directory. A window is not itself host authority or a runnable grant. Host-side checks enforce expiry even while the bridge is down. Existing non-expiring profiles are not retroactively migrated.
This closes local admission-window renewal on retry, not the complete unattended controller path. Continuous reconciliation, startup gates, selected-node prewarm, single dispatch and result correlation still need integration. Expiry gates new issuance/resolution, not active-cell cancellation or fleet revocation.
Managed host-local issuer lifecycle¶
NewManagedIssuer adds a lifecycle around bounded issuance. Deployment code
supplies a private host root, trusted Celln binary/composer, positive profile
lifetime, a 1–30 second sweep interval, and an explicit map from Agent identity
to independently configured authority loaders with uncached API readers. The
map is copied; journal contents never supply their own authority-source locations.
Start(ctx) recovers pending records and sweeps existing issuance before opening
the local provisioning gate. Sweeps repeat, with a 30-second context budget.
They withdraw tracked legacy, expired, wrong-boot or no-longer-configured profiles
and revalidate current tool/model approvals and host credential mapping for the
rest. Unknown profiles, changed bytes, corrupt history, clock failures or failed
filesystem operations close the gate and require a successful later sweep.
Unrelated profile bytes are never deleted. Each scanned directory is bounded to
1024 entries; automatic retention remains an operator follow-up.
ManagedIssuer.Issue only uses the configured loader for the frozen Agent and
always requires bounded issuance. It serializes with sweeps, refuses before
startup/after shutdown or stale sweep observation, and links in-flight issuance
to lifecycle cancellation. Duplicate lifecycle owners on the same instance are
refused. The existing OS issuer lock also excludes concurrent local CLI writes
during each managed operation. The host root must be reserved for this managed
scope; unrelated operator profiles are not silently adopted.
Status reports only the local provisioning gate, not Kubernetes availability,
selection readiness, execution permission or conformance. Every issuance still
revalidates live authority. No positive status renews profile expiry. During a
process crash, host expiry remains the admission bound; clean shutdown closes the
gate and cancels in-flight issuance but does not promise immediate fleet/live-cell
revocation of already issued authority.
The TLS host issuer service now exposes this lifecycle
through celln-tool serve-issuer, but it is not yet wired into the AgentRun
controller or Helm. The next integration must configure it on the correct host,
route catalogue provisioning through its gate,
prewarm that serving node and retain dispatch/replay identity. Existing CLI
operator commands remain explicit local operations, not a bypass-safe deployment
of the final automated user journey.
One-pass current-approval reconciliation¶
cellnreview.ReconcileIssued is the controller integration primitive for checking
one committed issuance. Its caller supplies the trusted, uncached Kubernetes
reader and independent approval-source configuration; the saved journal cannot
choose its own authority sources. Under the issuer lock it first recovers pending
records, then revalidates the frozen run, selection and model approval with a
five-second context deadline (or the caller's earlier deadline). The reader must
honor context cancellation. Missing/changed approvals, API errors or timeout,
and a removed/retargeted host credential mapping trigger durable withdrawal of
the exact profile. Token contents at the same approved host path are not read.
An unchanged observation does not rewrite or expand authority. Reconciliation never reissues, dispatches, or restores a withdrawn profile when the API returns. It preserves grant/audit records and refuses to delete different profile bytes. Filesystem failures are returned; they must not be reported as successful withdrawal. This API is not yet registered as a controller or autonomous watcher.
An issued result is only a point-in-time observation, not a readiness result or
permission lease. A watcher dying between checks leaves a bounded profile usable
until its original expiry (and a legacy profile without that time bound).
Before unattended dispatch, require the bounded profile mode described above,
startup recovery and continuous reconciliation. Existing v3 grants pin
the profile's exact bytes, so changing an expiry field in that profile cannot be
treated as transparent lease renewal. This also does not cancel an active cell
or establish fleet-wide revocation.
A successful issuance does not prove that the serving process has a warm mote, that the requested Harness/tool behavior conforms functionally, or that a model can complete the task. The remaining controller path must persist issuance identity, maintain withdrawal, prewarm the selected serving node, dispatch once, and correlate terminal results and receipts. Conversation and UI/YAML acceptance remain separate gates.
Evidence¶
Portable race tests cover idempotent publication, concurrent-issuer refusal, failed issuer/report/binding checks, post-issuance approval/mapping/composition withdrawal, capacity-independent lock release, and refusal to delete unrelated profile bytes. These tests use fake Kubernetes and a mocked command runner. Abrupt child-process exits additionally prove recovery before host issuance, after host issuance, after durable commit and during withdrawal. They bypass all Go defers and verify retained audit data, committed outcomes, released locks and repeatable recovery. This is process-crash testing, not a filesystem power-loss or deployed-controller proof.
The explicit real test uses the real compositor, signed runtime plus two tools,
real Celln issuer and KVM member verification, then repeats issuance and proves
profile withdrawal refuses further issuance while retaining grant bytes. It
still uses fake Kubernetes and makes zero model calls. Enable it only with
CELLN_ISSUANCE_KVM=1, CELLN_ISSUANCE_MATERIALIZER (the public-seed
prepare_issuance_fixture executable), CELLN_HARNESS_PACKAGE,
CELLN_COMPOSITION_FIXTURE and CELLN_COMPOSITION_BINARY, then run:
Require the explicit real-KVM issuance PASS line; an opt-in skip is not proof.