Skip to content

Populi remote execution rollout checklist

Use this checklist before widening Populi remote execution beyond local-first defaults—whether using today’s experimental relay or a future lease-authoritative path (ADR 017).

  • Documented scope: confirm the deployment matches a column in the work-type placement matrix (local / LAN / overlay).
  • No accidental public bind: Populi listeners and MCP HTTP gateways use loopback or controlled ingress unless TLS and auth are in place (deployment compose SSOT, MCP HTTP gateway contract).
  • Secrets: mesh tokens and JWT secrets live in Secrets / secret stores; vox secrets doctor passes for required workflows (Secrets SSOT).

Prove you can disable remote paths without redeploying code:

SwitchEffect (current docs)
VOX_ORCHESTRATOR_MESH_REMOTE_EXECUTE_EXPERIMENTAL=0 (unset/false)Disables experimental RemoteTaskEnvelope relay; local execution unchanged (orchestration unified).
VOX_ORCHESTRATOR_MESH_ROUTING_EXPERIMENTAL=0Disables hint-based routing score experiments (mens SSOT).
VOX_ORCHESTRATOR_MESH_CONTROL_URL unsetStops federation node snapshot reads from Populi (orchestrator/MCP) (env vars).
VOX_MESH_HTTP_JOIN=0MCP skips HTTP join/heartbeat while other mesh hooks may still run (mens SSOT).
VOX_MESH_ENABLED=0Disables mens hooks in processes that respect this flag (mens SSOT).

Staging drill: toggle each relevant switch, restart or reload the affected process per your platform, and confirm no remote fan-out and no unexpected control-plane traffic (packet capture or access logs).

  • Single owner: for lease-backed task classes (when implemented), reproduce lease acquisition, renewal, and expiry; confirm no concurrent execution on two nodes for the same correlation id.
  • Fallback: on lease loss, verify local fallback or documented fail-closed behavior per operator policy (ADR 017).
  • Cancellation: remote cancel paths propagate within agreed timeouts.
  • Results: result or failure delivery is idempotent on redeliver (mesh idempotency_key where used).
  • Logs or traces include task_id (or equivalent) for routed work; when lease placement ships, include lease_id and placement reason per placement observability.
  • Optional: VOX_MESH_CODEX_TELEMETRY emits populi_control_event rows without storing bearer material (mens SSOT).
  • CI / smoke: vox ci check-links and mdBook build succeed after doc changes; workspace tests for Populi/orchestrator crates pass for the PR that enables new behavior.
  • Rollback plan: document which env toggles return the fleet to local-only execution and who is allowed to flip them.
OutcomeCondition
GoKill-switch drill passed; matrix row matches workload; observability fields confirmed in pilot logs.
No-goAny unexplained duplicate execution, missing fallback on forced partition, or inability to disable relay via env within minutes.