Skip to content

The policy contract

How a policy meets a robot or a simulator. Deliberately boring: three verbs, one artifact rule, one wire.

The three verbs

load(run_dir)         checkpoint + stats.json
reset()               episode boundary
act(obs) -> action    obs: dict[feature-key -> array] + "task" string
  • Observation keys are the format vocabulary — the same names as the training data. There is no translation layer to get wrong.
  • Action chunking stays inside the policy. ACT's queue, a diffusion policy's horizon — the caller sees one action per step and never knows.
  • stats.json is part of the checkpoint artifact. Normalization statistics computed at training time travel next to the weights, and inference must invert them. A checkpoint without its stats is broken by definition.

Two bindings, one contract

In-process — call the three verbs directly. Used inside golden tests, where bit-determinism matters.

Over IPC (the default) — policy and environment run as separate processes exchanging iceoryx2 blackboard cells. This is how templates run evals, and how the real robot runs, with the same policy code:

env writes    eval/obs/<camera>   ImageCell   frame_id=k, episode_id=e
              eval/obs/state      VecCell     frame_id=k, episode_id=e
              eval/context        ContextCell task string, per episode
policy writes eval/action         VecCell     frame_id=k  (the ack of obs k)

Lockstep handshake: the env publishes observation k and polls for an action whose frame_id == k and episode_id == e; the policy polls for a new frame, computes, publishes. In simulation both sides poll fast (measured overhead: ~1.3 ms per exchange — the simulator step dominates by 10×). On a real robot the env side simply stops waiting: freshest-value reads, same cells, same policy process.

Why a wire instead of an import: the two processes can have different dependency lockfiles (policy stacks and simulator stacks conflict routinely); every message is inspectable mid-run by an agent; and the runtime's data recorder sits on the same cells, so eval rollouts become episodes with full provenance for free.

Hard-won rules (violate these and you will rediscover why)

  1. A fresh blackboard cell is zeroed — frame_id=0, valid_len=0 is indistinguishable from a real message unless both sides reject valid_len == 0. Both sides. We were bitten once per side.
  2. Match on episode_id as well as frame_id — a stale action from the previous episode can otherwise satisfy the handshake.
  3. The contract evolves by adding cells, never by changing existing ones. Depth cameras, observation history, richer verdicts — all additive. This single rule is what keeps rev-1 policies alive through rev-2 interfaces.

What lives elsewhere

Environment-specific facts — camera-name mapping, image orientation, warmup steps, episode length limits, success predicates — belong to the template (suites.yaml), never to this contract. A policy that conforms here runs against any environment that speaks these cells.