Skip to main content
Worlds expose named observation channels, each declaring a modality, encoding, schema, resolution, and nominal rate in the manifest. An agent subscribes to the subset it can consume. Every frame has the same shape regardless of what it carries: a per-channel sequence number, a capture timestamp, in streaming a send timestamp, and a payload. The payload is exactly the bytes of the channel’s declared modality — a JPEG for an image channel, UTF-8 JSON for a proprioception or event channel — and nothing else.
Delivery depends on the time model: in lockstep, the world sends one frame per channel per tick, usually inline on the control connection; in streaming, frames are pushed continuously on their channels with capture and send timestamps and sequence numbers, and the agent measures the staleness of each one against a clock it has synchronized to the world’s over the heartbeat. Every frame carries enough metadata (seq, ts_mono_ns, ts_send_ns, channel binding) for the agent to reason about ordering, loss, and latency — see timestamps and clocks.