Configuration

What an experiment measures, where its components run on one machine, and how plugins are found. The experiment config schema documents the YAML format; execution profiles and the plugin catalogue the other two from the user's side.

Experiment configs

config

Versioned experiment configuration and run manifest (design/experiment-schema.md).

SCHEMA_VERSION module-attribute

SCHEMA_VERSION = 'hylo.experiment/v0.1'

REMOTE_AGENTS module-attribute

REMOTE_AGENTS = ('remote',)

PLACEMENT module-attribute

PLACEMENT = ('runtime', 'python', 'env')

HYLO_PACKAGES module-attribute

HYLO_PACKAGES = ('hylo-core', 'hylo-eval', 'hylo-agents', 'hylo-memory', 'hylo-evolve', 'hylo-env-mujoco', 'hylo-cli')

ComponentRef

Bases: Contract

type instance-attribute

type: str

params class-attribute instance-attribute

params: dict[str, Any] = Field(default_factory=dict)

RemoteSpec

Bases: Contract

Where a component runs: never part of what is measured. runtime names a venv in this machine's profile (core.profile); python / env set or override it directly.

runtime class-attribute instance-attribute

runtime: str | None = None

python class-attribute instance-attribute

python: str | None = None

env class-attribute instance-attribute

env: dict[str, str] = Field(default_factory=dict)

EnvRef

Bases: ComponentRef

remote class-attribute instance-attribute

remote: RemoteSpec | None = None

BenchmarkRef

Bases: ComponentRef

split class-attribute instance-attribute

split: str = 'validation'

TraceOptions

Bases: Contract

record_privileged class-attribute instance-attribute

record_privileged: bool = True

image_every class-attribute instance-attribute

image_every: int = Field(default=1, ge=0)

ExperimentConfig

Bases: Contract

schema_version class-attribute instance-attribute

schema_version: Literal['hylo.experiment/v0.1'] = SCHEMA_VERSION

name instance-attribute

name: str

benchmark instance-attribute

benchmark: BenchmarkRef

env instance-attribute

env: EnvRef

agent instance-attribute

agent: ComponentRef

seed class-attribute instance-attribute

seed: int = 0

safety class-attribute instance-attribute

safety: dict[str, Any] = Field(default_factory=dict)

trace class-attribute instance-attribute

trace: TraceOptions = Field(default_factory=TraceOptions)

timing class-attribute instance-attribute

timing: Timing = Field(default_factory=Timing)

output_dir class-attribute instance-attribute

output_dir: str = 'runs'

workers class-attribute instance-attribute

workers: int = Field(default=1, ge=1)

tags class-attribute instance-attribute

tags: tuple[str, ...] = ()

load classmethod

load(path: str | Path) -> 'ExperimentConfig'

measured

measured() -> dict[str, Any]

What the experiment measures. Left out: where results go, how many workers, and where components run (env.remote, a remote agent's runtime / python / env), so the same experiment has the same digest on every machine. Timing is left out while it is the default (sync), so older digests stay valid.

canonical_json

canonical_json() -> str

digest

digest() -> str

with_overrides

with_overrides(patch: dict[str, Any]) -> 'ExperimentConfig'

Apply dotted-path overrides, e.g. {"agent.params.kp": 2.0}; the result is re-validated.

build_manifest

build_manifest(cfg: ExperimentConfig, *, run_id: str, extra: dict[str, Any] | None = None) -> dict[str, Any]

Execution profiles

profile

Execution profiles (hylo.profile/v0.1): where components run on this machine, kept out of experiment configs.

An experiment says what is measured. Where an interpreter lives, which rendering library to load, where the model cache is, differ per machine and must not change an experiment's identity. Configs name a runtime instead:

env:   {type: libero, remote: {runtime: libero}}
agent: {type: remote, params: {ref: lerobot.steered, runtime: libero, params: {...}}}

and each machine's profile says what that runtime is:

schema_version: hylo.profile/v0.1
runtimes:
  libero:
    python: /home/me/hylo/bench/envs/libero/.venv/bin/python
    env: {MUJOCO_GL: osmesa, LIBERO_CONFIG_PATH: /home/me/hylo/bench/envs/libero/config, HF_HOME: ...}

The profile is found at $HYLO_PROFILE, else ~/.config/hylo/profile.yaml. hylo profile shows and edits it; bench's envs/libero/setup.sh registers the runtime it builds. Explicit python / env in a config still work and take precedence over the runtime's (for a one-off); neither enters the experiment digest. The profile also says where this machine's run and artifact store is (store:, set by hylo store init PATH).

PROFILE_SCHEMA module-attribute

PROFILE_SCHEMA = 'hylo.profile/v0.1'

Runtime

Bases: Contract

python class-attribute instance-attribute

python: str | None = None

env class-attribute instance-attribute

env: dict[str, str] = Field(default_factory=dict)

Profile

Bases: Contract

schema_version class-attribute instance-attribute

schema_version: Literal['hylo.profile/v0.1'] = PROFILE_SCHEMA

runtimes class-attribute instance-attribute

runtimes: dict[str, Runtime] = Field(default_factory=dict)

store class-attribute instance-attribute

store: str | None = None

profile_path

profile_path() -> Path

load_profile

load_profile(path: str | Path | None = None) -> Profile

save_profile

save_profile(profile: Profile, path: str | Path | None = None) -> Path

resolve

resolve(runtime: str | None = None, python: str | None = None, env: dict[str, str] | None = None) -> tuple[str | None, dict[str, str]]

The interpreter and environment variables to start a component with: the named runtime from this machine's profile, overridden by explicit values.

Contract versions and plugin metadata

plugins

Contract versions and plugin metadata (RFC-0005 part 4).

Every plugin group is a contract with a version (hylo.policy/v0.1, hylo.agent/v0.1, ...). A distribution declares which version of each contract its plugins implement, and what is known about them, in one dictionary exposed under the hylo.meta entry-point group (a module with no heavy imports, so listing plugins never loads them):

[project.entry-points."hylo.meta"]
hylo-policy-lerobot = "hylo.policies.lerobot:META"

META = {
    "contracts": {"agents": "hylo.agent/v0.1", "policies": "hylo.policy/v0.1"},
    "maturity": "implemented",                                  # the default for this distribution's plugins
    "plugins": {"agents/lerobot.steered": {"weights": "per checkpoint: ..."},
                "datasources/lerobot.dataset": {"maturity": "unverified", "notes": "..."}},
}

The code licence is the distribution's own (license in its pyproject); weights names the licence of model weights a plugin downloads or needs, which is often different. Maturity:

implemented     run end to end on its intended target (a real simulator or robot), with recorded runs
unverified      tests pass, but never run end to end on its intended target here
needs-hardware  needs hardware this lab has not connected yet
test-double     for tests and conformance only (`testing.*` names)

The registry refuses to load a plugin whose declared contract is incompatible with this core: another major version, or a newer minor version (a plugin written against features this core does not have). A plugin without a declaration still loads; hylo plugins shows it as undeclared and hylo plugin check warns.

CONTRACTS module-attribute

CONTRACTS: dict[str, str] = {'envs': 'hylo.env/v0.1', 'agents': 'hylo.agent/v0.1', 'policies': 'hylo.policy/v0.1', 'reasoners': 'hylo.reasoner/v0.1', 'choosers': 'hylo.chooser/v0.1', 'planners': 'hylo.planner/v0.1', 'reflectors': 'hylo.reflector/v0.1', 'memory': 'hylo.memory/v0.1', 'scorers': 'hylo.scorer/v0.1', 'benchmarks': 'hylo.benchmark/v0.1', 'generators': 'hylo.generator/v0.1', 'learners': 'hylo.learner/v0.1', 'instruments': 'hylo.instrument/v0.1', 'datasources': 'hylo.datasource/v0.1', 'guards': 'hylo.guard/v0.1', 'gate_tests': 'hylo.gate_test/v0.1'}

CONTRACT_OBJECTS module-attribute

CONTRACT_OBJECTS: dict[str, str] = {'envs': 'hylo.core.env:Environment', 'agents': 'hylo.core.agent:Agent', 'policies': 'hylo.core.policy:ChunkPolicy', 'reasoners': 'hylo.core.reasoning:Reasoner', 'choosers': 'hylo.core.reasoning:Chooser', 'planners': 'hylo.core.reasoning:Planner', 'reflectors': 'hylo.core.reasoning:Reflector', 'memory': 'hylo.core.memory:MemoryStore', 'scorers': 'hylo.core.scoring:Scorer', 'benchmarks': 'hylo.core.benchmark:Benchmark', 'generators': 'hylo.core.improvement:Proposer', 'learners': 'hylo.core.learning:Learner', 'instruments': 'hylo.core.instrument:SuccessJudge', 'datasources': 'hylo.core.learning:DataSource', 'guards': 'hylo.core.safety:Guard', 'gate_tests': 'hylo.core.scoring:GateTest'}

MATURITY module-attribute

MATURITY = ('implemented', 'unverified', 'needs-hardware', 'test-double')

Maturity module-attribute

Maturity = Literal['implemented', 'unverified', 'needs-hardware', 'test-double', 'undeclared']

IncompatiblePlugin

Bases: RuntimeError

A plugin declares a contract version this core cannot run.

PluginMeta

Bases: Contract

group instance-attribute

group: str

name instance-attribute

name: str

target instance-attribute

target: str

distribution class-attribute instance-attribute

distribution: str | None = None

version class-attribute instance-attribute

version: str | None = None

contract class-attribute instance-attribute

contract: str | None = None

expected class-attribute instance-attribute

expected: str | None = None

compatible class-attribute instance-attribute

compatible: bool | None = None

maturity class-attribute instance-attribute

maturity: Maturity = 'undeclared'

licence class-attribute instance-attribute

licence: str | None = None

weights class-attribute instance-attribute

weights: str | None = None

notes class-attribute instance-attribute

notes: str | None = None

parse

parse(version: str) -> tuple[str, int, int]

compatible

compatible(declared: str, current: str) -> bool

Same contract and major version, and not newer than this core's minor version.

declarations cached

declarations() -> dict[str, dict[str, Any]]

{distribution: META} from the hylo.meta entry points.

describe

describe(group: str, ep: EntryPoint) -> PluginMeta

check

check(group: str, ep: EntryPoint) -> PluginMeta

Raise IncompatiblePlugin when the plugin declares a contract version this core cannot run.

catalogue

catalogue(groups: tuple[str, ...] | None = None) -> list[PluginMeta]

Every installed plugin with its metadata, without importing any plugin.

Plugin registry

registry

Plugin registry: explicit registration, module:attr references (vla-eval style) and hylo.<group> entry points (inspect-robots style, with autoload switchable off).

GROUPS module-attribute

GROUPS = ('envs', 'agents', 'policies', 'reasoners', 'choosers', 'planners', 'reflectors', 'memory', 'scorers', 'benchmarks', 'generators', 'learners', 'instruments', 'datasources', 'guards', 'gate_tests')

registry module-attribute

registry = Registry()

Registry

Registry(*, autoload: bool | None = None)

register

register(group: str, name: str) -> Callable[[Any], Any]

register_lazy

register_lazy(group: str, name: str, ref: str) -> None

Register a module:attr reference without importing it (keeps heavy deps optional).

resolve

resolve(group: str, name_or_ref: str) -> Any

reference

reference(group: str, name_or_ref: str) -> str

module:attr for a plugin without importing it (for components that run in another venv).

build

build(group: str, spec: dict[str, Any]) -> Any

spec = {"type": "", "params": {...}}; store ids in params (artifact:@, run:) become this machine's paths here, after the experiment digest was taken over the ids.