hylo-learn¶
Hylo learning pipeline: learner jobs, instrument calibration and labelling, external data ingestion, learners for runtime components
Repository hylo-harness, packages/hylo-learn. Licence: Apache-2.0. Depends on: hylo-core, hylo-eval, hylo-agents, numpy>=1.24.
Registered entry points
| Group | Name | Object |
|---|---|---|
| learners | critic.chunk_value |
hylo.learn.critic:CriticLearner |
| learners | judge.trace |
hylo.learn.judges:TraceJudgeLearner |
| learners | world_model.state |
hylo.learn.world_model:StateWorldModelLearner |
| Module | Summary |
|---|---|
hylo.learn |
|
hylo.learn.calibrate |
Instrument calibration and labelling (RFC-0004): is a judge good enough to label data without ground truth? |
hylo.learn.conformance |
Conformance for learning-side plugins: offline learners, online learners, data sources, success judges |
hylo.learn.critic |
Train a ChunkCritic from Hylo traces |
hylo.learn.data |
External data (RFC-0004): demonstrations, videos and other robots' logs, ingested as ordinary runs |
hylo.learn.jobs |
Run learner jobs (RFC-0004): in this process, or with another venv's interpreter |
hylo.learn.judges |
Learner "judge.trace": a success judge trained on development traces (RFC-0004 A3) |
hylo.learn.online |
Online learning runner (extension point): propose -> roll out -> update, then seal |
hylo.learn.world_model |
Learner "world_model.state" (RFC-0004 W2): fit agents.world_model.StateWorldModel on development traces |
hylo.learn¶
META
module-attribute
¶
hylo.learn.calibrate¶
Instrument calibration and labelling (RFC-0004): is a judge good enough to label data without ground truth?
python -m hylo.learn.calibrate --judge '{"type": "judge.trace", "params": {"path": ...}}' --runs DIR ...
calibration_report ¶
calibration_report(scores: Sequence[float], labels: Sequence[int], *, target_precision: float = 0.95, min_n: int = 50, min_recall: float = 0.3, runs: Sequence[str] = ()) -> CalibrationReport
calibrate_judge ¶
calibrate_judge(judge: SuccessJudge, roots: Iterable[str | Path], *, splits: Sequence[str] = DEV_SPLITS, judge_artifact: str | Path | None = None, target_precision: float = 0.95, min_n: int = 50, limit: int | None = None, seed: int = 0, task_prefix: str | None = None, records: list[dict[str, Any]] | None = None, log: Any = None) -> CalibrationReport
Score judge against simulator outcomes. Runs the judge was built from are excluded (no self-calibration);
with limit, a seeded random sample of episodes is used. Per-episode results are appended to records.
With judge_artifact, the report is written into that artifact's manifest (sealed again).
main ¶
python -m hylo.learn.calibrate --judge '{"type": "judge.vlm", "params": {}}' --runs DIR ... [--limit N]
label_run ¶
label_run(run_dir: str | Path, judge: SuccessJudge, judge_artifact: str | Path, *, threshold: float | None = None) -> dict[str, Any]
Label the unlabeled episodes of an ingested external run with a calibrated success judge.
Only a judge whose calibration is ready may label, at (at least) its calibrated high-precision threshold; an
episode below it stays unlabeled (the judge's score is recorded, but a low score is not called a failure). A run
the judge was built from is refused, and so is any run with simulator outcomes (it needs no judge).
hylo.learn.conformance¶
Conformance for learning-side plugins: offline learners, online learners, data sources, success judges.
They run against small runs made on the spot with testing.setup.check_experiment() (MuJoCo planar reach with the
IK controller when installed, the test double otherwise; or any experiment you pass), so a plugin can be checked
without any dataset. A learner that needs specific inputs (images, a policy venv) is checked on its own
data with the same functions.
sample_runs ¶
sample_runs(root: str | Path, splits: tuple[str, ...] = ('search', 'search_val', 'gate'), *, experiment: ExperimentConfig | None = None) -> Path
check_learner ¶
check_learner(name: str, params: dict[str, Any] | None = None, *, inputs: list[str] | None = None, experiment: ExperimentConfig | None = None) -> ConformanceReport
check_judge ¶
check_judge(judge: Any, *, runs: str | None = None, experiment: ExperimentConfig | None = None) -> ConformanceReport
hylo.learn.critic¶
Train a ChunkCritic from Hylo traces.
python -m hylo.learn.critic --runs runs/rsi-libero10/runs runs/evolve/libero10-llm --out critics/l10-v1 [--image-camera agentview] [--task-prefix libero/libero_10/]
Protocol integrity (RFC-0002): only episodes whose split is in --splits (default: search, search_val) are read;
gate and held-out episodes never reach the critic, so a critic-steered candidate is still judged on unseen
init states. The critic is fit on search and evaluated on search_val, then refit on both. The output directory is
sealed as a value_model artifact (RFC-0004) listing the runs it was trained on; CriticLearner is the same
trainer behind the learner interface ("critic.chunk_value").
Reports the offline diagnostics that decide whether re-ranking can help at all: - val AUC / Brier of the predicted value against the episode's final success; - the same for a state-only model: if (state + chunk) is no better than state alone, the critic cannot tell chunks apart and steering with it is noise.
CriticLearner ¶
episode_samples ¶
episode_samples(run_dir: str, episode_id: str, fcfg: dict[str, Any], gamma: float, splits: tuple[str, ...], task_prefix: str | None) -> dict[str, Any] | None
(state+chunk features, target) for every policy chunk of one episode, or None if it is filtered out.
collect ¶
collect(run_dirs: list[Path], cfg: FeatureConfig, *, gamma: float, splits: tuple[str, ...], task_prefix: str | None, workers: int) -> list[dict[str, Any]]
same_state_pairs ¶
Pairs of first chunks taken from the identical reset state (same task, same init seed) that led to different outcomes: the cases where re-ranking chunks could have changed the result.
cross_validate ¶
cross_validate(eps: list[dict[str, Any]], cfg: FeatureConfig, kw: dict[str, Any], *, folds: int = 5, log=print) -> dict[str, Any]
Grouped k-fold over init states (an init state never sits in both train and test).
train ¶
train(run_roots: list[str], out: str, *, image_camera: str | None = None, task_prefix: str | None = None, splits: tuple[str, ...] = ('search', 'search_val'), gamma: float = 0.995, horizon: int = 520, members: int = 5, hidden: int = 64, epochs: int = 60, workers: int = 16, seed: int = 0, holdout_seeds: tuple[int, ...] = (), log=print) -> dict[str, Any]
hylo.learn.data¶
External data (RFC-0004): demonstrations, videos and other robots' logs, ingested as ordinary runs.
A DataSource yields ExternalEpisodes (frames with state and camera images, actions if the source has them, an
outcome if the source states one). ingest writes them into the trace store as a run whose split is
external:<name>, so every builder reads them through the same split guard, and seals the run as a dataset
artifact recording origin and licence. Nothing external enters a gate or held-out split.
Two checks guard what external data may be used for:
- labels say who produced them: "source" (the dataset says so), "judge:label_run), or none; an unlabeled episode is never treated as a success;
- contamination_report compares each external episode's initial state with the initial states of protected
(gate / held-out) episodes of the same task; an external source that reproduces protected init states must not
be used to build anything that is then gated on them.
python -m hylo.learn.data ingest SPEC.json # {"source": {"type": ..., "params": ...}, "name": ..., "out": ...}
ingest ¶
ingest(episodes: Iterable[ExternalEpisode], out_root: str | Path, *, name: str, origin: str, licence: str, image_every: int = 1, assume_success: bool = False, params: dict[str, Any] | None = None) -> dict[str, Any]
Write episodes as a run under split external:
assume_success: label every episode a success on the source's authority (curated demonstrations); otherwise an episode is a success only if the source says so, and is left unlabeled when it says nothing.
contamination_report ¶
contamination_report(external_runs: Iterable[str | Path], protected_roots: Iterable[str | Path], *, state_keys: Sequence[str] = ('eef_pos', 'eef_axis_angle', 'gripper_qpos'), tolerance: float = 0.001) -> dict[str, Any]
External episodes whose initial state matches a protected (gate / held-out) episode of the same task.
Reads only the first observation of protected episodes; nothing from them is written anywhere.
hylo.learn.jobs¶
Run learner jobs (RFC-0004): in this process, or with another venv's interpreter.
launch starts python -m hylo.learn.jobs job.json with any interpreter that has hylo-learn installed
(it is light: hylo-core + hylo-agents), which resolves the learner through the plugin registry ("learners" group).
A job whose output is already sealed with the same job digest is not re-run (resumable campaigns).
job = TrainJob(learner="critic.chunk_value", inputs=["runs/rsi/cycle-01/runs"], out="runs/critics/c1",
params={"image_camera": "agentview"})
manifest = launch(job, python="envs/libero/.venv/bin/python")
run_job ¶
Run a learner in this process (its dependencies must be importable here).
launch ¶
launch(job: TrainJob, *, python: str | None = None, env: Mapping[str, str] | None = None, timeout_s: float | None = None, log_path: str | None = None) -> ArtifactManifest
Run a job with another interpreter (e.g. the policy's venv) and return the verified manifest.
hylo.learn.judges¶
Learner "judge.trace": a success judge trained on development traces (RFC-0004 A3).
Training data, from development-split runs only:
- the last recorded frame of every finished episode, labelled with the simulator outcome;
- early frames (progress < negative_before) of successful episodes as extra negatives: same scene, task not yet
done, which teaches the scene change rather than the scene identity.
The sealed artifact is loaded at run time by hylo.agents.judges.TraceSuccessJudge.
learner: {learner: judge.trace, params: {cameras: [agentview, wrist]}} -> sealed success_judge artifact
hylo.learn.online¶
Online learning runner (extension point): propose -> roll out -> update, then seal.
The rollouts are ordinary evaluations of a development split (search by default; gate and held-out are refused),
recorded under <out>/rollouts/iter-NNN/ like any run, so replay, judges and the catalog see them and the sealed
artifact's provenance lists them. The learner decides what to try and how to update: policy search (CEM),
on-policy RL (a GRPO-style learner would return its current policy and update from the rollouts' traces),
iterative self-training. Episodes per evaluation are the split's.
TrainJob(learner="examples.cem", inputs=[], out="runs/learned/cem", params={
"experiment": <ExperimentConfig as dict>, "iterations": 4, "split": "search", ...learner params})