hylo-proposer-llm¶
Hylo self-improvement with an LLM: steering-candidate proposer and lesson reflector (evolve time, never in the control loop)
Repository hylo-harness, adapters/hylo-proposer-llm. Licence: Apache-2.0. Depends on: hylo-core, hylo-agents, hylo-evolve, hylo-reasoner-llm.
Registered entry points
| Group | Name | Object |
|---|---|---|
| generators | llm.steering |
hylo.proposers.llm.proposer:LLMSteeringProposer |
| reflectors | llm.reflector |
hylo.proposers.llm.reflector:LLMReflector |
| Module | Summary |
|---|---|
hylo.proposers.llm |
|
hylo.proposers.llm.proposer |
LLM proposer for steering candidates (RFC-0002 step 3) |
hylo.proposers.llm.reflector |
LLM reflector (RFC-0003 M4): distil lessons from the search split's successes and failures |
hylo.proposers.llm¶
META
module-attribute
¶
META = {'contracts': {'generators': 'hylo.generator/v0.1', 'reflectors': 'hylo.reflector/v0.1'}, 'maturity': 'implemented', 'plugins': {'generators/llm.steering': {'weights': "the served model's own licence (e.g. a Qwen or GLM model run by Ollama)"}, 'reflectors/llm.reflector': {'maturity': 'unverified', 'weights': "the served model's own licence (e.g. a Qwen or GLM model run by Ollama)"}}}
hylo.proposers.llm.proposer¶
LLM proposer for steering candidates (RFC-0002 step 3).
The model reads what the search split shows (failure buckets, per-episode timelines of grasps and stalls, a few successes for contrast), the current steering config with its schema bounds, and the archive's earlier hypotheses with their verdicts. It never sees search_val, gate or held-out numbers. Every proposal must validate against SteeringConfig (field bounds, rule schema); invalid proposals are dropped and counted.
SYSTEM
module-attribute
¶
SYSTEM = 'You improve a frozen robot manipulation policy (a VLA) WITHOUT changing its weights, by editing the\nconfiguration of a steering wrapper around it. You see diagnostics from failed and successful episodes and propose\na few alternative steering configurations, each with ONE falsifiable hypothesis about which failures it fixes and why.\n\nThe steering wrapper:\n- samples action chunks from the policy with seeded noise; n_samples > 1 draws several chunks and selector\n "gripper_vote_medoid" executes the medoid of the gripper-majority samples (cost grows linearly with n_samples);\n- executes `execution_steps` actions of each 50-step chunk before asking the policy again (null = all 50);\n- noise_scale scales the policy\'s sampling noise (1.0 = as trained);\n- rules: runtime triggers. feature in [still_steps, grasp_attempts, disagreement, steps], op in [>=, >, <=, <],\n action "resample" (params: noise_scale) or "backoff" (params: steps, dz, open_gripper); cooldown and max_fires\n are in control steps / per episode. disagreement requires n_samples >= 2.\n\nExperience (when present): `lessons` are notes distilled from earlier episodes, each with an id and helpful/harmful\nvotes; `skills` are earlier steering configurations with a status: verified (shown to help), refuted (shown not to\nhelp), proposed (tried, not yet shown either way). Build on verified skills and well-supported lessons, do not\nre-propose refuted ones, and list the ids of the lessons each candidate relies on in "lessons_used".\n\nRules of the game: propose changes that plausibly generalise across tasks and init states; do not repeat an\nearlier hypothesis that was refuted; keep each candidate a coherent single idea.\nReply with JSON only: {"candidates": [{"hypothesis": "...", "steering": {<complete steering config>},\n"lessons_used": ["<lesson id>", ...]}]}'
LLMSteeringProposer ¶
LLMSteeringProposer(model: str = 'qwen3.5:35b-a3b', base_url: str = 'http://127.0.0.1:11434', api: str = 'ollama', n_candidates: int = 4, n_failure_examples: int = 6, n_success_examples: int = 2, cache_dir: str | None = None, path: str = STEERING_PATH, max_tokens: int = 1500, **client_kwargs: Any)
client
instance-attribute
¶
client = LLMClient(model=model, base_url=base_url, api=api, cache_dir=cache_dir, max_tokens=max_tokens, **client_kwargs)
propose ¶
propose(diagnosis: Mapping[str, Any], cfg: ExperimentConfig, parent_id: str | None, archive: list[ArchiveEntry]) -> list[tuple[Candidate, str]]
hylo.proposers.llm.reflector¶
LLM reflector (RFC-0003 M4): distil lessons from the search split's successes and failures.
Like ReasoningBank it learns from failures as well as successes (pitfalls next to strategies); like ExpeL / ACE it never rewrites the lesson book, it emits operations (ADD / UPVOTE / DOWNVOTE / EDIT / RETIRE) that LessonBook applies deterministically. It reads exactly what the steering proposer reads (search-split diagnostics and timelines), never search_val, gate or held-out data, so lessons cannot carry selection-split information into proposals. Every ADD must cite episodes it was shown; uncited or out-of-scope operations are dropped.
SYSTEM
module-attribute
¶
SYSTEM = 'You maintain a lesson book for a robot manipulation agent (a frozen VLA policy with a steering wrapper).\nFrom episode timelines (grasp steps, still periods, end-effector heights, outcome) you extract short, reusable\nlessons: "strategy" lessons describe what successful episodes did, "pitfall" lessons describe failure patterns to\navoid. A lesson is useful only if it is specific (names the phase, object or signal) and could change a decision.\n\nYou may only:\n- ADD a new lesson: {"op": "ADD", "scope": "global" | "task:<task id from the input>", "kind": "strategy"|"pitfall",\n "title": "<= 12 words", "content": "<= 60 words", "evidence": ["<episode id from the input>", ...]}\n- UPVOTE or DOWNVOTE an existing lesson the new episodes confirm or contradict: {"op": "UPVOTE"|"DOWNVOTE",\n "id": "<lesson id>", "evidence": ["<episode id>"]}\n- EDIT an existing lesson\'s content: {"op": "EDIT", "id": "<lesson id>", "content": "..."}\nDo not restate existing lessons as new ones; vote on them instead. Use scope "global" only for patterns seen in\nseveral tasks. Reply with JSON only: {"ops": [...]}'
LLMReflector ¶
LLMReflector(model: str = 'qwen3.5:35b-a3b', base_url: str = 'http://127.0.0.1:11434', api: str = 'ollama', max_new: int = 4, per_task: int = 2, max_tasks: int = 6, cache_dir: str | None = None, max_tokens: int = 1500, timeout_s: float = 600.0, **client_kwargs: Any)