simulo.evaluate

Run a trained policy on numbered episodes, judge each one, and return the report.

Builds the task’s environment with num_envs environments and runs episodes 0 to episodes - 1, each to its end. An episode’s number fixes its starting state, so every evaluation and every checkpoint gets the same starts. The eval file’s one @app.success function judges each episode on the step it ends. Saves episodes.csv (one row per episode) and recordings of up to 10 failures as job outputs, and returns the report; return it from the job.

env_spacing is the distance in metres between environment origins; pass the same value training uses so evaluated environments sit as far apart as trained ones.

policy is the checkpoint simulo run eval.py --policy policy_x:best passes in. actions is "best" (the policy’s best-guess action, what you would deploy) or "sampled" (training’s random variation on).

simulo.evaluate(
    task: TaskProtocol,
    policy: PolicyCheckpoint,
    *,
    episodes: int = 100,
    num_envs: int = 100,
    actions: 'best' | 'sampled' = 'best',
    env_spacing: float = 4.0,
) -> EvaluationReport

---

Note

Call it from an eval file’s @app.job(type="eval") job and return its report. Start the job with simulo run eval.py --policy policy_x:best. Like simulo.preview, simulo.evaluate resolves to a torch-free stand-in at submit time and to the runtime function on the GPU worker, with this signature in both.