Policies and checkpoints

A policy is what a training job produces: the trained network that picks your robot’s actions. Every training job makes exactly one policy, and the policy’s ID is the job’s ID with a different prefix:

Training job	Its policy
job_brisk-heron-4f8k2m	policy_brisk-heron-4f8k2m

A preview job makes no policy. It returns a report and a recording so you can fix robot or task problems before starting a training job.

You never create or name a policy yourself. It is a view of its training job: its status, tags, and organization are the job’s, and it exists from the moment you submit the job. simulo run prints both IDs as soon as the job is created.

Checkpoints: best and latest

A checkpoint is a saved copy of the policy at one point in training. A policy holds at most two:

• latest is the most recent checkpoint the job saved.
• best is the checkpoint with the highest mean episode reward measured at any save. Reinforcement learning rewards are noisy, so the best policy is often not the last one.

You name a checkpoint as the policy ID, a colon, and the kind: policy_brisk-heron-4f8k2m:best or policy_brisk-heron-4f8k2m:latest. Every command that takes a checkpoint uses this form.

A checkpoint file holds the trainer’s full state, including the optimizer, which is what lets a later job continue training from it. To run the policy outside Simulo, export it instead (below).

Saving is automatic

A job declared with type="train" saves checkpoints on its own. You add no callback, choose no file names, and return nothing special. When RLTrainer.train() runs inside a training job, it:

1. saves a checkpoint every 50 iterations, and at the last iteration if that isn’t already a save, replacing latest each time;
2. at each save, compares the mean reward of the episodes that finished since the previous save with the best so far, and replaces best when it is strictly higher.

If no episode finished since the previous save, that save cannot become best, so a very short run can end with only latest. To save more or less often, pass checkpoint_every to train():

stats = trainer.train(max_iterations=600, checkpoint_every=25)

The checkpoints stay on the GPU machine while the job runs. When the machine finishes the job, whether it completed, failed, was cancelled, or timed out, it uploads them to the policy before reporting the final status. After a successful run, simulo run prints what was saved:

Checkpoints:  best (iteration <n>)  latest (iteration <n>)
Next: simulo policy get policy_brisk-heron-4f8k2m:best
      simulo run eval.py --policy policy_brisk-heron-4f8k2m:best
      simulo run train.py --max-iterations <2 x latest iteration> --from policy_brisk-heron-4f8k2m:best
      simulo export policy_brisk-heron-4f8k2m:best

A policy is sealed when its job ends

Once the training job reaches completed, failed, cancelled, or timed_out, its policy never changes again. Simulo refuses any checkpoint upload for a finished job, so what you download later is exactly what the job produced.

A job that stops early keeps whatever it saved. A training job that is cancelled or times out after 300 iterations still has the checkpoints from its saves up to then, and you can continue from them.

A training job that finishes normally without saving any checkpoint is marked failed with the reason no_checkpoint_saved, because it produced no policy. See Job lifecycle.

Find and download policies

simulo policy list                 # policies you trained
simulo policy list --org           # every policy in your organization
simulo policy list --tag baseline  # repeat --tag to require several

The table shows each policy’s status, which checkpoints it has, the best checkpoint’s reward, and its tags. --json prints the full records, including each checkpoint’s iteration, reward, size, and SHA-256 digest.

simulo policy get policy_brisk-heron-4f8k2m:best
simulo policy get policy_brisk-heron-4f8k2m:best -o ./best.pt

The file is checked against its recorded digest before it is written. Without -o, it is saved in the current directory as policy_brisk-heron-4f8k2m-best.pt. The Console’s Policies page lists the same policies, with the same Owner: Mine / Organization choice and a tag filter. Open one to see its checkpoints with a download button for each, the jobs that used it (filterable by type), a Play card that shows and copies the simulo run play.py --policy command for the checkpoint you pick, an Evaluate card with the corresponding eval.py command, an Evaluations list, and an Export panel.

Anyone in your organization can use any of its policies by ID, with simulo policy get, --from, or simulo export. A policy in another organization looks exactly like one that does not exist.

If you write a job ID with a kind, such as job_brisk-heron-4f8k2m:best, where a checkpoint belongs, the command tells you the policy ID to use instead:

job_brisk-heron-4f8k2m is a job; did you mean policy_brisk-heron-4f8k2m:best?

Inspect one policy

simulo policy inspect policy_brisk-heron-4f8k2m

policy_brisk-heron-4f8k2m   CartpoleTask, cartpole robot (simulo/robot/cartpole:v1)
Trained by       job_brisk-heron-4f8k2m (completed)
Started from     scratch
Tags             baseline

Checkpoints
  best     iteration <n>   reward <reward>
  latest   iteration <n>   reward <reward>

Evaluations
  job_calm-otter-9p3x7d   policy_brisk-heron-4f8k2m:best   87 of 100, 79% to 93%

simulo policy inspect POLICY_ID shows one policy’s training context: the task it was trained on and its robot, the job that trained it (and its status), the checkpoint it started from (or scratch), its tags, and every checkpoint with its iteration and reward. Its Evaluations section lists the newest evaluation jobs, the checkpoint each judged, and each completed job’s success count and likely range. Use simulo result JOB_ID for the full report. --json prints the full policy record instead of this view. A bare job ID gets its own hint, without a checkpoint kind:

job_brisk-heron-4f8k2m is a job; did you mean policy_brisk-heron-4f8k2m?

Continue training with --from

simulo run train.py --from policy_brisk-heron-4f8k2m:best --max-iterations 900

The new job starts from that checkpoint instead of from scratch, makes its own policy, and counts the checkpoint’s iterations toward --max-iterations. The task must still fit the checkpoint: the same observation size, action size, and training algorithm. See Continue training for the full guide, including what --from requires and when it’s refused.

Export a checkpoint

simulo export policy_brisk-heron-4f8k2m:best

Export runs as its own export job. It converts the checkpoint to ONNX, checks that the converted model gives the same actions as the original, and downloads an ONNX bundle into the current directory: the model plus its input and output descriptions, test vectors, and a verify.py you can run without Simulo installed. See Export a policy to ONNX.

Play or evaluate a checkpoint

simulo run play.py --policy policy_brisk-heron-4f8k2m:best

A play job runs a checkpoint back through its task instead of training it, so you can watch it or record it. An evaluation runs a checkpoint on fixed episodes and reports how often its success rule passed. Both need a checkpoint, policy_x:best or policy_x:latest. --from continues training on a training file, while --policy plays or evaluates a checkpoint on a play or eval file. Each is refused on the other kind of file, and the refusal names the flag you meant. Evaluation explains the success rule, report, recordings, and checkpoint comparisons. See Evaluate a trained policy for the complete workflow.

Play also checks the checkpoint against the task it’s playing:

• The observation and action sizes must still match. A checkpoint was trained against fixed sizes. If the task’s sizes no longer match, the job is refused before its first episode:

  policy_brisk-heron-4f8k2m:best was trained with 4 observations and 1 action; CartpoleTask now has 5 observations and 1 action. Play a policy trained on this task, or retrain with simulo run train.py.

• Any other change to the task is allowed, but noted. If task.py changed in some other way since training, one line at the start of the play job says so, and the job still runs:

  task.py has changed since policy_brisk-heron-4f8k2m was trained

See Play a policy and record it for the full workflow, including the recording it makes.