Evaluate & roll out

The inference template (simulo create --type inference) scaffolds a full policy lifecycle as three @app.jobs in one app: train produces a checkpoint, evaluate scores it, rollout plays it back and records the result. This guide runs all three in order.

simulo create policy_demo --type inference
cd policy_demo

1. Train

simulo run app.py --job train --num-envs 512 --max-iterations 20

Trains a PPO policy and saves two outputs to the checkpoint volume: the trainer’s own checkpoint format (which evaluate reads back into a trainer) and a separately-exported TorchScript policy (which rollout plays directly, with no trainer involved at all).

2. Evaluate

simulo run app.py --job evaluate --num-episodes 10 --num-rounds 5

Replays the checkpoint for --num-rounds independent rounds of --num-episodes episodes each, then aggregates mean reward, reward std-dev, mean episode length, and a success rate against a configurable reward target — written to a JSON report on the reports volume and returned as the job’s result. The field names and shape below are what the job really returns; the numbers are an illustrative example, not a measured benchmark — your rewards depend on the task, the training length, and the run itself:

{
  "num_rounds": 5,
  "num_episodes": 10,
  "reward_target": 75.0,
  "mean_reward": 261.4,
  "std_reward": 12.8,
  "mean_length": 297.5,
  "success_rate": 1.0
}

3. Roll out

simulo run app.py --job rollout --num-steps 200

Plays the exported TorchScript policy with simulo.RLPlayer — inference only, no trainer, no torch.no_grad() bookkeeping to think about — and records every step to an MCAP flight recording via simulo.RecordConfig.

simulo recordings

Downloads and verifies the MCAP, and prints where it landed. See Record and inspect MCAP to open it.

Order matters

evaluate and rollout both read outputs train writes to the shared checkpoint volume. Running either one before train has populated that volume fails with a clear “checkpoint not found” error — not a silent no-op, and not a partial result.

Fresh checkpoint volume

simulo run app.py --job rollout --num-steps 200

FileNotFoundError: Checkpoint not found: .../policy_demo_policy.pt

Run --job train first.

After train

rollout finds the exported policy and plays it immediately — no re-training needed to iterate on evaluation or recording settings.

Observe each stage

simulo jobs                 # all three jobs, most recent first
simulo logs --follow        # the stage currently running
simulo result                # that stage's own return value

Next

Keep training the same policy further: Continue training. Watch the training stage live instead of via logs: Watch a run live.