Quickstart

This page takes you from nothing installed to a trained policy on your disk. You need Python 3.11 or newer. You don’t need a GPU.

The job and policy IDs below are examples. Simulo gives each job its own ID, so copy the IDs from your own terminal.

pip install simulo
simulo login

pip install simulo installs the SDK and the simulo command. It installs no PyTorch, CUDA, or simulator; those run in the cloud.

simulo login opens your browser. Sign in, or create an account, and the terminal confirms who you are. See Log in if you’re on a machine without a browser.

2. Create a training app

simulo create mybot

Created app in mybot/ (eval.py + play.py + preview.py + task.py + train.py + .simuloignore).
Next steps:
  cd mybot
  simulo run preview.py
  simulo run train.py
  simulo run eval.py --policy policy_<job>:best  # evaluate the trained policy
  simulo run play.py --policy policy_<job>:best  # after training

task.py describes the task: a cartpole from the Simulo global catalog, what the robot observes, the reward, and when an episode ends. preview.py checks the robot inside that task and makes no policy. train.py holds one training job, declared with @app.job(type="train"), that trains the task with PPO. eval.py holds the success rule that measures a trained checkpoint. play.py holds a type="play" job that runs a checkpoint back once you have one. Create your first app explains all six files.

3. Preview the task

cd mybot
simulo run preview.py

The preview checks the robot inside the task before you spend time training. It reports Settle, Joint sweep, Random moves, and Training loop, writes a recording, and makes no policy. Read every finding and fix any failed check before continuing. Preview before training shows a passing report, a failed starting pose, and how to inspect the recording.

Run the preview again after every robot or task change.

4. Train the policy

simulo run train.py

This command checks and packages the app on your computer, uploads it, and creates a training job in the cloud. It prints the job’s ID and the ID of the policy the job is making, then follows the job’s logs:

TRAIN  job_brisk-heron-4f8k2m
Policy:  policy_brisk-heron-4f8k2m
Following logs (--detach to submit without waiting)...

If no GPU is free yet, the job waits in the queued state until one is. While it trains, you’ll see lines such as [RLTrainer] Best checkpoint saved at iteration <n> (reward=<r>). When the job finishes, the CLI prints the checkpoints it saved and what to run next:

Checkpoints:  best (iteration <n>)  latest (iteration <n>)
Next: simulo policy get policy_brisk-heron-4f8k2m:best
      simulo run eval.py --policy policy_brisk-heron-4f8k2m:best
      simulo run train.py --max-iterations <2 x latest iteration> --from policy_brisk-heron-4f8k2m:best
      simulo export policy_brisk-heron-4f8k2m:best

Pressing Ctrl-C stops following the logs. The job keeps running in the cloud, and simulo logs --follow picks the logs up again. To stop the job itself, run simulo cancel <job-id>.

The starter trains for only 2 iterations on 16 environments, so this first run is short. Job parameters are command-line flags, so a longer run is:

simulo run train.py --num-envs 512 --max-iterations 600

simulo run train.py --help lists the parameters a job accepts. The starter’s job stops after 10 minutes (timeout=10 * 60 in train.py); raise that before you start a long run.

The starter pins an exact catalog version, simulo/robot/cartpole:v1. When you change assets, add --frozen --strict-assets to refuse any asset without an exact version, and any asset that is deprecated or hasn’t been validated on the job’s runtime. See Choose and validate assets.

5. Evaluate the policy

Run the starter’s success rule against the best checkpoint:

simulo run eval.py --policy policy_brisk-heron-4f8k2m:best

The 2-iteration starter policy is barely trained. It will likely succeed in few or none of the 100 episodes, and that is expected. Evaluation is how you find that out. At a near-zero result, NEXT says Watch the worst episode; check training progress.

Run the longer training command from step 4 to get a policy worth evaluating:

simulo run train.py --num-envs 512 --max-iterations 600

Here is the start of a report for a better-trained cartpole policy. Your policy’s numbers will differ:

RESULT
  87 of 100 episodes succeeded. Likely range: 79% to 93%.

The report also shows the count for each success check and commands for recorded failures. See Evaluate your policy to read the report and download one.

6. Inspect the policy

Every training job makes one policy. Its ID is the job’s ID with job_ replaced by policy_. A policy holds up to two checkpoints: best, the one with the highest reward during training, and latest, the last one saved. Simulo saves both automatically; your job code doesn’t save them.

simulo policy list
simulo policy get policy_brisk-heron-4f8k2m:best

simulo policy list shows your policies with their status, their checkpoints, and the best reward. simulo policy get downloads one checkpoint and checks its SHA-256 digest before it writes the file:

Saved /home/you/mybot/policy_brisk-heron-4f8k2m-best.pt (sha256 verified)

You always name the checkpoint: policy_brisk-heron-4f8k2m:best or policy_brisk-heron-4f8k2m:latest.

To see everything about one policy at once, inspect it by ID instead of its checkpoints:

simulo policy inspect policy_brisk-heron-4f8k2m

This prints the task and robot it was trained on, the job that made it, what it started from, its tags, and every checkpoint with its iteration and reward. See Policies and checkpoints for a full example.

7. Continue or export

To keep training a policy, start a new training job from one of its checkpoints with --from:

simulo run train.py --from policy_brisk-heron-4f8k2m:latest --max-iterations 20

--max-iterations is the total, including the iterations the checkpoint already has, so this job trains only the ones it doesn’t have yet. The new job makes a new policy. The policy you started from doesn’t change. See Continue training.

To run the policy outside Simulo, export a checkpoint to an ONNX bundle:

simulo export policy_brisk-heron-4f8k2m:best

The bundle holds the ONNX model, its input and output descriptions, test vectors, and a verify.py that checks the model runs on your machine without Simulo installed. See Export a policy to ONNX.

8. Play it back

Run the checkpoint back through its task to watch it or record it:

simulo run play.py --policy policy_brisk-heron-4f8k2m:best

PLAY  job_quiet-cedar-7k3m4q
Policy:  policy_brisk-heron-4f8k2m:best (iteration <n>)
...
[RLPlayer] Completed: 3 episodes, <steps> steps
Next: simulo recordings job_quiet-cedar-7k3m4q

simulo recordings job_quiet-cedar-7k3m4q

The starter’s play.py adds one camera only while playing, so the recording holds the policy’s actions and that camera’s video. If task.py changed since training in a way that no longer matches the checkpoint’s observation or action count, play refuses the job instead of running it; see Play a policy and record it for what that check says and what else it catches.

9. Tag your jobs

A tag is a label you attach when you submit. Repeat --tag to add more than one:

simulo run train.py --tag baseline --tag first-run
simulo jobs --tag baseline
simulo policy list --tag baseline

A job’s tags never change after you submit it, and its policy shows the same tags. Tags use lowercase letters, digits, ., _, and -.

Preview before training: read findings, inspect the recording, fix the task, and rerun.
Observe jobs and policies: logs, results, the Console, and cancelling a job.
Evaluate your policy: measure its success rule and inspect a recorded failure.
Continue training: how --from chooses a starting point.
Play a policy and record it: what --policy checks and what the recording holds.
Choose and validate assets: train a different robot.