Recordings & live view

Simulo gives you two ways to see what a job did beyond its logs: MCAP recordings you download and inspect after the run, and a live view of the 3D scene you watch in your browser while the job runs. They are independent: a job can have either, both, or neither.

MCAP recordings

A recording is an MCAP file: time-stamped messages such as observations, actions, rewards, and episode boundaries, and sometimes camera video. A job’s recordings come from five places:

• A preview job. simulo run preview.py records the robot state, each report finding on its timeline, and video by default. Downloading it also writes ready-made Lichtblick and Foxglove layouts. See Preview.

• A play job. simulo run play.py --policy policy_x:best records the policy’s actions and, if the task adds a camera for play, that camera’s video. See Play a policy and record it.

• An evaluation job. simulo run eval.py --policy policy_x:best records representative failed episodes so you can inspect why the success rule did not pass. If every episode succeeds, it records up to three of the lowest-reward successes. The download includes ready-made Lichtblick and Foxglove layouts. See Evaluation.

• Anomaly captures. A training job that declares DebugOnAnomaly writes a short recording of the steps before something went wrong. See Callbacks and anomaly capture.

• MCAP files the job writes. An .mcap file your job writes into a mounted volume during the run is uploaded as one of its recordings.

Recordings upload just after the job reaches its final status. MCAP files your job wrote to a volume are uploaded when it ends completed, failed, or timed_out. Anomaly captures are uploaded only when the job ends completed. A cancelled job’s recordings are never kept.

Download them with simulo recordings:

simulo recordings                              # the only recording from your most recent job you submitted (train, play, preview or eval)
simulo recordings JOB_ID                       # a specific job
simulo recordings JOB_ID NAME -o ./run.mcap    # one of several, by name
simulo recordings JOB_ID --all -o ./out/       # every recording, into a folder

With several recordings and no NAME or --all, the command lists their names so you can pick one. simulo outputs JOB_ID shows recordings and checkpoints together.

Open a recording in Foxglove or Lichtblick, or read it with the mcap Python library. Record and inspect MCAP walks through the useful panels and topics.

Live view

To watch a training job while it runs, submit it with --viewstream, then run simulo view:

simulo run train.py --viewstream --num-envs 512 --max-iterations 600 --detach
simulo view

simulo view opens your browser onto the running job’s 3D scene. Leave out the job ID to use your most recent job you submitted (train, play, preview or eval). If the job is still queued or the simulation is still starting, the command waits, so you run it once. A job submitted without --viewstream can’t be viewed, and a finished job is refused.

The browser viewer includes Rotate and Pan camera modes, camera framing and environment-focus controls, live connection and job status, job details, and a link to the same job in the Simulo Console. A job has one live view at a time. The viewer reconnects automatically after a brief interruption, and closes with the job’s final status when the job stops.

Caution

Streaming renders every step, so training runs noticeably slower while you watch: roughly four to five times slower, measured on a 512-environment benchmark. Use it to see what the robot is doing, and leave it off when speed matters.

Note

--viewstream isn’t part of a job’s content-addressed identity, so it can’t tell two otherwise-identical submits apart. Resubmit the same command with a different --viewstream while the first is still queued or running, and Simulo refuses it instead of silently attaching to the wrong one: wait for the first to finish, or cancel it, then resubmit with the flag you want.

Watch a run live covers the viewer itself.