Train and Deploy a LeRobot Policy

armnet-lerobot-eval is the Armnet-native equivalent of lerobot-eval for managed real-robot cells. It supports all default LeRobot policy architectures, as well as 3rd party plugins via the standard register_third_party_plugins() import system.

Evaluate a Hugging Face Policy

If your policy is public on the Hub, pass the repo ID directly. This one is a diffusion policy trained on the BusyBox green-button task, so you can run it as-is:

armnet-lerobot-eval \
  --policy.path=armnet/busybox_push_green_button_diffusion \
  --armnet.embodiment=lerobot/so-101 \
  --armnet.task=push_green_button \
  --eval.n_episodes=3

Because push_green_button is a BusyBox task, the panel scores each episode itself and stages the next one, so the run needs no operator and the reported success rate comes straight from the hardware.

Upload a Local Policy Checkpoint

If your policy checkpoint is not hosted on Hugging Face Hub, upload it to your Armnet Volume:

armnet volume upload \
  ./outputs/train/my_policy/checkpoints/005000/pretrained_model \
  policies/my_policy

Then refer to it with a volume:// path:

armnet-lerobot-eval \
  --policy.path=volume://policies/my_policy \
  --armnet.embodiment=lerobot/so-101 \
  --armnet.task=push_green_button \
  --eval.n_episodes=10 \
  --eval.batch_size=1 \
  --armnet.episode_time_s=30

The runtime resolves volume://policies/my_policy to ctx.volume.path("policies/my_policy").

Evaluating Private Hugging Face Models

If your policy or dataset requires Hugging Face authentication, store your token as a Armnet Secret:

armnet secret create huggingface-token hf_...

Then request it in your job:

armnet-lerobot-eval \
  --policy.path=your-org/private-policy \
  --armnet.embodiment=lerobot/so-101 \
  --armnet.task=push_green_button \
  --eval.n_episodes=10 \
  --eval.batch_size=1 \
  --secrets HF_TOKEN=huggingface-token

The runtime container receives HF_TOKEN as an environment variable, which Hugging Face libraries can use automatically.

Common Flags

armnet-lerobot-eval \
  --policy.path=volume://policies/my_policy \
  --policy.device=cuda \
  --policy.use_amp \
  --eval.n_episodes=10 \
  --eval.batch_size=1 \
  --eval.fps=20 \
  --eval.start_seed=0 \
  --armnet.episode_time_s=30 \
  --armnet.embodiment=lerobot/so-101 \
  --armnet.task=push_green_button \
  --detach

Result Shape

The job returns:

{
  "policy_path": "...",
  "policy_revision": None,
  "eval_dataset_repo_id": "my-user/eval_push_green_button_20260811_142530",
  "eval_dataset_url": "https://huggingface.co/datasets/...",
  "per_episode": [
    {
      "episode_ix": 0,
      "completed": True,
      "success": True,
      "duration_s": 12.3,
      "ticks": 246,
      "seed": 0,
      "scored_by": "busybox",
      "state_clamped": [],
    }
  ],
  "aggregated": {
    "pc_success": 100.0,
    "n_episodes": 1,
    "n_aborted": 0,
    "eval_s": 12.3,
  },
  "robot_telemetry": {...},
}

scored_by tells you where the verdict came from: the environment's own name when its instrumentation decided ("busybox"), "operator" for a human verdict, or "monitor" for the automated completion model. pc_success is computed over scored episodes only; aborted episodes are counted separately in n_aborted.

Train with Background Armnet Evals

armnet-lerobot-train wraps LeRobot training and watches for new saved checkpoints. When a checkpoint appears, it uploads that checkpoint to your Armnet volume and submits a detached real-robot eval job in the background. Training continues even if the remote eval fails.

This example trains on the public armnet/busybox_push_green_button dataset and evaluates on the task that dataset was recorded for, so the training data and the scoring hardware line up:

armnet-lerobot-train \
  --dataset.repo_id=armnet/busybox_push_green_button \
  --policy.type=act \
  --policy.repo_id=your-org/busybox_push_green_button_act \
  --output_dir=outputs/train/my_policy \
  --steps=20000 \
  --save_freq=2000 \
  --armnet.eval.embodiment=lerobot/so-101 \
  --armnet.eval.task=push_green_button \
  --armnet.eval.n_episodes=10 \
  --armnet.eval.episode_time_s=30 \
  --armnet.eval.secrets HF_TOKEN=huggingface-token

Important notes:

• --output_dir is required so the wrapper can discover checkpoints.
• Remote evals are submitted with detach=True and stream_logs=False.
• The wrapper uploads checkpoints under a volume prefix such as policies/training-evals/<run_name>/<step_id>.
• If the wrapper process has an active wandb run, completed remote eval metrics are logged as remote_eval/*.

Hugging Face Checkpoint Transport

By default, armnet-lerobot-train uploads eval checkpoints to your Armnet volume. You can instead upload eval checkpoints to the policy's Hugging Face model repo:

armnet-lerobot-train \
  ... \
  --armnet.eval.use_hf_checkpoints

When enabled, each eval checkpoint is uploaded to a Hub branch named:

eval-step_005000

and the remote eval job is launched with:

policy_path=<policy repo id>
policy_revision=eval-step_005000

This keeps main available for the latest/final model while allowing old eval checkpoints to be reloaded by branch name.