Armnet Robots¶

A cell's arms and cameras are attached to its edge, a Raspberry Pi next to the robot. Your runtime runs elsewhere and reaches them over the network. There are two ways to drive them from LeRobot code, and both give the same observations and take the same actions.

	Round trips to observe	Round trips to act
Plain LeRobot robots	one per arm, one per camera, one per depth map	one per arm
Armnet robots	one	one

Armnet's own record, teleop and eval runtimes use the Armnet robots. A runtime of your own can use either.

Plain LeRobot robots¶

Unmodified LeRobot code runs on a cell. The runtime's import swap replaces LeRobot's motor buses and cameras with proxies that forward each call to the edge:

from lerobot.robots.so_follower import SO101Follower, SO101FollowerConfig

robot = SO101Follower(
    SO101FollowerConfig(
        port=ctx.cell.robot_port,
        id=ctx.cell.robot_id,
        cameras=ctx.camera_configs,
    )
)

Every device is its own round trip. An SO-101 with a wrist and an overhead camera costs three to observe and one to act, and a depth map read on top is another. The round trips run one after another, so on a 30 fps loop they add up. Use these when you want code that also runs on a robot plugged into your own machine.

Armnet robots¶

The Armnet robots are LeRobot robots built to keep round trips down. get_observation() asks the edge for every arm's joints and every camera's newest frame in one request, and send_action() writes every arm in one request. They live in armnet_runtime.lerobot:

Robot	Config	Replaces
`ArmnetSO101Follower`	`ArmnetSO101FollowerConfig`	`SO101Follower`
`ArmnetBiSO101Follower`	`ArmnetBiSO101FollowerConfig`	`BiSOFollower`
`ArmnetYAMFollowerRobot`	`ArmnetYAMFollowerConfig`	the YAM plugin's `YAMFollowerRobot`
`ArmnetBiYAMFollowerRobot`	`ArmnetBiYAMFollowerConfig`	two YAM followers driven as a pair

The SO-101 robots subclass LeRobot's SOFollower and BiSOFollower, so observation_features, action_features, calibration and the observation and action keys are LeRobot's own. A dataset recorded with one replays on the other.

from armnet_runtime.lerobot import ArmnetSO101Follower, ArmnetSO101FollowerConfig

robot_id, calibration_dir = ctx.cell.prepare_calibration_dir()
robot = ArmnetSO101Follower(
    ArmnetSO101FollowerConfig(
        port=ctx.cell.robot_port,
        id=robot_id,
        calibration_dir=calibration_dir,
        cameras=ctx.camera_configs,
        depth_cameras=["top"],
    )
)
robot.connect(calibrate=False)
try:
    observation = robot.get_observation()  # one round trip
    robot.send_action(action)              # one round trip
finally:
    robot.disconnect()

A bimanual SO-101 cell takes one config per arm, and cameras that belong to neither arm go on the pair:

from lerobot.robots.so_follower import SOFollowerConfig
from armnet_runtime.lerobot import ArmnetBiSO101Follower, ArmnetBiSO101FollowerConfig

calibration = ctx.cell.prepare_bimanual_calibration_dir()
robot = ArmnetBiSO101Follower(
    ArmnetBiSO101FollowerConfig(
        id=calibration.robot_id,
        calibration_dir=calibration.calibration_dir,
        left_arm_config=SOFollowerConfig(port=ctx.cell.arm("left").robot_port, cameras={"wrist": ...}),
        right_arm_config=SOFollowerConfig(port=ctx.cell.arm("right").robot_port, cameras={"wrist": ...}),
        cameras={"top": ...},
    )
)

The arm cameras come back as left_wrist and right_wrist, the shared ones under their own names (top). That is LeRobot's naming for a bimanual robot, plus the shared cameras BiSOFollower has no place for.

The YAM robots do not need LeRobot's YAM plugin, nor a CAN stack in your image: the edge drives the arms. left_arm and right_arm of the bimanual robot are ArmnetYAMFollowerRobots, so code that commands an arm directly (robot.left_arm.command_joint_pos(...)) keeps working.

What the edge does¶

The edge reads the joints first, then collects each camera's newest frame, which its capture threads have already encoded. The reply carries how old each frame was, in robot.last_frame_ages_ms, keyed by camera name plus "joints". A write is bounded by the edge before it reaches the motors, so these robots take no max_relative_target: the edge already holds each goal within the arm's limit of where the joint is, against a position read microseconds before the write. send_action() returns the goal as the edge wrote it.

Depth¶

Name the cameras whose depth you want in depth_cameras. Each observation then also brings back their depth maps, in robot.last_depth, keyed by camera name. A depth map and the color frame of the same name come from the same capture, so they line up during fast motion. The color frame can be one frame older than the newest one the camera has, because the depth map finishes encoding after it.

The maps arrive as PNG and stay that way until you call .decode(), which returns uint16 millimetres. Decoding a 1280×720 map takes about 5 ms, a sixth of a 30 fps tick, so do it off the control loop.

observation = robot.get_observation()
depth_png = robot.last_depth["top"]
# elsewhere, on another thread:
depth_mm = depth_png.decode()

The depth maps are not in the observation dict, so observation_features and a dataset built from it are the same as LeRobot's.

When a device drops out¶

A wrist camera that falls off the USB bus stops sending frames, and the edge reports it. The robot then reports itself not connected and the call raises LeRobot's DeviceNotConnectedError, as LeRobot's own robots do. The arms stay held by the edge: disconnect() leaves them there, and connect() attaches again once an observation comes back whole. Retrying disconnect then connect until it succeeds is how Armnet's eval runtimes recover from a camera blip.

is_connected is answered locally, without a round trip.

Edge version¶

These robots need an edge that serves robot_get_observation and robot_send_action. Against an older one the first observation fails with an error that names the edge as the problem. Plain LeRobot robots work with either.

Both kinds of robot ask the edge to send images as raw bytes rather than base64 text, which makes a reply about a quarter smaller. An edge too old to do that sends base64, and both read it.