Skip to content

Teleoperation

Teleoperation lets an operator drive the robot in real time, with live camera video.

  • Direction: Downlink (cloud → robot) for control input
  • Topic: the command key on the local Zenoh session (or a configured subtopic under command/**)
  • Encoding: Cap'n Proto over Zenoh

Teleop has two parts: control input (gamepad/joystick) delivered as commands, and live video streamed from the robot's cameras.

Control input (gamepad / joy)

Operator gamepad input is delivered to the robot as a teleop command (TeleopJoy). Your robot software subscribes to the command topic and applies it to motion control.

Field Notes
axes Analog stick/trigger values.
buttons Button states.
timestamp Epoch-ms sample time.

For the complete TeleopJoy schema (sourceId, sessionId, axes, buttons, timestampMs), see Command Messages → teleopJoy.

!!! tip Treat teleop as a safety-critical, latency-sensitive stream. Apply a watchdog: stop the robot if teleop samples stop arriving.

Live video

During a teleop session the agent streams V4L2 camera video to the operator over a real-time peer connection. Up to two camera tracks can be active and switched live. Configure cameras in Camera Configuration.

Session flow (conceptual)

  1. Operator starts a teleop session from the cloud.
  2. The agent starts camera streaming; video flows robot → operator.
  3. Operator gamepad input flows cloud → robot as TeleopJoy commands on the local command key.
  4. On session end, video stops; resume normal operation.