AI and Robot Learning
How Much Compute Does On-Robot Inference Need?
A learned policy on a robot needs enough compute for its control rate and its power budget. What perception, planning and policy actually consume, and where the split belongs.
A perception stack with a learned policy typically needs 20 to 200 TOPS of accelerator throughput and draws 15 to 120 W, which on a battery robot is 10 % to 20 % of the whole energy budget. The binding constraint is rarely raw throughput; it is latency at a fixed control rate and the heat the robot can shed.
Where the compute goes
| Workload | Rate | Latency budget | Power | Runs on |
|---|---|---|---|---|
| Camera capture and rectification | 30 to 60 Hz | 5 to 15 ms | 2 to 8 W | Image pipeline or GPU |
| Depth computation | 15 to 30 Hz | 10 to 40 ms | 3 to 15 W | Sensor or accelerator |
| Object detection and segmentation | 5 to 30 Hz | 20 to 80 ms | 5 to 40 W | Accelerator |
| Learned action policy | 3 to 50 Hz | 100 to 300 ms | 10 to 60 W | Accelerator |
| Motion planning | on demand | 10 to 500 ms | 2 to 10 W | CPU |
| State estimation | 200 to 1,000 Hz | < 1 ms | 1 to 3 W | CPU, real-time core |
| Joint control | 1 to 20 kHz | < 0.1 ms | 1 to 2 W | Microcontroller |
Read the rate column downward. The fastest loops need the least compute and the strictest timing, and they belong on dedicated hardware. The slowest loops need the most compute and tolerate the loosest timing, and they belong on an accelerator. Mixing them on one processor is how a perception update ends up delaying a torque command.
Onboard, edge or cloud
| Location | Added latency | Suitable for | Risk |
|---|---|---|---|
| Microcontroller on the joint | < 0.1 ms | Control, safety | None, but tiny capacity |
| Onboard accelerator | 0 ms | Perception, policy | Power and heat |
| Wired edge server in the cell | 1 to 5 ms | Heavy perception, planning | Cable, single point of failure |
| Wireless edge server | 5 to 40 ms | Supervisory, non-critical | Link loss, jitter |
| Cloud | 30 to 300 ms | Fleet learning, offline analysis | Unsuitable for any control loop |
The rule that follows: anything in a control loop runs onboard or on a wired link. Anything that can wait a second runs wherever it is cheapest. Fleets that split this way keep working when the network does not, which is the actual test.
Sizing the module before the robot exists
Compute is chosen early and regretted late. A short calculation before selection avoids the most common outcome, which is a module that works on the bench and throttles in the machine.
| Step | Value | Note |
|---|---|---|
| Perception at 15 Hz | 35 W | Detection plus segmentation |
| Policy at 10 Hz with chunking | 40 W | Chunk of 16 actions |
| Navigation and mapping | 12 W | Runs continuously |
| Operating system and networking | 8 W | |
| Subtotal, sustained | 95 W | |
| Thermal headroom at 65 % utilisation | 146 W | Module must be rated for this |
| Energy per 8 hour shift | 760 Wh | Compute alone |
| Share of a 2.4 kWh pack | 32 % | Before any motion at all |
The last row is the one that changes designs. A third of the battery consumed by computation, before the wheels turn or the arm moves, is a common and unwelcome discovery, and it is why efficient models are worth more on a robot than on a server.
Heat is the real limit
A 60 W compute module inside a sealed robot arm has nowhere to put the heat. Options are all unattractive: a fan, which fails in dusty environments and breaks the ingress protection rating; a heat pipe to the chassis, which adds mass and design constraint; or accepting thermal throttling, which turns a deterministic 80 ms inference into an occasional 200 ms one. Sizing compute at 60 % to 70 % of its thermal envelope rather than at peak is the practical compromise, and it should be verified after an hour of continuous operation rather than from a datasheet.
Frequently asked questions
How much compute does a robot need for learned control?
Typically 20 to 200 TOPS of accelerator throughput at 15 to 120 W for perception plus a learned policy. The exact figure depends far more on the required control rate and model size than on the task.
Can inference run in the cloud?
Not for anything in a control loop. Round-trip latency of 30 ms to 300 ms plus the risk of link loss rules it out. Cloud is appropriate for fleet learning, analytics and offline processing.
How is a 200 ms policy compatible with smooth motion?
Through action chunking. The model predicts a sequence of 8 to 50 future actions in one inference and a lower-level controller executes them, so inference latency is absorbed rather than felt.
What limits onboard compute in practice?
Heat, then power. A sealed arm or a battery robot cannot shed 60 W easily, and thermal throttling turns a predictable inference time into an unpredictable one. Size to 60 % to 70 % of the thermal envelope and verify after an hour of running.
Should everything run on one computer?
No. Joint control belongs on a microcontroller, state estimation on a real-time core, and perception and policy on an accelerator. Mixing them lets a perception update delay a torque command, which is the failure this split exists to prevent.
Sources
- Real-time programming backgroundROS 2 design article on separating deadline-bound from best-effort workloads
- arXiv robotics preprints, on-robot inference and action chunkingPrimary literature on policy inference latency and chunked action prediction
- ROBOTS guideIEEE Spectrum, published power and compute figures for deployed robots