Robot Software
Real-Time Robot Control on Linux: What PREEMPT_RT Buys You
PREEMPT_RT turns Linux into a usable real-time platform with worst-case latency in the tens of microseconds, but only after the userspace mistakes that dominate jitter are removed.
A tuned Linux system with the real-time preemption patches, mainlined into the kernel in 2024, reaches worst-case scheduling latency in the region of 20 to 100 microseconds on suitable hardware. That is enough for a 1 kHz joint control loop with margin. It is not enough to save code that allocates memory, takes page faults or logs to disk inside the loop, and in practice userspace mistakes cause more jitter than the kernel ever does.
What real-time actually means here
Real-time is about bounded worst case, not about speed. A loop that usually completes in 50 microseconds and occasionally takes 4 milliseconds is not real-time, even though its average is excellent. For robot control the question is whether the deadline is met every cycle, because a missed cycle on a torque controller is a visible motion artefact and, at worst, an instability.
| Configuration | Typical worst case | Suitable for |
|---|---|---|
| Standard kernel, no tuning | 1 to 20 ms | Planning, perception, user interface |
| Standard kernel, tuned | 200 to 1,000 us | 100 Hz to 250 Hz loops |
| Preemptible kernel, tuned | 20 to 100 us | 1 kHz joint control |
| Isolated core, preemptible, tuned | 10 to 40 us | 4 kHz to 10 kHz current loops |
| Dedicated microcontroller | 1 to 5 us | Commutation, safety functions |
The userspace mistakes that dominate
Every one of these has been measured causing millisecond-scale spikes on systems whose kernel latency was already excellent.
- Dynamic memory allocation in the loop. Any allocation can trigger a system call and a page fault. Preallocate everything, then lock memory so it cannot be paged out.
- Page faults on first touch. Memory that is allocated but never written is not resident. Touch every page during initialisation.
- Logging and file I/O. A write to disk in the control path is a blocking operation of unbounded duration. Log to a preallocated ring buffer and drain it from a non-real-time thread.
- Priority inversion. A high-priority thread waiting on a lock held by a low-priority one. Use priority-inheriting mutexes, and prefer lock-free structures at the boundary.
- Frequency scaling and idle states. Deep processor sleep states cost tens of microseconds to exit. Disable them on the control core.
- Interrupt affinity. A network interrupt landing on the control core steals time. Pin interrupts elsewhere and isolate the core.
Building a loop budget
At 1 kHz the period is 1,000 microseconds and the budget breaks down roughly as follows for a joint controller on a mid-size arm.
| Item | Typical | Note |
|---|---|---|
| Scheduling latency | 20 to 100 us | Kernel contribution |
| Fieldbus receive | 50 to 150 us | Depends on cycle alignment |
| State estimation | 30 to 120 us | Filtering, kinematics |
| Control computation | 50 to 300 us | Rises sharply with whole-body optimisation |
| Safety and limit checks | 10 to 40 us | Never skip under load |
| Fieldbus transmit | 30 to 100 us | |
| Total worst case | 190 to 810 us | Against a 1,000 us deadline |
The margin looks comfortable until the control computation grows. Whole-body optimisation at 1 kHz is where budgets break, which is why many systems run balance and whole-body control at 200 Hz to 500 Hz and only the joint loop at 1 kHz or above.
The architecture most robots end up with
A split, and it is worth designing deliberately rather than discovering.
- Microcontroller or FPGA for commutation, current control and hard safety functions, at microsecond scale.
- Preemptible Linux with isolated cores for joint and whole-body control at 200 Hz to 2 kHz.
- Standard Linux for perception, planning, logging and communication, with no deadline guarantee.
Keeping the boundary explicit is what prevents a perception update from delaying a torque command, and it is the single most useful structural decision in a robot software stack.
Frequently asked questions
Can Linux do hard real-time control?
With the real-time preemption support, isolated cores and careful userspace, worst-case latency reaches roughly 20 to 100 microseconds, which is sufficient for 1 kHz joint control. Microsecond-scale commutation and safety functions still belong on dedicated hardware.
What causes latency spikes if the kernel is tuned?
Almost always userspace: memory allocation in the loop, page faults, file logging, priority inversion on a mutex, or processor idle states and frequency scaling on the control core.
How long should a latency test run?
Hours, under realistic system load. Spikes cluster around rare events such as network bursts, filesystem flushes and thermal throttling, so a short test systematically underestimates the worst case.
What loop rate does a robot actually need?
Joint position and torque control typically runs at 1 kHz, current control at 4 kHz to 20 kHz on dedicated hardware, and whole-body or balance control at 200 Hz to 1 kHz. Planning and perception run far slower without deadline guarantees.
Is a real-time kernel enough on its own?
No. It removes the kernel as a source of unbounded delay, which is necessary but not sufficient. Preallocated and locked memory, priority-inheriting locks, interrupt affinity and core isolation are all required to realise the benefit.
Sources
- Linux kernel documentation, real-time schedulingKernel documentation on real-time scheduling classes and throttling
- Real-time Linux projectLinux Foundation, status of the preemption work and tuning guidance
- Real-time programming backgroundROS 2 design article on real-time requirements in robot software