Perception and Sensors
Bin Picking: What 3D Vision Accuracy Do You Actually Need?
Bin picking rarely fails on depth accuracy. It fails on segmentation, reachability and collision. Here is the accuracy that matters and the error budget that decides the cell.
Pose accuracy of 0.5 to 2 mm and 1 to 3 degrees is enough for most bin picking, because the gripper absorbs the rest. What actually breaks cells is segmentation of touching parts, reachability inside the bin and collision with bin walls. Buying a more accurate sensor to fix those is the most common and most expensive misdiagnosis in the field.
The error budget
Accuracy at the pick is the sum of several contributions, and the sensor is usually not the largest one.
| Source | Typical error | Reduced by |
|---|---|---|
| Sensor depth noise | 0.2 to 1.0 mm | Better sensor, averaging, lighting |
| Pose estimation from the point cloud | 0.3 to 2.0 mm | CAD model matching instead of geometric heuristics |
| Hand-eye calibration | 0.2 to 1.5 mm | Proper calibration procedure, rigid mounting |
| Robot absolute accuracy | 0.5 to 3.0 mm | Kinematic calibration |
| Part movement during approach | 0 to 5 mm | Slower approach, stable bin, no vibration |
| Gripper compliance | absorbs 2 to 5 mm | Chamfered fingers, floating mount |
What actually causes failures
- Segmentation of touching parts. Two identical components in contact frequently merge into one detected object, producing a grasp pose between them. This is a geometry and algorithm problem, not a resolution problem.
- Reachability. A pose can be perfectly measured and unreachable, because the required approach vector puts the arm through a bin wall or into a joint limit. Deep bins with tall walls are the usual cause.
- Collision during extraction. The grasp succeeds and the part collides on the way out, because the planner considered the gripper but not the held object's swept volume.
- The last layer. Parts lying flat against the bin floor have almost no graspable feature exposed. Cells that work at 80 % full often stall at 15 % full.
- Surface behaviour. Oily, shiny or freshly machined parts return noisy depth, and the accuracy printed on the datasheet was measured on a matte target.
The fixes that work, ranked
| Intervention | Typical benefit | Cost |
|---|---|---|
| Robot kinematic calibration | +5 to 15 pp success | low |
| Shallower bin or tilting fixture | +10 to 25 pp | low |
| Gripper with mechanical compliance | +5 to 15 pp | low |
| CAD-based pose estimation | +5 to 20 pp | medium |
| Controlled lighting enclosure | +5 to 20 pp | medium |
| Bin shaking or agitation | +5 to 15 pp on the last layer | medium |
| Higher accuracy sensor | +0 to 5 pp | high |
The ordering is deliberate and it holds across most installations. The cheapest interventions are mechanical and geometric; the most expensive one is the sensor, and it usually delivers the least.
Cycle time is the other requirement
Accuracy is only half the specification. A bin picking cell has to produce a pick every so often, and the perception stage is a serial part of that cycle.
| Stage | Typical time | Can it overlap |
|---|---|---|
| Scene capture | 0.2 to 1.5 s | Yes, during the previous place |
| Point cloud processing | 0.2 to 1.0 s | Yes |
| Pose estimation and grasp ranking | 0.3 to 2.0 s | Yes |
| Reachability and collision check | 0.1 to 0.8 s | Partly |
| Robot motion to pick | 1.0 to 2.5 s | No |
| Grasp and extract | 0.8 to 2.0 s | No |
| Motion to place | 1.0 to 2.5 s | No |
| Serial total if nothing overlaps | 3.6 to 12.3 s | |
| With perception overlapped | 2.8 to 7.0 s |
Overlapping perception with the previous place motion is worth 20 % to 40 % of the cycle and costs nothing but software structure. Cells that capture the scene only after the robot returns are leaving that entirely on the table.
Specifying the requirement
- State pose accuracy at the working distance, on the actual surface finish, not on a calibration target.
- State it as end-to-end place accuracy including robot error, not sensor accuracy alone.
- Specify the bin emptying percentage required, because the last layer is a different problem from the first.
- Specify behaviour with touching and overlapping parts, which is where most real failures occur.
Frequently asked questions
How accurate must 3D vision be for bin picking?
Typically 0.5 mm to 2 mm in position and 1 to 3 degrees in orientation, because a compliant gripper absorbs several millimetres. Higher accuracy rarely improves success, because the failures are elsewhere.
Why does my bin picking cell fail even with a good camera?
Most often segmentation of touching parts, reachability inside a deep bin, collision during extraction, or the last layer where parts lie flat with no graspable feature exposed.
What is the biggest error contributor?
Frequently the robot itself. Absolute accuracy on an uncalibrated arm is 0.5 mm to 3 mm, which is comparable to or larger than the sensor error and is unaffected by buying a better camera.
Why does the cell stall when the bin is nearly empty?
Because parts lying flat on the bin floor expose almost no graspable feature and the approach vector is obstructed by the walls. Tilting fixtures, agitation or a manual last-layer step are the usual remedies.
What should a specification demand?
End-to-end place accuracy on the real surface finish at the working distance, the required bin emptying percentage, and defined behaviour with touching and overlapping parts. Sensor accuracy alone is not a specification.
Sources
- ISO 9283, manipulating industrial robots, performance criteria and test methodsPose accuracy definitions used in the error budget
- Robotics at NISTNational Institute of Standards and Technology, robot performance measurement and test methods
- arXiv robotics preprints, bin picking and pose estimationPrimary literature on segmentation and grasp pose estimation