Dex-net includes a high-resolution 3-D sensor and two arms, each controlled by a different neural network. One arm is equipped with a conventional robot gripper and another with a suction system. The robot’s software scans an object and then asks both neural networks to decide, on the fly, whether to grab or suck a particular object. It runs on an off-the-shelf industrial machine made by Swiss robotics company ABB.
The software learns by attempting to pick up objects in a virtual environment. Dex-Net can generalize from an object it has seen before to a new one. The robot can "nudge" such virtual objects to examine it if it is unsure how to grasp it. The trial data set was 6.7 million point clouds, grasps and analytic grasp metrics generated from thousands of 3D models. Grasps are defined as a gripper's planar position, angle and depth relative to an RGB-D sensor.
Mean picks per hour
A metric called mean picks per hour (MPPH) is calculated by multiplying the average time per pick and the average probability of success for a specific set of objects. The new metric allows labs working on picking robots to compare their results.
- Knight, Will. "Exclusive: This is the most dexterous robot ever created". MIT Technology Review. Retrieved 2018-04-04.
- Mahler, Jeffrey; Liang, Jacky; Niyaz, Sherdil; Laskey, Michael; Doan, Richard; Liu, Xinyu; Ojea, Juan Aparicio; Goldberg, Ken (2017-03-27). "Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics". arXiv:1703.09312 [cs.RO].