Grasping an object using a robotic arm faces many complexities regarding object identification when multiple objects are present, along with the positioning, angle, and gripper key to achieve a good grasp. For this task, several libraries offering computer vision-based models were examined, and the ur5_robotic_grasping library was selected, which uses the GR-ConvNet (Generative Residual Convolutional Neural Network) model. The model provides results that contribute to determining the optimal position and orientation for grasping an object with a robotic arm, using color and depth cameras.
The library used in the project provides a simulation environment for the model, as well as pre-trained models based on the Cornell Grasping Dataset. The library also offers a wrapper for using the model and support for converting model outputs into actions for the robotic arm, such as converting a pixel in an image to a point in the robot’s workspace.