In our project a pipeline was created to solve the Pick & Place problem for rectangular objects, and a data collection framework was built for further ML and algorithmic purposes.

The project relies on the uFactory Lite 6 robotic arm and its software API. An integrated system was built around the arm, including solutions to combine a camera and a vacuum pump installed on the robot. The code framework is based on classical Image Processing algorithms and using a pretrained neural network in order to automatically find the working space of the robot, identifying blocks to pick up and their optimal pick-up position, and the management of robot pick & place and data collection.

Our solution reached a measured 86% percent accuracy of correct pick-up and placement of blocks in the systems. Future improvements are varied and include the implementation to deal with block’s edge case position and training a network (RL based) that is able to improve the pick-up estimation.