REPLAB is a reproducible and self-contained hardware stack (robot arm, camera, and workspace), that aims to drive wide participation by lowering the barrier to entry into robotics and to enable easy scaling to many robots. In this project we used REPLAB’s stack to grasp wood blocks placed on a tray and added new infrastructure for block classification by size. This new infrastructure includes both analytical, and AI based classifiers for the blocks.
The Analytical classifier uses standard image processing methods in order to count pixels of the block’s area and determine its class (small, medium, large, no block).
The AI based classifier uses a CNN which we trained on a large dataset of image-class pairs that we collected both manually and automatically by operating the robot arm to grasp blocks and bring them close to the camera. This classifier was more successful than the analytical one by 25%, with 95% success rate.
In this project we also tried training the CNN to classify the blocks from distance without bringing them in front of the camera. This experiment wasn’t successful due to inaccuracies in the dataset.
Since the REPLAB stack comes with pretrained grasping nets, we decided to try and train the fullimage net with 4k sample grasps that we collected. We compared it with the original trained net and with another non-AI method called principal axis.
The original fullimage was the best of all three (40% success rate), while the newly trained fullimage produced poor results compared to the other two (3% success rate).We concluded that the new fullimage failed due to insufficient number of samples to train on. The original net was trained on 92k samples while ours on 4k.