This project focuses on making the Segment Anything Model (SAM), better at automatic segmentation and more useful for robotic automation tasks. SAM has been trained on a huge dataset and can handle segmentation well when given manual prompts such as points or boxes. However, it struggles when it comes to fully automatic segmentation without manual prompts.
To train the automatic segmentation, the conventional approach creates 1024 points, each generating a mask compared to the ground truth, which greatly increases computation time and power. In contrast, our approach creates 1024 points per object, labeling them 1 for positive ground truth and 0 otherwise, resulting in n masks corresponding to the number of objects in the image. We applied a method called iterative sampling, which helps catch and correct false negative areas. Unlike the usual approach of training only the mask decoder, we trained also the prompt encoder. Our results showed a clear improvement, with the F1 score for point prompts on mixed-object data increased from 0.87 to 0.96. We demonstrated that high-quality SAM fine-tuning can be performed using standard consumer GPUs, and our training methods made automatic segmentation more accurate and efficient.