PlantSAM: An object detection-driven segmentation pipeline for herbarium specimens.

Premise: Deep learning-based classification of herbarium images is hampered by background heterogeneity, which introduces noise and artifacts that can potentially mislead models and degrade their accuracy. Addressing these effects is essential to enhance overall performance. Methods: We introduce PlantSAM, an automated segmentation pipeline that integrates YOLOv10 for plant region detection and the Segment Anything Model (SAM2) for segmentation. YOLOv10 generates bounding box prompts to guide SAM2, enhancing segmentation accuracy. Both models were fine-tuned on herbarium images and evaluated using intersection over union (IoU) and Sørensen-Dice coefficient metrics. Results: PlantSAM achieved state-of-the-art segmentation performance, with an IoU of 0.94 and a Sørensen-Dice coefficient of 0.97. Incorporating segmented images into classification models led to consistent performance improvements across five tested botanical traits, with accuracy gains of up to 4.36% and F1 score improvements of 4.15%. Conclusions: Our findings highlight the importance of background removal in herbarium image analysis, as it significantly enhances classification performance by enabling models to focus more effectively on the foreground plant structures.