Article
Version 1
Preserved in Portico This version is not peer-reviewed
Improving Existing Segmentators Performance with Zero-Shot Segmentators
Version 1
: Received: 22 July 2023 / Approved: 25 July 2023 / Online: 26 July 2023 (08:25:16 CEST)
A peer-reviewed article of this Preprint also exists.
Nanni, L.; Fusaro, D.; Fantozzi , C.; Pretto , A. Improving Existing Segmentators Performance with Zero-Shot Segmentators. Entropy 2023, 25, 1502. Nanni, L.; Fusaro, D.; Fantozzi , C.; Pretto , A. Improving Existing Segmentators Performance with Zero-Shot Segmentators. Entropy 2023, 25, 1502.
Abstract
This paper explores the potential of using the SAM segmentator to enhance the segmentation capability of known methods. SAM is a promptable segmentation system that offers zero-shot generalization to unfamiliar objects and images, eliminating the need for additional training. The open-source nature of SAM on GitHub allows for easy access and implementation. In our experiments, we aim to improve the segmentation performance by providing SAM with checkpoints extracted from the masks produced by DeepLabv3+, then merging the segmentation masks provided by these two networks. Additionally, we examine the \enquote{oracle} method (as upper bound baseline performance), where segmentation masks are inferred only by SAM with checkpoints extracted from ground truth.
In addition, we tested in the CAMO datasets an ensemble of PVTv2 transformers; combining the ensemble and SAM yields state-of-the-art performance in that dataset.
The results of our study provide valuable insights into the potential of incorporating the SAM segmentator into existing segmentation techniques.
We release with this paper the open-source implementation of our method.
Keywords
segmentation; deep learning; ensemble; SAM zero-shot segmentator
Subject
Computer Science and Mathematics, Computer Science
Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Comments (0)
We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.
Leave a public commentSend a private comment to the author(s)
* All users must log in before leaving a comment