P²SAM

Official PyTorch implementation of the following paper:
Part-aware Personalized Segment Anything Model for Patient-Specific Segmentation
Chenhui Zhao and Liyue Shen
University of Michigan
 

Overview

We propose P²SAM, a training-free method that adapts a promptable segmentation model to new patients using only one-shot patient-specific data. P²SAM incorporates a part-aware prompt mechanism and a distribution-based retrieval approach to enhance generalization across:

• Tasks: P²SAM enhances generalization across different patient-specific segmentation tasks.
• Models: P²SAM can be plugged into various promptable segmentation models, such as SAM, fine-tuned SAM, and SAM 2.
• Domains: P²SAM performs effectively in both medical and natural image domains.

Updates

• (2025-01) Release P²SAM's SAM fine-tuning code and fine-tuned models.
• (2025-01) Release P²SAM code for adaptive nsclc segmentation.
• (2025-01) Release P²SAM code for endoscopy video segmentation.
• (2024-07) Release P²SAM code for personalized segmentation.

Getting Started

Create an Python environment with:

conda env create -f environment.yaml

Prepare datasets with:

• Datasets

Prepare pre-trained and fine-tuned models with:

• Models

Fine-tune SAM on custom datasets (using SAM's iteratively training strategy) with:

• Fine-tuning

Test P²SAM on out-of-distribution datasets with:

• Testing

Acknowledgement

This repository is built using the DEiT, SAM, PerSAM repositories.

Citation

If you find this repository helpful, please consider citing:

@article{zhao2024part,
  title={Part-aware Personalized Segment Anything Model for Patient-Specific Segmentation},
  author={Zhao, Chenhui and Shen, Liyue},
  journal={arXiv preprint arXiv:2403.05433},
  year={2024}
}