add inference code

af-74413592 · Apr 21, 2023 · e782492 · e782492
1 parent 5e1159c
commit e782492
Show file tree

Hide file tree

Showing 24 changed files with 1,052 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -16,6 +16,7 @@ https://user-images.githubusercontent.com/22233070/230440971-ad015101-205f-4b02-
 
 
 ## News
+- 2023.4 Inference code and model of SegGPT are available at [here](SegGPT/SegGPT_inference/README.md).
 - 2023.4 We combined SAM and SegGPT to enable one touch for segmentation in all images (一触百通), as well as any segmentation in a video (分割视频中的一切). Check it out [here](https://huggingface.co/spaces/BAAI/SegGPT).
 - 2023.4 Enjoy the [SegGPT demo](https://huggingface.co/spaces/BAAI/SegGPT).
 - 2023.3 Code and model of Painter are available.

diff --git a/SegGPT/README.md b/SegGPT/README.md
@@ -25,13 +25,15 @@ Enjoy the [Demo](https://huggingface.co/spaces/BAAI/SegGPT)
 [[Demo]](https://huggingface.co/spaces/BAAI/SegGPT)
 
 ## **Run the demo**
-- We provide a UI  with gradio for running the demo locally. Running the following command in a terminal will launch the demo: 
+- We provide a UI with gradio for running the demo locally. Running the following command in a terminal will launch the demo: 
     ```
     python app_gradio.py
     ```
 - We combined SAM and SegGPT to enable one touch for segmentation in all images (一触百通), as well as any segmentation in a video.
 - This demo is also hosted on HuggingFace [here](https://huggingface.co/spaces/BAAI/SegGPT).
 
+## **Inference**
+- We provide the code for SegGPT inference, please refer to [SegGPT_inference/README.md](SegGPT_inference/README.md)
 
 <div align="center">
 <image src="rainbow.gif" width="720px" />

diff --git a/SegGPT/SegGPT_inference/.gitignore b/SegGPT/SegGPT_inference/.gitignore
@@ -0,0 +1,5 @@
+output*
+*.pth
+**.pyc
+**/__pycache__/**
+__pycache__
diff --git a/SegGPT/SegGPT_inference/README.md b/SegGPT/SegGPT_inference/README.md
@@ -0,0 +1,57 @@
+## **SegGPT Usage**
+- We release the [SegGPT model](https://huggingface.co/BAAI/SegGPT/blob/main/seggpt_vit_large.pth) and inference code for segmentation everything, as well as some example images and videos.
+### Installation
+```
+git clone https://github.com/baaivision/Painter
+cd Painter/SegGPT/SegGPT_inference && wget https://huggingface.co/BAAI/SegGPT/blob/main/seggpt_vit_large.pth
+pip install -r requirements.txt
+```
+### Usage
+Everything in an image with a prompt.
+```
+python seggpt_inference.py \
+--input_image examples/hmbb_2.jpg \
+--prompt_image examples/hmbb_1.jpg \
+--prompt_target examples/hmbb_1_target.png \
+--output_dir ./
+```
+
+Everything in an image with multiple prompts.
+```
+python seggpt_inference.py \
+--input_image examples/hmbb_3.jpg \
+--prompt_image examples/hmbb_1.jpg examples/hmbb_2.jpg \
+--prompt_target examples/hmbb_1_target.png examples/hmbb_2_target.png \
+--output_dir ./
+```
+
+Everything in a video using a prompt image.
+```
+python seggpt_inference.py \
+--input_video examples/video_1.mp4 \
+--prompt_image examples/video_1.jpg \
+--prompt_target examples/video_1_target.png \
+--output_dir ./
+```
+
+Everything in a video using the first frame as the prompt.
+```
+python seggpt_inference.py \
+--input_video examples/video_1.mp4 \
+--prompt_target examples/video_1_target.png \
+--output_dir ./
+```
+
+Processing a long video with prompts from both a target image and the predictions of the previous NUM_FRAMES frames.
+```
+NUM_FRAMES=4
+python seggpt_inference.py \
+--input_video examples/video_3.mp4 \
+--prompt_target examples/video_3_target.png \
+--num_frames $NUM_FRAMES \
+--output_dir ./
+```
+
+<!-- <div align="center">
+<image src="rainbow.gif" width="720px" />
+</div> -->
diff --git a/SegGPT/SegGPT_inference/examples/hmbb_1.jpg b/SegGPT/SegGPT_inference/examples/hmbb_1.jpg
diff --git a/SegGPT/SegGPT_inference/examples/hmbb_1_target.png b/SegGPT/SegGPT_inference/examples/hmbb_1_target.png
diff --git a/SegGPT/SegGPT_inference/examples/hmbb_2.jpg b/SegGPT/SegGPT_inference/examples/hmbb_2.jpg
diff --git a/SegGPT/SegGPT_inference/examples/hmbb_2_target.png b/SegGPT/SegGPT_inference/examples/hmbb_2_target.png
diff --git a/SegGPT/SegGPT_inference/examples/hmbb_3.jpg b/SegGPT/SegGPT_inference/examples/hmbb_3.jpg
diff --git a/SegGPT/SegGPT_inference/examples/hmbb_3_target.png b/SegGPT/SegGPT_inference/examples/hmbb_3_target.png
diff --git a/SegGPT/SegGPT_inference/examples/video_1.jpg b/SegGPT/SegGPT_inference/examples/video_1.jpg
diff --git a/SegGPT/SegGPT_inference/examples/video_1.mp4 b/SegGPT/SegGPT_inference/examples/video_1.mp4
diff --git a/SegGPT/SegGPT_inference/examples/video_1_target.png b/SegGPT/SegGPT_inference/examples/video_1_target.png
diff --git a/SegGPT/SegGPT_inference/examples/video_2.jpg b/SegGPT/SegGPT_inference/examples/video_2.jpg
diff --git a/SegGPT/SegGPT_inference/examples/video_2.mp4 b/SegGPT/SegGPT_inference/examples/video_2.mp4
diff --git a/SegGPT/SegGPT_inference/examples/video_2_target.png b/SegGPT/SegGPT_inference/examples/video_2_target.png
diff --git a/SegGPT/SegGPT_inference/examples/video_3.jpg b/SegGPT/SegGPT_inference/examples/video_3.jpg
diff --git a/SegGPT/SegGPT_inference/examples/video_3.mp4 b/SegGPT/SegGPT_inference/examples/video_3.mp4
diff --git a/SegGPT/SegGPT_inference/examples/video_3_target.png b/SegGPT/SegGPT_inference/examples/video_3_target.png