Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
mabaochang authored Mar 27, 2023
1 parent e0f5b5e commit 1f172e5
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions gptq/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ CUDA_VISIBLE_DEVICES=0 python test_kernel.py
## Model inference with the saved model
```
# BELLE-7B-gptq: local saved model path from Huggingface
git clone https://huggingface.co/BelleGroup/BELLE-7B-gptq
# model inference with the saved model
CUDA_VISIBLE_DEVICES=0 python bloom_inference.py BELLE-7B-gptq --wbits 8 --groupsize 128 --load BELLE-7B-gptq/bloom7b-2m-8bit-128g.pt --text "hello"
```
Expand Down

0 comments on commit 1f172e5

Please sign in to comment.