Skip to content

Commit

Permalink
Merge pull request LianjiaTech#552 from wen2cheng/main
Browse files Browse the repository at this point in the history
Update readme
  • Loading branch information
shuaijiang authored Nov 24, 2023
2 parents b4513ff + 105b780 commit 12a52d5
Showing 1 changed file with 24 additions and 0 deletions.
24 changes: 24 additions & 0 deletions models/Belle-VL/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,30 @@

* 多模态指令数据:指令微调阶段,数据主要来自[LLava](https://github.com/haotian-liu/LLaVA), [LRV-Instruction](https://github.com/FuxiaoLiu/LRV-Instruction), [LLaVAR](https://github.com/SALT-NLP/LLaVAR),[LVIS-INSTRUCT4V](https://github.com/X2FD/LVIS-INSTRUCT4V)等开源项目,我们也对其中部分数据进行了翻译,在此真诚的感谢他们为开源所做出的贡献!

### 模型使用
``` python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_dir = '/path/to_finetuned_model/'
img_path = 'you_image_path'
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_dir, trust_remote_code=True).eval()
model.generation_config = GenerationConfig.from_pretrained(model_dir, trust_remote_code=True)
question = '详细描述一下这张图'

query = tokenizer.from_list_format([
{'image': img_path}, # Either a local path or an url
{'text': question},
])
response, history = model.chat(tokenizer, query=query, history=None)
print(response)

#or
query = f'<img>{img_path}</img>\n{question}'
response, history = model.chat(tokenizer, query=query, history=None)
print(response)
```

### [MME Benchmark](https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Evaluation)
| Category | Score |
|------------------------|-------|
Expand Down

0 comments on commit 12a52d5

Please sign in to comment.