Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add impersonate feature to API /v1/chat/completions #6342

Open
wants to merge 4 commits into
base: dev
Choose a base branch
from

Conversation

Yiximail
Copy link
Contributor

@Yiximail Yiximail commented Aug 22, 2024

The feature same as the impersonate feature on webui.

Add an impersonate param to active this feature in /v1/chat/completions.

Updated:
Prepend user_input in impersonate reply for consistency
It will always returns the entire message now

Checklist:

Test Images

Impersonate:
image

Normal response:
image

@Yiximail
Copy link
Contributor Author

Yiximail commented Oct 31, 2024

Is there something wrong with it...? You can tell me, and I can fix it
Sorry for the mention @oobabooga

I use this quite often in api.
I just updated the webui today and realized that it wasn't merged.

@jianmomo
Copy link

Is there something wrong with it...? You can tell me, and I can fix it Sorry for the mention @oobabooga

I use this quite often in api. I just updated the webui today and realized that it wasn't merged.
I'm not sure what issues there might be with my modifications.
I directly copied the contents of typing.py and completions.py, and added a parameter for mimicry in the request. However, there's a problem with the responses—they always start with "ai...".

image

@Yiximail
Copy link
Contributor Author

Yiximail commented Nov 29, 2024

Is there something wrong with it...? You can tell me, and I can fix it Sorry for the mention @oobabooga
I use this quite often in api. I just updated the webui today and realized that it wasn't merged.
I'm not sure what issues there might be with my modifications.
I directly copied the contents of typing.py and completions.py, and added a parameter for mimicry in the request. However, there's a problem with the responses—they always start with "ai...".

image

咳咳,这图不合适吧

请提供一下你的模型,和完整prompt(运行的时候加上 --verbose 参数) 带上seed更好

一般这个commit不影响生成内容才对,只是处理了一下让它不会创建新的assistant对话,而是接着在user的content中续写补充


啊,我知道了,这是chat模式的bug

image

设置了 bot_name 但是没生效

image

它读取的依然是默认的 Assistant 角色卡 的内容,bot_name 没覆盖它
那和这个commit没什么关系啦,不一起改了

(好少用chat模式……)

@jianmomo
Copy link

jianmomo commented Dec 2, 2024

OK, thank you. It seems that I need to learn the framework code carefully. Thank you very much for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants