We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
vLLM中存在生成参数allowed_token_ids,可以控制LLM在推理时仅从给定的token中生成。 类似于LMDeploy中bad_token_ids的反选?
我在LMDeploy的bad_token_ids加入了除我希望生成的token以外所有token的ids,但这会导致程序崩溃。
希望能在LMDeploy的生成参数中加入类似allowed_token_ids功能的参数,或者能否告诉我现有的版本能够如何实现这样的功能。
vLLM: https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py#L87
No response
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Motivation
vLLM中存在生成参数allowed_token_ids,可以控制LLM在推理时仅从给定的token中生成。
类似于LMDeploy中bad_token_ids的反选?
我在LMDeploy的bad_token_ids加入了除我希望生成的token以外所有token的ids,但这会导致程序崩溃。
希望能在LMDeploy的生成参数中加入类似allowed_token_ids功能的参数,或者能否告诉我现有的版本能够如何实现这样的功能。
Related resources
vLLM: https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py#L87
Additional context
No response
The text was updated successfully, but these errors were encountered: