Generation never stop until hits the limit.

[mortilla]

I'm using SillyTavern as a client. The same model in TGWUI works fine.

The log:

INFO: Generation options: {'max_tokens': 300, 'temperature': 0.7, 'temperature_last': False, 'smoothing_factor': 0.0, 'top_k': 0, 'top_p': 0.9, 'top_a': 0.0, 'min_p': 0.0, 'tfs': 1.0, 'typical': 1.0, 'mirostat': False, 'max_temp': 0.0, 'min_temp': 0.0, 'temp_exponent': 1.0, 'mirostat_tau': 3.5, 'mirostat_eta': 0.09, 'token_repetition_penalty': 1.0, 'token_frequency_penalty': 0.0, 'token_presence_penalty': 0.0, 'token_repetition_range': 0, 'token_repetition_decay': 0, 'token_healing': False, 'auto_scale_penalty_range': False, 'generate_window': 512, 'add_bos_token': True, 'ban_eos_token': False, 'stop_conditions': ['\nsome_user:', '<|im_start|>user', '<|im_end|>', '<|im_start|>assistant'], 'logit_bias': None}
INFO:
INFO: Metrics: 300 tokens generated in 18.91 seconds (15.87 T/s, context 1294 tokens)

[bdashore3]

I'd need a bit more information for this.

1. What's the model?
2. What's the prompt and response when this does happen?

It's also important to note that generation lengths vary a lot depending on multiple factors such as prompt/character, model, and sampling settings. Instruct modes and system prompts can also play a role.

[mortilla]

As I were collecting the information per your request, I noticed that other models (namely Panchovix/goliath-120b-exl2) works as expected. The issue is only reproducible with the model I converted myself from DiscoResearch/DiscoLM-120b.