The custom segment length cannot exceed 1000 anymore?

[mjmjmmmjjj]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.15.1

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot]

@mjmjmmmjjj Hello there! 👋 I'm here to help you with any bugs, questions, or contributions you might have.

Yes, the custom segment length in Dify is currently limited to a maximum of 1000 tokens. This is enforced by the configuration setting INDEXING_MAX_SEGMENTATION_TOKENS_LENGTH: 1000 in the docker-compose.yaml file. The code in api/core/indexing_runner.py also confirms that any custom segment length must be between 50 and 1000 tokens ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[crazywoola]

This is only for bug report, if you would like to ask a question, please head to Discussions.