[Improve][Zeta] Improve the Success Rate of Resource Allocation for Large Tasks in Dynamic Slot Mode.

[xiaochen-zhou]

Purpose of this pull request

By selecting workers with the least remaining eligible resources, aim to preserve those with larger resources to improve the success rate of tasks requiring significant resources.

Does this PR introduce any user-facing change?
no

How was this patch tested?
exist tests

Check list

• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• If you are contributing the connector code, please check that the following files are updated:
  1. Update plugin-mapping.properties and add new connector information in it
  2. Update the pom file of seatunnel-dist
  3. Add ci label in label-scope-conf
  4. Add e2e testcase in seatunnel-e2e
  5. Update connector plugin_config
• Update the release-note.

[hailin0]

cc @Hisoka-X

[Hisoka-X]

why? cc @zhangshenghang

[Hisoka-X]

Hi @xiaochen-zhou , this PR seems to do the same optimization as #8233.

[Hisoka-X]

This change will cause unbalanced load in the cluster. Why not consider controlling job scheduling through tags? https://seatunnel.apache.org/docs/2.3.8/seatunnel-engine/resource-isolation

[xiaochen-zhou]

This change will cause unbalanced load in the cluster. Why not consider controlling job scheduling through tags? https://seatunnel.apache.org/docs/2.3.8/seatunnel-engine/resource-isolation

This change will cause unbalanced load in the cluster. Why not consider controlling job scheduling through tags? https://seatunnel.apache.org/docs/2.3.8/seatunnel-engine/resource-isolation

This operation is after tag_filer and targets the success rate within a team. For example, I have three teams: SLA, High Priority, Medium Priority, and Low Priority. I want to improve the success rate of large resource tasks within the high priority tasks.

[xiaochen-zhou]

This change will cause unbalanced load in the cluster. Why not consider controlling job scheduling through tags? https://seatunnel.apache.org/docs/2.3.8/seatunnel-engine/resource-isolation

However, I think that the unbalanced load is only at the team level, and I feel that it won't have any negative impact.

[Hisoka-X]

This change will cause unbalanced load in the cluster. Why not consider controlling job scheduling through tags? https://seatunnel.apache.org/docs/2.3.8/seatunnel-engine/resource-isolation

However, I think that the unbalanced load is only at the team level, and I feel that it won't have any negative impact.

OK, we should add new strategy not change the RANDOM strategy bahavior.

[hailin0]

This change will cause unbalanced load in the cluster. Why not consider controlling job scheduling through tags? https://seatunnel.apache.org/docs/2.3.8/seatunnel-engine/resource-isolation

However, I think that the unbalanced load is only at the team level, and I feel that it won't have any negative impact.

OK, we should add new strategy not change the RANDOM strategy bahavior.

+1

[liunaijie]

Hi @xiaochen-zhou , this PR seems to do the same optimization as #8233.

+1, looks it's similar. cc @zhangshenghang

[zhangshenghang]

> Hi @xiaochen-zhou , this PR seems to do the same optimization as #8233. +1, looks it's similar. cc @zhangshenghang

@xiaochen-zhou Hi , because I made modifications to the structure, you can wait until # 8223 is merged before adding modifications, and I will complete the PR optimization as soon as possible .

…

2024年12月9日 下午5:11，Jarvis ***@***.***> 写道： Hi @xiaochen-zhou <https://github.com/xiaochen-zhou> , this PR seems to do the same optimization as #8233 <#8233>. +1, looks it's similar. cc @zhangshenghang <https://github.com/zhangshenghang> — Reply to this email directly, view it on GitHub <#8242 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHAOLX5EDZUHHI5T6DYYA4D2EVNFTAVCNFSM6AAAAABTH7FOP2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRXGM2DGMJSGM>. You are receiving this because you were mentioned.

[xiaochen-zhou]

Hi @xiaochen-zhou , this PR seems to do the same optimization as #8233. +1, looks it's similar. cc @zhangshenghang
@xiaochen-zhou Hi , because I made modifications to the structure, you can wait until # 8223 is merged before adding modifications, and I will complete the PR optimization as soon as possible .
…
2024年12月9日 下午5:11，Jarvis @.***> 写道： Hi @xiaochen-zhou https://github.com/xiaochen-zhou , this PR seems to do the same optimization as #8233 <#8233>. +1, looks it's similar. cc @zhangshenghang https://github.com/zhangshenghang — Reply to this email directly, view it on GitHub <#8242 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHAOLX5EDZUHHI5T6DYYA4D2EVNFTAVCNFSM6AAAAABTH7FOP2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRXGM2DGMJSGM. You are receiving this because you were mentioned.

ok.