[pull] master from Oneflow-Inc:master #4
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Co-authored-by: oneflow-ci-bot <[email protected]>
Co-authored-by: oneflow-ci-bot <[email protected]>
Co-authored-by: JiangDongHua <[email protected]> Co-authored-by: oneflow-ci-bot <[email protected]>
- 推进解耦cuda nccl和oneflow的深度绑定，重构EagerCclCommMgr及ccl::Comm等模块，方便在kernel里直接使用设备无关的（类似primitive）的ccl通信调用实现，替代直接使用nccl apis，推进后续多设备兼容。 - 后续支持/适配不同设备（cuda/npu/xpu等）时，原则上在kernel以及其他调用通信api的代码处，原则上不应该直接调用类似nccl这样的设备耦合的通信apis，而应该直接使用oneflow::ccl::Send/Recv/AllReduce/.... 等父类api（具体位于`oneflow/user/kernels/collective_communication/include`目录下）并提供子类实现 - 后续各设备需继承oneflow::ccl通信apis实现自己的子类通信apis。 - 如cuda设备需要通过nccl api实现oneflow::ccl::CudaSend/CudaRecv/CudaAllReduce....等。 - npu设备需要通过hccl api实现oneflow::ccl::NpuSend/NpuRecv/NpuAllReduce等
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )