[Driver][SYCL][New offload] Support external host compiler #19505
Conversation
Use of -fsycl-host-compiler allows for users to use an external compiler to perform the host compilation when performing offload builds. The new model has the expectation for the device 'object' to be embedded within the host object via the `clang-offload-packager`. These fat objects are consumed by the `clang-linker-wrapper`. External host compilers do not have the ability to embed the device binary when creating the host object. Fortunately, the `clang-linker-wrapper` allows for consumption of bundled fat objects. Update the driver when using -fsycl-host-compiler and the new offloading model to generate bundled files instead of embedded objects. This support is for object generation only at this time. Preprocessing and assembly file intermediates are not yet supported.
| @@ -7758,9 +7758,17 @@ Action *Driver::BuildOffloadingActions(Compilation &C, | |||
| return HostAction; | |||
| } | |||
|
|
|||
| // For SYCL offloading with -fsycl-host-compiler enabled, we do not have the | |||
| // ability to embed the packaged file. | |||
There was a problem hiding this comment.
Can you share info on why we do not have the ability to embed the packaged file?
There was a problem hiding this comment.
The clang based compiler has the ability to embed external binaries via the -fembed-offload-object option. This occurs when generating the host object. When we are using a 3rd party compiler to create the host object, there is no way to embed the device object into it during compile time, so we are using bundles instead (i.e. the old model fat object)
| if (!(isa<CompileJobAction>(HostAction) || | ||
| // preprocessing only ignore embedding. When needing to do bundling for | ||
| // SYCL, allow the building of offloading actions to add the device side to | ||
| // the bundle. |
There was a problem hiding this comment.
When needing to do bundling for
// SYCL, allow the building of offloading actions to add the device side to
// the bundle.
Can you elaborate what this means?
There was a problem hiding this comment.
The device side actions are created below this. Typical compilations only need to go through this section when we know the device compilations are going to be packaged together. When using -fsycl-host-compiler, we need for these actions to occur beyond the IR stage of the host compilation, allowing the bundle call to contain the host object and the generated device binary.
| // Tests the abilities involved with using an external host compiler | ||
| // with the new offload model. | ||
|
|
||
| /// Enabling with -fsycl-host-compiler |
There was a problem hiding this comment.
This comment is not descriptive. What are we enabling here?
There was a problem hiding this comment.
Just enabling the compilation flow with -fsycl-host-compiler and testing the fundamental tool calls when the option is used.
Use of -fsycl-host-compiler allows for users to use an external compiler to perform the host compilation when performing offload builds. The new model has the expectation for the device 'object' to be embedded within the host object via the
clang-offload-packager. These fat objects are consumed by theclang-linker-wrapper.External host compilers do not have the ability to embed the device binary when creating the host object. Fortunately, the
clang-linker-wrapperallows for consumption of bundled fat objects.Update the driver when using -fsycl-host-compiler and the new offloading model to generate bundled files instead of embedded objects. This support is for object generation only at this time. Preprocessing and assembly file intermediates are not yet supported.