Still can't compile SYCL branch on NixOS

[izelnakri]

Hi again @sarnex I've put the config you've asked me to do:

{
  description = "Development environment for Intel DPC++ Compiler";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        llvm = pkgs.llvmPackages_21;
        buildPackages = with pkgs; [
          cmake
          ninja
          python3
          git
          spirv-tools
        ];
      in
      {
        devShells.default = llvm.libcxxStdenv.mkDerivation {
          name = "clang-shell";
          dontUnpack = true;
          dontBuild = true;
          dontInstall = true;

          nativeBuildInputs = buildPackages;

          shellHook = ''
            export LIBCXX_INCLUDE_DIR="${llvm.libcxx.dev}/include/c++/v1"
            export CXXFLAGS="-stdlib=libc++ -isystem $LIBCXX_INCLUDE_DIR -isystem ${llvm.libcxxStdenv.cc.libc.dev}/include"

            echo "To configure the project, run:"
            echo "----------------------------------------------------------------------------------"
            echo 'python buildbot/configure.py --use-libcxx \'
            echo '  --cmake-opt="-DCMAKE_C_COMPILER=$CC" \'
            echo '  --cmake-opt="-DCMAKE_CXX_COMPILER=$CXX" \'
            echo '  --cmake-opt="-DCMAKE_CXX_FLAGS=$CXXFLAGS" \'
            echo '  --cmake-opt="-DSYCL_LIBDEVICE_CXX_FLAGS=$CXXFLAGS" \'
            echo "----------------------------------------------------------------------------------"
            echo ""
            echo "Then, run:"
            echo 'python buildbot/compile.py'

            echo "🧠 Using nix dev environment with clang"
          '';
        };
      }

    );
}

and I still get this error:

[3756/3936] Generating ../../lib/imf-fp32-host.o
FAILED: lib/imf-fp32-host.o /home/izelnakri/Github/llvm/build/lib/imf-fp32-host.o
cd /home/izelnakri/Github/llvm/build/tools/libdevice && /home/izelnakri/Github/llvm/build/bin/clang-21 -c --target=x86_64-unknown-linux-gnu -D__LIBDEVICE_HOST_IMPL__ -stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -fPIC -fcf-protection=full /home/izelnakri/Github/llvm/libdevice/imf_wrapper.cpp -o /home/izelnakri/Github/llvm/build/./lib/imf-fp32-host.o
In file included from /home/izelnakri/Github/llvm/libdevice/imf_wrapper.cpp:13:
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:54:32: error: use of undeclared identifier 'fegetround'
   54 |   const int roundingOriginal = fegetround();
      |                                ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:55:3: error: use of undeclared identifier 'fesetround'
   55 |   fesetround(rdMode);
      |   ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:61:3: error: use of undeclared identifier 'fesetround'
   61 |   fesetround(roundingOriginal);
      |   ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:72:32: error: use of undeclared identifier 'fegetround'
   72 |   const int roundingOriginal = fegetround();
      |                                ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:73:3: error: use of undeclared identifier 'fesetround'
   73 |   fesetround(rdMode);
      |   ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:75:3: error: use of undeclared identifier 'fesetround'
   75 |   fesetround(roundingOriginal);
      |   ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:90:32: error: use of undeclared identifier 'fegetround'
   90 |   const int roundingOriginal = fegetround();
      |                                ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:91:3: error: use of undeclared identifier 'fesetround'
   91 |   fesetround(rdMode);
      |   ^~~~~~~~~~
/home/izelnakri/Github/llvm/libdevice/device_imf.hpp:93:3: error: use of undeclared identifier 'fesetround'
   93 |   fesetround(roundingOriginal);
      |   ^~~~~~~~~~
9 errors generated.
[3763/3936] Building CXX object tools/sycl/...keFiles/syclbin-dump.dir/syclbin-dump.cpp.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/izelnakri/Github/llvm/buildbot/compile.py", line 99, in <module>
    ret = main()
  File "/home/izelnakri/Github/llvm/buildbot/compile.py", line 95, in main
    return do_compile(args)
  File "/home/izelnakri/Github/llvm/buildbot/compile.py", line 45, in do_compile
    subprocess.check_call(cmake_cmd, cwd=abs_obj_dir)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/lib/python3.13/subprocess.py", line 419, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '/home/izelnakri/Github/llvm/build', '--', 'deploy-sycl-toolchain', '-j', '8']' returned non-zero exit status 1.

I've put the DSYCL_LIBDEVICE_CXX_FLAGS and correct LIBCXX_INCLUDE_DIR and CXXFLAGS this time as you suggested, seems you didnt reproduce the issue as I mentioned on NixOS. I appreciate that you were willing to help so far but lets run on less assumptions and get it fixed this time after 3 weeks.

Currently the Intel build system assumes:

• People have a certain version of a clang or gcc compiler(so warnings dont happen like it happens on my computer), since LLVM doesnt use Nix, it is hard to know which version of the compiler that is.
• It was good to point out Nix patches/flags nix adds but as we've seen it is not applicable in this scenario.
• Currently we dont have incremental builds, which is a problem because feedback loop is too long, it takes like more than 30mins to compile or get compilation errors.

Previously, I've done some cmake custom(bad) fixes to fix this compilation error, then received glibc global variables reference compilation errorr, where this doesnt get triggered:

#if defined(__LIBDEVICE_HOST_IMPL__)
#include <cfenv>
#pragma STDC FENV_ACCESS ON

so global errors happen I suppose.

My compiler flags:

22:57:06 izelnakri ~/Github/llvm ❄ -> master nix develop
warning: Git tree '/home/izelnakri/Github/llvm' is dirty
To configure the project, run:
----------------------------------------------------------------------------------
python buildbot/configure.py --use-libcxx \
  --cmake-opt="-DCMAKE_C_COMPILER=$CC" \
  --cmake-opt="-DCMAKE_CXX_COMPILER=$CXX" \
  --cmake-opt="-DCMAKE_CXX_FLAGS=$CXXFLAGS" \
  --cmake-opt="-DSYCL_LIBDEVICE_CXX_FLAGS=$CXXFLAGS" \
----------------------------------------------------------------------------------

Then, run:
python buildbot/compile.py
🧠 Using nix dev environment with clang

[izelnakri@omnibook:~/Github/llvm]$ NIX_DEBUG=1 clang++ setup.cpp
HARDENING: disabled flags: pacret pie fortify3 shadowstack strictflexarrays3 strictflexarrays1 nostrictaliasing trivialautovarinit
HARDENING: Is active (not completely disabled with "all" flag)
HARDENING: enabling pic
HARDENING: enabling stack-clash-protection
HARDENING: enabling format
HARDENING: enabling zerocallusedregs
HARDENING: enabling stackprotector
HARDENING: enabling fortify
HARDENING: enabling strictoverflow
extra flags before to /nix/store/xk5y113wfqljr4y8wpsay7f5jj4pyvl7-clang-21.1.0-rc1/bin/clang++:
  -fPIC
  -fstack-clash-protection
  -Wformat
  -Wformat-security
  -Werror=format-security
  -fzero-call-used-regs=used-gpr
  -fstack-protector-strong
  --param
  ssp-buffer-size=4
  -O2
  -U_FORTIFY_SOURCE
  -fwrapv
  -fno-omit-frame-pointer
  -mno-omit-leaf-frame-pointer
  -Wl\,-dynamic-linker=/nix/store/g2jzxk3s7cnkhh8yq55l4fbvf639zy37-glibc-2.40-66/lib/ld-linux-x86-64.so.2
  -target
  x86_64-unknown-linux-gnu
original flags to /nix/store/xk5y113wfqljr4y8wpsay7f5jj4pyvl7-clang-21.1.0-rc1/bin/clang++:
  setup.cpp
extra flags after to /nix/store/xk5y113wfqljr4y8wpsay7f5jj4pyvl7-clang-21.1.0-rc1/bin/clang++:
  -U_FORTIFY_SOURCE
  -D_FORTIFY_SOURCE=2
  -B/nix/store/g2jzxk3s7cnkhh8yq55l4fbvf639zy37-glibc-2.40-66/lib/
  -idirafter
  /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include
  --gcc-toolchain=/nix/store/xvqbvva4djgyascjsnki6354422n4msk-gcc-14.3.0
  -B/nix/store/mchp1grs7zh2acgd0sp8f6ni9fkcdsnk-clang-21.1.0-rc1-lib/lib
  -nostdlibinc
  -resource-dir=/nix/store/3s0ab22v0b4n9cjcfbaxaha82y475xf1-clang-wrapper-21.1.0-rc1/resource-root
  -frandom-seed=smy6yk42hd
  -isystem
  /nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/include
  -isystem
  /nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/include
  -isystem
  /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include
  -isystem
  /nix/store/x7jlpliiypfwl89vc85hgzcixcpfabda-compiler-rt-libc-21.1.0-rc1-dev/include
  -isystem
  /nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/include
  -isystem
  /nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/include
  -isystem
  /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include
  -isystem
  /nix/store/x7jlpliiypfwl89vc85hgzcixcpfabda-compiler-rt-libc-21.1.0-rc1-dev/include
  -isystem
  /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1
  -L/usr/lib
  -L/usr/lib32
  -stdlib=libc++
  -Wl\,-rpath
  -Wl\,/home/izelnakri/Github/llvm/outputs/out/lib
  -L/nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/lib
  -L/nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/lib
  -L/nix/store/k1p4s0ddj18nxzss6c52s12n6zdb47cq-libcxx-21.1.0-rc1/lib
  -L/nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/lib
  -L/nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/lib
  -L/nix/store/k1p4s0ddj18nxzss6c52s12n6zdb47cq-libcxx-21.1.0-rc1/lib
  -L/nix/store/g2jzxk3s7cnkhh8yq55l4fbvf639zy37-glibc-2.40-66/lib
  -L/nix/store/xvqbvva4djgyascjsnki6354422n4msk-gcc-14.3.0//lib
  -L/nix/store/fxwmabprl1nyxskbrygqlaps5kwg3fjv-gcc-14.3.0-libgcc/lib
  -L/nix/store/mchp1grs7zh2acgd0sp8f6ni9fkcdsnk-clang-21.1.0-rc1-lib/lib
clang++: error: no such file or directory: 'setup.cpp'

Also mentioning @blenderfreaky he managed to build this in nix build sandbox environment but had other issues.

configure flag I get from configure.py:

[izelnakri@omnibook:~/Github/llvm]$ python buildbot/configure.py --use-libcxx \
  --cmake-opt="-DCMAKE_C_COMPILER=$CC" \
  --cmake-opt="-DCMAKE_CXX_COMPILER=$CXX" \
  --cmake-opt="-DCMAKE_CXX_FLAGS=$CXXFLAGS" \
  --cmake-opt="-DSYCL_LIBDEVICE_CXX_FLAGS=$CXXFLAGS"
args:Namespace(build_number=None, branch=None, base_branch=None, pr_number=None, builder_dir=None, src_dir=None, obj_dir=None, l0_headers=None, l0_loader=None, build_type='Release', cuda=False, native_cpu=False, hip=False, hip_platform='AMD', level_zero_adapter_version='ALL', host_target='host', enable_all_llvm_targets=False, no_assertions=False, docs=False, werror=False, shared_libs=False, cmake_opt=['-DCMAKE_C_COMPILER=clang', '-DCMAKE_CXX_COMPILER=clang++', '-DCMAKE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include', '-DSYCL_LIBDEVICE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include'], cmake_gen='Ninja', use_libcxx=True, use_lld=False, llvm_external_projects=None, ci_defaults=False, enable_backends=None, disable_preview_lib=False, disable_jit=False, add_security_flags=None, native_cpu_libclc_targets=None, use_zstd=False)
[Cmake Command]: cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=host '-DLLVM_EXTERNAL_PROJECTS=sycl;llvm-spirv;opencl;xpti;xptifw;libdevice;sycl-jit' -DLLVM_EXTERNAL_SYCL_SOURCE_DIR=/home/izelnakri/Github/llvm/sycl -DLLVM_EXTERNAL_LLVM_SPIRV_SOURCE_DIR=/home/izelnakri/Github/llvm/llvm-spirv -DLLVM_EXTERNAL_XPTI_SOURCE_DIR=/home/izelnakri/Github/llvm/xpti -DXPTI_SOURCE_DIR=/home/izelnakri/Github/llvm/xpti -DLLVM_EXTERNAL_XPTIFW_SOURCE_DIR=/home/izelnakri/Github/llvm/xptifw -DLLVM_EXTERNAL_LIBDEVICE_SOURCE_DIR=/home/izelnakri/Github/llvm/libdevice -DLLVM_EXTERNAL_SYCL_JIT_SOURCE_DIR=/home/izelnakri/Github/llvm/sycl-jit '-DLLVM_ENABLE_PROJECTS=clang;sycl;llvm-spirv;opencl;xpti;xptifw;libdevice;sycl-jit' -DSYCL_BUILD_PI_HIP_PLATFORM=AMD -DLLVM_BUILD_TOOLS=ON -DLLVM_ENABLE_ZSTD=OFF -DLLVM_USE_STATIC_ZSTD=ON -DSYCL_ENABLE_WERROR=OFF -DCMAKE_INSTALL_PREFIX=/home/izelnakri/Github/llvm/build/install -DSYCL_INCLUDE_TESTS=ON -DLLVM_ENABLE_DOXYGEN=OFF -DLLVM_ENABLE_SPHINX=OFF -DBUILD_SHARED_LIBS=OFF -DSYCL_ENABLE_XPTI_TRACING=ON -DLLVM_ENABLE_LLD=OFF -DLLVM_SPIRV_ENABLE_LIBSPIRV_DIS=OFF -DXPTI_ENABLE_WERROR=OFF -DSYCL_CLANG_EXTRA_FLAGS= '-DSYCL_ENABLE_BACKENDS=opencl;level_zero_v2;level_zero' -DSYCL_ENABLE_EXTENSION_JIT=ON -DSYCL_ENABLE_MAJOR_RELEASE_PREVIEW_LIB=ON -DBUG_REPORT_URL=https://github.com/intel/llvm/issues -DSYCL_INSTALL_DEVICE_CONFIG_FILE=OFF -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ '-DCMAKE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include' '-DSYCL_LIBDEVICE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include' /home/izelnakri/Github/llvm/llvm -DLLVM_ENABLE_LIBCXX=ON -DLLVM_LIBCXX_USED=ON

[sarnex]

I was able to successfully compile to project once I passed -DSYCL_JIT_ENABLE_WERROR=OFF to configure.py using the flake.nix in this issue. I used nix develop, hopefully that's the right one.

If it's not working for you then either the Docker image I'm using isn't an accurate representation of the NixOS environment or there is some other issue in your environment that is not present on a fresh install.

Here are the entire steps of everything I did:

docker run -it nixos/nix bash
nix-env -iA nixpkgs.nano nixpkgs.ncurses
cd /tmp
nano flake.nix
(pasted the above flake.nix)
nix --extra-experimental-features nix-command --extra-experimental-features flakes develop
git clone  https://github.com/intel/llvm
cd llvm
python buildbot/configure.py --use-libcxx \
  --cmake-opt="-DCMAKE_C_COMPILER=$CC" \
  --cmake-opt="-DCMAKE_CXX_COMPILER=$CXX" \
  --cmake-opt="-DCMAKE_CXX_FLAGS=$CXXFLAGS" \
  --cmake-opt="-DSYCL_LIBDEVICE_CXX_FLAGS=$CXXFLAGS" -DSYCL_JIT_ENABLE_WERROR=OFF
python buildbot/compile.py

[blenderfreaky]

@izelnakri I'm surprised your build fails. I think that should run through. Could you try using nix develop -i to clear the environment or run in a clean/fresh shell? I'm running two LLVM compilations right now, so I can't repro the exact steps atm, but what @sarnex did looks correct, and I'd expect you to get the same result.

Btw, nix develop is the right command here. Note that the docker environment is not a 1:1 representation, as sandboxing is off by default. But that's only relevant when building derivations (i.e. running nix build) and it shouldn't affect the stuff you ran.

Is there a particular reason why you're trying to use clang & libc++ in the first place, instead of gcc & libstdc++?

I was able to get my derivation to build fully. There's some dynamic linking errors when running the final binaries that I'm currently looking into, but the build runs through fully now. For reference, here's my current derivation

Just to keep track of all related threads: #19347 (previous issue) #19635 (dependency vendoring) #19636 (configure script PR) #19637 (minor purity fix PR)

[izelnakri]

If it's not working for you then either the Docker image I'm using isn't an accurate representation of the NixOS environment

Yes, the docker image is not NixOS since it misses key commands like nixos-rebuild etc.

I added -DSYCL_JIT_ENABLE_WERROR=OFF as well, and now I get this error on:

python buildbot/configure.py --use-libcxx \
  --cmake-opt="-DCMAKE_C_COMPILER=$CC" \
  --cmake-opt="-DCMAKE_CXX_COMPILER=$CXX" \
  --cmake-opt="-DCMAKE_CXX_FLAGS=$CXXFLAGS" \
  --cmake-opt="-DSYCL_LIBDEVICE_CXX_FLAGS=$CXXFLAGS -DSYCL_JIT_ENABLE_WERROR=OFF"

[3751/3936] Generating ../../lib/imf-bf16-host.o
FAILED: lib/imf-bf16-host.o /home/izelnakri/Github/llvm/build/lib/imf-bf16-host.o
cd /home/izelnakri/Github/llvm/build/tools/libdevice && /home/izelnakri/Github/llvm/build/bin/clang-21 -c --target=x86_64-unknown-linux-gnu -D__LIBDEVICE_HOST_IMPL__ -stdlib=libc++ -fPIC -fcf-protection=full /home/izelnakri/Github/llvm/libdevice/imf_wrapper_bf16.cpp -o /home/izelnakri/Github/llvm/build/./lib/imf-bf16-host.o
In file included from /home/izelnakri/Github/llvm/libdevice/imf_wrapper_bf16.cpp:14:
In file included from /home/izelnakri/Github/llvm/libdevice/imf_bf16.hpp:13:
/home/izelnakri/Github/llvm/libdevice/imf_impl_utils.hpp:12:10: fatal error: 'cstddef' file not found
   12 | #include <cstddef>
      |          ^~~~~~~~~
1 error generated.
[3758/3936] Building CXX object tools/sycl/tools/syclbin-dump/CMakeFiles/syclbin-dump.dir/syclbin-dump.cpp.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/izelnakri/Github/llvm/buildbot/compile.py", line 99, in <module>
    ret = main()
  File "/home/izelnakri/Github/llvm/buildbot/compile.py", line 95, in main
    return do_compile(args)
  File "/home/izelnakri/Github/llvm/buildbot/compile.py", line 45, in do_compile
    subprocess.check_call(cmake_cmd, cwd=abs_obj_dir)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/lib/python3.13/subprocess.py", line 419, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '/home/izelnakri/Github/llvm/build', '--', 'deploy-sycl-toolchain', '-j', '8']' returned non-zero exit status 1.

I could perhaps try again with adding --disable-jit and do the nix develop -i but I highly doubt they will work, given my flake.nix is already very specific and I tried with -DSYCL_JIT_ENABLE_WERROR=OFF now. Here is my compiler flags:

[izelnakri@omnibook:~/Github/llvm]$ python buildbot/configure.py --use-libcxx \
  --cmake-opt="-DCMAKE_C_COMPILER=$CC" \
  --cmake-opt="-DCMAKE_CXX_COMPILER=$CXX" \
  --cmake-opt="-DCMAKE_CXX_FLAGS=$CXXFLAGS" \
  --cmake-opt="-DSYCL_LIBDEVICE_CXX_FLAGS=$CXXFLAGS -DSYCL_JIT_ENABLE_WERROR=OFF"
args:Namespace(build_number=None, branch=None, base_branch=None, pr_number=None, builder_dir=None, src_dir=None, obj_dir=None, l0_headers=None, l0_loader=None, build_type='Release', cuda=False, native_cpu=False, hip=False, hip_platform='AMD', level_zero_adapter_version='ALL', host_target='host', enable_all_llvm_targets=False, no_assertions=False, docs=False, werror=False, shared_libs=False, cmake_opt=['-DCMAKE_C_COMPILER=clang', '-DCMAKE_CXX_COMPILER=clang++', '-DCMAKE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include', '-DSYCL_LIBDEVICE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include -DSYCL_JIT_ENABLE_WERROR=OFF'], cmake_gen='Ninja', use_libcxx=True, use_lld=False, llvm_external_projects=None, ci_defaults=False, enable_backends=None, disable_preview_lib=False, disable_jit=False, add_security_flags=None, native_cpu_libclc_targets=None, use_zstd=False)
[Cmake Command]: cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_TARGETS_TO_BUILD=host '-DLLVM_EXTERNAL_PROJECTS=sycl;llvm-spirv;opencl;xpti;xptifw;libdevice;sycl-jit' -DLLVM_EXTERNAL_SYCL_SOURCE_DIR=/home/izelnakri/Github/llvm/sycl -DLLVM_EXTERNAL_LLVM_SPIRV_SOURCE_DIR=/home/izelnakri/Github/llvm/llvm-spirv -DLLVM_EXTERNAL_XPTI_SOURCE_DIR=/home/izelnakri/Github/llvm/xpti -DXPTI_SOURCE_DIR=/home/izelnakri/Github/llvm/xpti -DLLVM_EXTERNAL_XPTIFW_SOURCE_DIR=/home/izelnakri/Github/llvm/xptifw -DLLVM_EXTERNAL_LIBDEVICE_SOURCE_DIR=/home/izelnakri/Github/llvm/libdevice -DLLVM_EXTERNAL_SYCL_JIT_SOURCE_DIR=/home/izelnakri/Github/llvm/sycl-jit '-DLLVM_ENABLE_PROJECTS=clang;sycl;llvm-spirv;opencl;xpti;xptifw;libdevice;sycl-jit' -DSYCL_BUILD_PI_HIP_PLATFORM=AMD -DLLVM_BUILD_TOOLS=ON -DLLVM_ENABLE_ZSTD=OFF -DLLVM_USE_STATIC_ZSTD=ON -DSYCL_ENABLE_WERROR=OFF -DCMAKE_INSTALL_PREFIX=/home/izelnakri/Github/llvm/build/install -DSYCL_INCLUDE_TESTS=ON -DLLVM_ENABLE_DOXYGEN=OFF -DLLVM_ENABLE_SPHINX=OFF -DBUILD_SHARED_LIBS=OFF -DSYCL_ENABLE_XPTI_TRACING=ON -DLLVM_ENABLE_LLD=OFF -DLLVM_SPIRV_ENABLE_LIBSPIRV_DIS=OFF -DXPTI_ENABLE_WERROR=OFF -DSYCL_CLANG_EXTRA_FLAGS= '-DSYCL_ENABLE_BACKENDS=opencl;level_zero_v2;level_zero' -DSYCL_ENABLE_EXTENSION_JIT=ON -DSYCL_ENABLE_MAJOR_RELEASE_PREVIEW_LIB=ON -DBUG_REPORT_URL=https://github.com/intel/llvm/issues -DSYCL_INSTALL_DEVICE_CONFIG_FILE=OFF -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ '-DCMAKE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include' '-DSYCL_LIBDEVICE_CXX_FLAGS=-stdlib=libc++ -isystem /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1 -isystem /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include -DSYCL_JIT_ENABLE_WERROR=OFF' /home/izelnakri/Github/llvm/llvm -DLLVM_ENABLE_LIBCXX=ON -DLLVM_LIBCXX_USED=ON

Best way to test NixOS packages is to run NixOS VM, this is also what nixpkgs tests do for each package when package maintainers add tests. NixOS tests are amazing, they can run/coordinate multiple VMs(that each defined as key for the machine name and value is a minimal configuration.nix), then testScript key defines a python snippet to coordinate these VMs(waiting for certain systemd services to start or executing certain commands with failure/success as function). Best is to run the nixos image with qemu manually of course, which is also what pkgs.nixosTest does internally.

Lastly, my flags inside my nix develop shell:

[izelnakri@omnibook:~/Github/llvm]$ NIX_DEBUG=1 clang++ test.cpp
HARDENING: disabled flags: pacret pie fortify3 shadowstack strictflexarrays3 strictflexarrays1 nostrictaliasing trivialautovarinit
HARDENING: Is active (not completely disabled with "all" flag)
HARDENING: enabling pic
HARDENING: enabling stack-clash-protection
HARDENING: enabling format
HARDENING: enabling zerocallusedregs
HARDENING: enabling stackprotector
HARDENING: enabling fortify
HARDENING: enabling strictoverflow
extra flags before to /nix/store/xk5y113wfqljr4y8wpsay7f5jj4pyvl7-clang-21.1.0-rc1/bin/clang++:
  -fPIC
  -fstack-clash-protection
  -Wformat
  -Wformat-security
  -Werror=format-security
  -fzero-call-used-regs=used-gpr
  -fstack-protector-strong
  --param
  ssp-buffer-size=4
  -O2
  -U_FORTIFY_SOURCE
  -fwrapv
  -fno-omit-frame-pointer
  -mno-omit-leaf-frame-pointer
  -Wl\,-dynamic-linker=/nix/store/g2jzxk3s7cnkhh8yq55l4fbvf639zy37-glibc-2.40-66/lib/ld-linux-x86-64.so.2
  -target
  x86_64-unknown-linux-gnu
original flags to /nix/store/xk5y113wfqljr4y8wpsay7f5jj4pyvl7-clang-21.1.0-rc1/bin/clang++:
  test.cpp
extra flags after to /nix/store/xk5y113wfqljr4y8wpsay7f5jj4pyvl7-clang-21.1.0-rc1/bin/clang++:
  -U_FORTIFY_SOURCE
  -D_FORTIFY_SOURCE=2
  -B/nix/store/g2jzxk3s7cnkhh8yq55l4fbvf639zy37-glibc-2.40-66/lib/
  -idirafter
  /nix/store/vm10zh43xgfxvgrqs8brz6v6xyrq9qin-glibc-2.40-66-dev/include
  --gcc-toolchain=/nix/store/xvqbvva4djgyascjsnki6354422n4msk-gcc-14.3.0
  -B/nix/store/mchp1grs7zh2acgd0sp8f6ni9fkcdsnk-clang-21.1.0-rc1-lib/lib
  -nostdlibinc
  -resource-dir=/nix/store/3s0ab22v0b4n9cjcfbaxaha82y475xf1-clang-wrapper-21.1.0-rc1/resource-root
  -frandom-seed=vy0cqhbyrg
  -isystem
  /nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/include
  -isystem
  /nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/include
  -isystem
  /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include
  -isystem
  /nix/store/x7jlpliiypfwl89vc85hgzcixcpfabda-compiler-rt-libc-21.1.0-rc1-dev/include
  -isystem
  /nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/include
  -isystem
  /nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/include
  -isystem
  /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include
  -isystem
  /nix/store/x7jlpliiypfwl89vc85hgzcixcpfabda-compiler-rt-libc-21.1.0-rc1-dev/include
  -isystem
  /nix/store/i705cg2xkr48j1dzr0mh2p6yfx9w4k8v-libcxx-21.1.0-rc1-dev/include/c++/v1
  -stdlib=libc++
  -Wl\,-rpath
  -Wl\,/home/izelnakri/Github/llvm/outputs/out/lib
  -L/nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/lib
  -L/nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/lib
  -L/nix/store/k1p4s0ddj18nxzss6c52s12n6zdb47cq-libcxx-21.1.0-rc1/lib
  -L/nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/lib
  -L/nix/store/4g8s7rhsjzdlpjq7cywv2nf5yyc1v1dm-spirv-tools-1.4.313.0/lib
  -L/nix/store/k1p4s0ddj18nxzss6c52s12n6zdb47cq-libcxx-21.1.0-rc1/lib
  -L/nix/store/g2jzxk3s7cnkhh8yq55l4fbvf639zy37-glibc-2.40-66/lib
  -L/nix/store/xvqbvva4djgyascjsnki6354422n4msk-gcc-14.3.0//lib
  -L/nix/store/fxwmabprl1nyxskbrygqlaps5kwg3fjv-gcc-14.3.0-libgcc/lib
  -L/nix/store/mchp1grs7zh2acgd0sp8f6ni9fkcdsnk-clang-21.1.0-rc1-lib/lib
clang++: error: no such file or directory: 'test.cpp'

[sarnex]

Since it is working on NixOS for @blenderfreaky and myself, it sounds like this is a NixOS environment issue that me/other developers working in this repo aren't really qualified to investigate or solve. Maybe you guys could figure it out together or post in a NixOS community area?

[izelnakri]

I was missing the --disable-jit, it worked this time! We'll create a PR to flake.nix for this project most likely so people can even download/run the relevant branch with:

nix run github:intel/llvm

or:

nix run github:intel/llvm test.cpp

apart from our PR to nixpkgs for nix profile add nixpkgs#dpcpp or icx or icpx naming(we need to decide).

I have this build/bins:

[izelnakri@omnibook:~/Github/llvm/build/bin]$ ls
append-file  clang-cl               clang-offload-deps      clang-tblgen      llvm-foreach     llvm-objcopy  spirv-to-ir-wrapper  sycl-prof
clang        clang-cpp              clang-offload-extract   file-table-tform  llvm-link        llvm-spirv    syclbin-dump         sycl-sanitize
clang++      clang-linker-wrapper   clang-offload-packager  llc               llvm-lit         llvm-tblgen   sycl-ls              sycl-trace
clang-21     clang-offload-bundler  clang-offload-wrapper   llvm-ar           llvm-min-tblgen  opencl-aot    sycl-post-link

Are clang refers to icx and clang++ refers to icpx @sarnex ?

[sarnex]

Great! Instead of --disable-jit you should be able to use -DSYCL_JIT_ENABLE_WERROR=OFF too.

It's a bit confusing but icx and icpx is actually not for the compiler in this repo. We have an internal fork of this repo with extra changes that composes the official oneAPI release. That closed source compiler is what provides the icx and icpx binaries.

In this repo we only provide clang and clang++ and as far as I know we don't have any official guidance on what executable names to use specifically for this repo. In the past some have used dpcpp/dpc++.

[sarnex]

Closing as issue seems resolved, thanks!

[izelnakri]

So does ./build/bin/clang and ./build/bin/clang++ still serve the purpose of icx or icpx or are they missing very important optimizations? Would I be able to get HW acceleration on my GPU and maybe even NPU if I compile an inferencing program with or without openvino? @sarnex

[sarnex]

There are missing features and optimizations compared to the closed source compiler, if they are important or not depends on your use case. For example, the closed source compiler has support for OpenMP offloading to Intel GPUs while this repo does not.

You will be able to run SYCL code on your CPU or GPU with this repo. For an NPU you need openvino but I believe this repo can be used to satisfy the openvino compiler depenedency.

[izelnakri]

Great, the reason Im asking is we will package and distribute this compiler on nixpkgs eventually and all other inferencing software will depend on it (like ollama, llama.cpp etc). When ollama, llama.cpp compiled with withOneAPI = true option in nix configuration.nix or home.nix(for non-NixOS systems), something like that ;) So is it fine as it is for now? and more in the plans to introduce more of these features/optimizations to this repository?

[sarnex]

i dont think we have any announcements to make in that area, i wouldn't suggest waiting for a feature you hope will be added to this repo in the future, i recommend working with what's here today