Cannot run task after poetry install

[anoadragon453]

Describe the bug
I'm attempting to run a small script after poetry install completes in my repo. poetry install downloads some binaries that are dynamically linked against ld, and thus do not work on NixOS. I'm trying to automatically run patchelf after poetry install completes in order to patch ruff, thus allowing NixOS developers to not have to worry about doing it themselves or manually running a command after each poetry install.

Here is my devenv module:

{
  # Configure packages to install.
  # Search for package names at https://search.nixos.org/packages?channel=unstable
  packages = with pkgs; [
    openssl

    patchelf
    autoPatchelfHook
  ];

  # Install Python and manage a virtualenv with Poetry.
  languages.python.enable = true;
  languages.python.poetry.enable = true;
  # Automatically activate the poetry virtualenv upon entering the shell.
  languages.python.poetry.activate.enable = true;
  languages.python.poetry.install.installRootPackage = true;
  languages.python.poetry.install.enable = true;
  
  tasks = {
    "sygnal:patchpythonbinaries" = {
      description = "Patch binaries dynamically linked to ld, which will not work on NixOS";
      exec = ''
        patchelf --remove-rpath .venv/bin/ruff
        autoPatchelf .venv/bin/ruff
      '';
      after = [ "devenv:enterShell" ];
    };
  };
}

I first tried to put the above patchelf; autoPatchelf invocation in enterShell, but that failed as poetry install runs after whatever a devenv module defines in enterShell. Meaning patchelf would be called, and then ruff would be downloaded by poetry install.

Interestingly, poetry install is defined as a task that apparently runs after enterShell:

devenv/src/modules/languages/python.nix

Lines 457 to 486 in fa53082

	tasks = {
	"devenv:python:virtualenv" = lib.mkIf cfg.venv.enable {
	description = "Initialize Python virtual environment";
	exec = initVenvScript;
	exports = [ "PATH" "VIRTUAL_ENV" ];
	};
	"devenv:python:poetry" = lib.mkIf cfg.poetry.install.enable {
	description = "Initialize Poetry";
	exec = initPoetryScript;
	exports = [ "PATH" ] ++ lib.optional cfg.poetry.activate.enable "VIRTUAL_ENV";
	after = lib.optional cfg.venv.enable "devenv:python:virtualenv";
	};
	"devenv:python:uv" = lib.mkIf cfg.uv.sync.enable {
	description = "Initialize uv sync";
	exec = initUvScript;
	exports = [ "PATH" ];
	after = lib.optional cfg.venv.enable "devenv:python:virtualenv";
	};
	"devenv:enterShell".after = lib.optional cfg.venv.enable "devenv:python:virtualenv"
	++ lib.optional cfg.poetry.install.enable "devenv:python:poetry"
	++ lib.optional cfg.uv.sync.enable "devenv:python:uv";
	};
	enterShell = ''
	export PYTHONPATH="$DEVENV_PROFILE/${package.sitePackages}''${PYTHONPATH:+:$PYTHONPATH}"
	'';
	};

I attempted all sorts of combinations of trying to get my task to run after both "devenv:enterShell" and "devenv:python:poetry", but to no avail. If I do:

      before = [ "devenv:enterShell" ];

then my task runs, but does so before the poetry install, so fails as ruff does not exist:

direnv: loading ~/code/sygnal/.envrc
direnv: using flake path:///home/work/code/nix-flakes/project-flakes/sygnal --impure
[0/1 built, 0.0 MiB DL] querying devenv-shell-env on https://devenv.cachix.orgdirenv: ([/nix/store/3mydh7746lji25ry2aygsy5i4s0i23x2-direnv-2.35.0/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
direnv: nix-direnv: Renewed cache
Running tasks     devenv:enterShell
Failed            sygnal:patchpythonbinaries 3ms
Succeeded         devenv:python:poetry       2582ms
Dependency failed devenv:enterShell          
1 Succeeded, 1 Failed, 1 Dependency Failed   2.59s

--- sygnal:patchpythonbinaries failed with error: Task exited with status: exit status: 1
--- sygnal:patchpythonbinaries stdout:
--- sygnal:patchpythonbinaries stderr:
0002.59: patchelf: getting info about '.venv/bin/ruff': No such file or directory

however, if I do the logical thing and:

      after = [ "devenv:enterShell" ];

my task doesn't run at all:

direnv: loading ~/code/sygnal/.envrc
direnv: using flake path:///home/work/code/nix-flakes/project-flakes/sygnal --impure
direnv: ([/nix/store/3mydh7746lji25ry2aygsy5i4s0i23x2-direnv-2.35.0/bin/direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
direnv: nix-direnv: Renewed cache
Running tasks     devenv:enterShell
Succeeded         devenv:python:poetry       2675ms
Succeeded         devenv:enterShell          7ms
2 Succeeded                                  2.68s

So I'm at a loss. This is using a flake, not devenv.nix/yaml/lock.

To reproduce

My devenv module can be found here, along with all other relevant files: https://github.com/element-hq/nix-flakes/blob/anoa/devenv_repro/project-flakes/sygnal/module.nix

Version

Using flakes, devenv v1.3.1.

[sandydoo]

I'm not familiar with the tasks implementation, but something tells me we have an issue in the graph traversal somewhere, ie. we're not picking up indirect dependencies of enterShell + the ordering appears reversed.

[sandydoo]

I think I've found the issue. We'll discuss it next week and have a fix out promptly.

[dmarcoux]

After a search on GitHub, I have found a way to make this work. Credits go to this commit. It's a matter of defining both before and after for the task. Personally, I don't use Poetry, but only rely on venv with a requirements.txt file. This is how I've patched ruff:

devenv.nix

  (...)

  languages.python = {
    enable = true;
    version = "3.12.7";
    venv = {
      enable = true;
      requirements = ./requirements.txt;
    };
  };

  tasks = {
    # Patch ruff to make it runnable
    "venv:patchelf" = {
      exec = "${lib.getExe pkgs.patchelf} --set-interpreter ${pkgs.stdenv.cc.bintools.dynamicLinker} $VIRTUAL_ENV/bin/ruff";
      after = [ "devenv:python:virtualenv" ]; # Runs after this
      before = [ "devenv:enterShell" ]; # Runs before this
    };
  };