haskellPackages: update stackage and hackage

[sternenseemann]

This Merge

This PR is the regular merge of the haskell-updates branch into master.

This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates. You may be able to find an up-to-date Hydra build report at cdepillabout/nix-haskell-updates-status.

We roughly aim to merge these haskell-updates PRs at least once every two weeks. See the @NixOS/haskell team calendar for who is currently in charge of this branch.

haskellPackages Workflow Summary

Our workflow is currently described in pkgs/development/haskell-modules/HACKING.md.

The short version is this:

• We regularly update the Stackage and Hackage pins on haskell-updates (normally at the beginning of a merge window).
• The community fixes builds of Haskell packages on that branch.
• We aim at at least one merge of haskell-updates into master every two weeks.
• We only do the merge if the mergeable job is succeeding on hydra.
• If a maintained package is still broken at the time of merge, we will only merge if the maintainer has been pinged 7 days in advance. (If you care about a Haskell package, become a maintainer!)

More information about Haskell packages in nixpkgs can be found in the nixpkgs manual.

---

This is the follow-up to #351154. Come to #haskell:nixos.org if you have any questions.

[K900]

I think we have to make a call here, and have haskell-updates target staging, not master. We have a variety of things that depend on haskell-adjacent packages (like pandoc) that are not part of the haskell-updates jobset and can't realistically be without it becoming a second staging, and every haskell-updates merge causes an unhealthy amount of rebuilds for master - e.g. #351154 rebuilds Qt, among other things.

[maralorn]

How many rebuilds are that?
I wasn't aware of this problem.

[vcunat]

OfBorg's estimate from that iteration was around 51k rebuilds total (all 4 platforms). The haskell-updates jobset prepared about 14k, which is partially because we deactivated its darwin builds some time ago.

[sternenseemann]

So it seems that most (if not all) of the rebuilds are caused by the use of shellcheck in writeShellApplication. I was aware of this, but perceived it as an API for external users, so I never realized it was used in nixpkgs/NixOS to this extent.

I guess we could do (some of) the following things:

• Clamp down on the use of writeShellApplication in nixpkgs. Executing shellcheck can be done in passthru.tests for nixpkgs, so I don't really see why it needs to be used in this way, it currently is. Even without the unfortunate situation with haskell-updates, this would reduce rebuild time for a lot of packages probably.
• Send haskell-updates to staging. This of course will significantly increase the time to deliver updates to packages. The 2 weeks is of course just a soft goal that's more a function of how maintenance duties have been organized. In practice, it depends on availability of maintainer and build time—sometimes it goes quicker, sometimes slower. However, downstream users are occasionally dissatisfied even with the speed we can deliver updates at the moment. A possible setup would be the following: haskell-updates is based on master so that changes can be tested with less disruptions and then merged into staging (or an early staging-next). We'd probably need to tie the automatic merges to the ones for staging, so that haskell-updates doesn't get ahead of staging. We'd also start being more liberal with merging low rebuild fixes into master.
• Extend the normal jobsets to include Haskell specific jobs and use them instead of the custom one, making haskell-updates a second staging. Probably infeasible as long as there are as many jobs rebuilt by Haskell changes.

[sternenseemann]

This iteration won't reach 24.11. We'll probably want to backport some stuff, in particular anything relating to HLS and GHC 9.8.3.

[sternenseemann]

Merging with the following outstanding issues:

• GHC 9.12 packages barely work. This is also an issue on master and we'll chip away at it in the future, but we don't need to block this PR on it. Will need a new release of e.g. jailbreak-cabal: Support for Cabal >= 3.14 jailbreak-cabal#25
• GHC 9.8.4's HLS is broken. This will be solved by the next Hackage bump. Also an issue on master.
• Darwin is broken for GHC < 9.6. Tracked at Build failure: haskell.compiler.ghc{8107,902,92*,94*} on *-darwin #367686, also an issue on master.
• tasty-papi / mplayer-spot fail due to broken non-Haskell dependencies.

[vcunat]

I guess I cancel the jobset? It's still generating new jobs...

[sternenseemann]

Yeah, those jobs won’t be shared with any other jobset. I’ll start the next iteration in the next few days.