Teach `cabal.project` about "dev dependencies"

[michaelpj]

Many build tools have the concept of a "dev dependency". These are dependencies which are managed by the build tool, but which are not used by any of the actual build products. Instead, they are there to provide tools that developers of the project will use.

Why have such things managed by the build tool? Can't users just install them separately? There are a few advantages, such as convenience and consistency between developers, but I think there is one that is very important in our case: the build tool may have information that makes it much easier for it to get the "right" tool. For example, many development tools themselves expect to interact with the language toolchain and build tool. It can therefore be important to pick a tool version that's compatible with the toolchain that you're using.

This is becoming very pertinent in the Haskell tooling ecosystem as more things depend on the ghc library. A tool which depends on ghc, like haskell-language-server, must be built with a version of ghc that exactly matches the compiler the user is using for a particular project (which may indeed differ between projects!). As it stands, this creates a lot of work for users and developers. Users must carefully manage their tool installations to make sure they have precisely the right version of the tool installed for the project in question.

But we already have a tool that has all the information about what version of ghc you are using and how that affects what versions of haskell-language-server you can use: namely cabal!

---

So much for the motivation, what am I concretely suggesting?

• cabal.project learns a new field: dev-tools, which is a list of component specifications a la build-tool-depends.
• Some cabal command will build all the dev-tools and install them into some project-local location.
  • I don't know what an appropriate command is or an appropriate location (although I think it should be project-local): suggestions welcome!

I don't think the dev-tools build should include constraints from the components in the project - the implicit constraints on ghc, Cabal etc. should be enough.

---

Partial duplicate of #5588, but I'm making quite a different case, which I think is much more compelling than just convenience.

Many of the other build tools which have "dev dependencies" also use them for things that have better solutions in cabal, e.g. for build tools, test dependencies etc. I think there's still a useful role for dev dependencies that are really for developers only.

A short list of things I expect people might want to put into dev-tools:

• Language servers: haskell-language-server/ghcide etc.
• Formatters: ormoulu/stylish-haskell/brittany etc.
• Linters: hlint/stan (stan does actually rely on .hie files and hence depends on ghc!)

One could even imagine people who build their Haskell with alternative systems like shake might use a cabal.project to get the shake binaries in the first place.

[michaelpj]

I realised it's worth saying something about why consistency matters.

Some people like to format their codebase with stylish-haskell or remove all lints with hlint. Some people like to enforce that these conditions are met. But "is this codebase formatted correctly" also depends on the version of stylish-haskell you use. So your entire team needs to have the right version installed.

Just being able to write dev-tools: stylish-haskell:exe=2.0.0 or whatever would solve this quite effectively.

[michaelpj]

To be clear what I meant about the command, I imagine something like cabal dev-install or cabal install --dev would do it, but I don't really know enough about how cabal install works to comment sensibly at the moment.

[phadej]

I think this can be implemented as out-of-cabal-install prototype first to figure out the details.

[phadej]

In general I guess this is fine, yet why don't you use Nix? what if there is some dev-dependency which is not on Hackage, or not written in Haskell at all?

For stan you need same GHC, yet for HLint you don't want to recompile it with different GHCs (you really don't, it takes so long, that you'll lose interest in the project before it completes). There are details to be figured out.

[michaelpj]

I do use Nix! However:

• It's nice for things to work for people who don't use Nix too
• Nix-based tools can benefit from this information as well.
  • e.g. haskell.nix has a bunch of machinery to get "another Haskell package but definitely built against the same boot libs as your project" for precisely this case. It would be nicer to just get this information from cabal, then haskell.nix can just DTRT out of the box.

For sure, you're still on the hook for non-Haskell dev dependencies, which is why I think this isn't a compelling feature if we're
just talking about convenience of installation. But for me the ghc dependency issue is really quite annoying, and I think could be solved uniquely well by cabal.

I wouldn't mind compiling multiple versions of hlint for different projects. It's slow and annoying, sure, but it's much less annoying than having the wrong hlint. But yes, hlint is a weak case compared to the tools with ghc dependencies.

---

OOI, how would you go about solving this out-of-cabal-install? Don't you need access to the solver etc.?

[michaelpj]

I think haskell-language-server is a nice one to consider, because:

• It's very finnicky to install currently
• We'd like it to be easy to install even for beginners, people would like to use it on university courses etc.
• It would be really easy to install with this proposal

[phadej]

I will believe when I see a PoC. And as I see it, it can be done as an external cabal-install-dev-tools executable.

[phadej]

Side note: I really hope that GitHub discussions will soon be available for everyone. This is not an issue, this is an open-ended discussion.

• Discussing between two us is not fruitful
• Yet I don't want whole Cabal user base to discuss stuff on Cabal issue-tracker.

This is wrong place for this kind of discussions. But we don't have better one.

[michaelpj]

Happy to close and take it elsewhere, but I do think it's useful to have something for people to find if they have a similar idea and I don't know where they would look except here.

[phadej]

See #6445 (comment)

From what I see, the decision is made, so I'm saying this with the hope to find a better way to stay on top of these discussions and contribute in a timely manner when and where they happen.

[alexbiehl]

FWIW we have extra-packages in the cabal.project file. In principle it'd allow you to do cabal run stylish-haskell if you had extra-packages: stylish-haskell. Unfortunately it needs a bit of tuning in the target selector to fully work.

[alexbiehl]

I traced the target selector code and found that if you use fully qualified syntax like

$ cabal run :pkg:stylish-haskell:exe:stylish-haskell

It happily starts building stylish-haskell but fails with

cabal: Unknown executable stylish-haskell in package
stylsh-hskll-0.11.0.0-646a8e66

So I guess we might want to

1. Make the target selector syntax a bit more ergonomic. I think this has to do with how we represent extra-packages as NamedPackage. They don't carry component info like the other local package types do and can't give good hints for the target selector resolution.

2. Figure our why the cabal run code can't find the exe in the package plan (see matchingPackagesByUnitId)

[michaelpj]

Yes, I tried extra-packages, and it's close to what I want, but @phadej indicated that it's somewhat unsupported, so I'm abandoning that. I also hit the issues you mentioned with accessing the executable component properly, but I assumed that was down to extra-packages being janky.

However, I've realised that I can do what I want by wrapping cabal v2-install:

• It takes into account project constraints.
• It can install executables from arbitrary packages.

I'm going to try out a little tool based around this idea.