Expose `cabal.project` in library component?

[mitchellwrosen]

Would it be possible to expose cabal.project data structures and parsing code in a public cabal-install library component?

This would be helpful for cabal-install wrapper tools. For example, I may want to write a tool that automagically runs one ghcid session for each component in a package (since ghci can only load one component at a time). Doable for single-package projects using the Cabal library, but more difficult for entire projects with multiple .cabal files in different subdirectories.

[phadej]

Note: since cabal-install-2.4 there are no such thing as cabal-install lib component.

I'm not saying that there couldn't be some small specific independent library, but that library ought to have well-designed API (i.e. not everything cabal-install uses internally).

[23Skidoo]

This has been discussed previously, see #5476. I think that perfect is the enemy of good enough here, so personally am +1 on following the GHC API approach and exposing the existing internals with no guarantees of stability between major releases. Though I'd like to do this in gradual fashion and expose the solver lib first.

[phadej]

The #5476 already mentioned the fact which is worth repeating:

If we don't care about minor releases breaking stuff, than having a public library is ok. But in that case cabal-install would be a PITA library to depend upon.

An alternative is to be very strict of which patches to let in / backport.

E.g. http://hdiff.luite.com/cgit/cabal-install/diff?id=2.4.1.0&id2=2.4.0.0 has a breaking API change right at the top of the diff.

[23Skidoo]

Right. I think that we can live with the latter if major cabal-install releases are frequent (every six months or so).

[hvr]

If we don't care about minor releases breaking stuff

If for one strongly care about this; and this would effectively mean that we won't be able to fix some things within a patch-level or minor-release, except with high costs, due to the hard requirement to not break the API contract; and fixes might be delayed by ~6 months because of this. Are we sure we want to incur that cost?

[phadej]

@hvr, I don't think having a major release of cabal-install without corresponding Cabal major is a biggie.

E.g.

• cabal-install-3.0 uses Cabal-3.0
• cabal-install-3.2 has a "breaking" fix, still uses Cabal-3.0
• cabal-install-3.4 uses Cabal-3.4

So Cabal-3.2 won't ever exist.

I actually don't care whether cabal-install and Cabal versions are in sync.

[23Skidoo]

On a related note, we should start date-versioning WIP lib:Cabal snapshots. So e.g. when a breaking change is added to Cabal-3.1, the version is bumped to Cabal-3.1.20190624 and cabal-install-3.1's dependency on lib:Cabal gets bumped to Cabal >= 3.1.20190624. Same for filterConfigureFlags. We can date-version cabal-install itself as well.

[ekmett]

@phadej> But in that case cabal-install would be a PITA library to depend upon.

It isn't already? Almost every piece of code that I have that depends on Cabal/cabal-install internals breaks with each new release already. I don't see this as making that any worse.

[hvr]

I don't think having a major release of cabal-install without corresponding Cabal major is a biggie.

We can do that at some additional cost (note that we currently rely on the versions being synced to simplify some of the codepaths -- those will become harder to maintain when this symmetric simplificiation doesn't hold anymore); and it's just kinda ironic since we intentionally made lib:Cabal and exe:cabal sync up their major versions starting with cabal-install-1.16, and now you've basically saying that it was pointless to do so in the first place...

[emilypi]

I'm implemented this in #7358 after considering this discussion. The PR deals with a few knock-on effects of the decision to not expose the cabal-install library coming from a different angle: coverage is broken. This is particularly egregious for several reasons:

1. The dogfooding effort required to test the cabal-install internals becomes a circus exercise for the user and the devs. Users must jump through Makefile hoops and lousy tooling support as a result of library and doc definitions not being immediately available. This incentivizes fewer tests, infrequent contributions, and poor coverage, as well as poor code hygiene because discoverability is low. I also notice we are constantly re-exporting the same shit in many different modules, and we have like, 3 different different Preludes working redundantly.

2. The devs need to maintain this dogfooding nonsense which bleeds from the project right up into the CI practice. I just went through this. It sucks. Don't make me do that again. It's unnecessary. We're already PVP compliant. Like @ekmett said, the API breaks and is broken in fun ways all over the place, and we do a major version bump every release except on very rare occasions. I would rather converge on correctness than stay in this broken state.

3. Until HPC manages to adequately support sublibrary components and multiproject builds, the lesser sin is exposing the library. I'd rather have tooling that works. Right now, cabal-install has so many broken commands that I can't really justify not doing this. cabal-install is at 27% top-level coverage. That is abysmal for a production executable. Getting this up is a high priority.

4. You are effectively lying as the distributor of the executable when you hand people the source dist with the prod cabal file and say "yes this is definitely cabal-install". You're not testing with the same data you're telling me is relevant for the package. Tests are an important part of the spec! This is actually a problem from an audit standpoint.

The calculus is pretty clear. I'll be merging it asap.

[phadej]

@emilypi are you including cabal-testsuite tests in your coverage calculations? That's is where the most testing is happening.

[emilypi]

@phadej no, I'm considering just cabal-install, masking coverage for everything else. I do agree that Cabal seems to be well-covered with the cabal-testsuite, but it would take more follow-up work to get cabal-testsuite into modern shape (a test framework, not using the executable approach etc) so that we could actually generate reports for it as well.