Skip to content

cabal new-build & {alex,happy} #4009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hvr opened this issue Oct 19, 2016 · 7 comments
Closed

cabal new-build & {alex,happy} #4009

hvr opened this issue Oct 19, 2016 · 7 comments

Comments

@hvr
Copy link
Member

hvr commented Oct 19, 2016

UPDATE: Partially (specifically for the case when alex/happy are to be installed into the nix store because they were requested via build-tools), this turns out to be a regression introduced by 8dc39db

new-build currently isn't able to self-bootstrap happy and alex (which are two special cases in Haskell's ecosystem and for it's imho legitimate to add special logic to new-build), and this is causing problems for the new-build based Travis scripts which don't install alex/happy via binary packages anymore:

$ cabal get happy-1.19.5
Unpacking to happy-1.19.5/
$ cd happy-1.19.5/
$ cabal new-build
Resolving dependencies...
In order, the following will be built (use -v for more details):
 - happy-1.19.5 (exe:happy) (first run)
[1 of 1] Compiling Main             ( /tmp/happy-1.19.5/dist-newstyle/build/x86_64-linux/ghc-8.0.1.20161014/happy-1.19.5/setup/setup.hs, /tmp/happy-1.19.5/dist-newstyle/build/x86_64-linux/ghc-8.0.1.20161014/happy-1.19.5/setup/Main.o )

/tmp/happy-1.19.5/dist-newstyle/build/x86_64-linux/ghc-8.0.1.20161014/happy-1.19.5/setup/setup.hs:21:1-8: warning: [-Wtabs]
    Tab character found here, and in 39 further locations.
    Please use spaces instead.
Linking /tmp/happy-1.19.5/dist-newstyle/build/x86_64-linux/ghc-8.0.1.20161014/happy-1.19.5/setup/setup ...
Configuring happy-1.19.5...
Preprocessing executable 'happy' for happy-1.19.5...
setup: The program 'happy' is required but it could not be found

and similiarly for alex:

$ cabal get alex-3.2.1
Unpacking to alex-3.2.1/
$ cd alex-3.2.1/
$ cabal new-build
Resolving dependencies...
In order, the following will be built (use -v for more details):
 - alex-3.2.1 (exe:alex) (first run)
[1 of 1] Compiling Main             ( /tmp/alex-3.2.1/dist-newstyle/build/x86_64-linux/ghc-8.0.1.20161014/alex-3.2.1/setup/setup.hs, /tmp/alex-3.2.1/dist-newstyle/build/x86_64-linux/ghc-8.0.1.20161014/alex-3.2.1/setup/Main.o )
Linking /tmp/alex-3.2.1/dist-newstyle/build/x86_64-linux/ghc-8.0.1.20161014/alex-3.2.1/setup/setup ...
Configuring alex-3.2.1...
Preprocessing executable 'alex' for alex-3.2.1...
setup: The program 'happy' is required but it could not be found

after adding a line build-tools: happy == 1.19.* to alex.cabal, we get now:

$ cabal new-build
Resolving dependencies...
In order, the following will be built (use -v for more details):
 - alex-3.2.1 (exe:alex) (configuration changed)
Configuring alex-3.2.1...
Preprocessing executable 'alex' for alex-3.2.1...
setup: The program 'alex' is required but it could not be found

IMO, it ought to be possible to install happy without having a prior version of happy installed, and likewise possible to install alex without having a prior alex installed.

And in fact, alex includes pre-generated .hs code in its .tar.gz for that:

./src/Parser.y
./src/Scan.x
...
./dist/build/alex/alex-tmp/Scan.hs
./dist/build/alex/alex-tmp/Parser.hs

and similarly happy includes:

./src/AttrGrammarParser.ly
./src/Parser.ly
...
./dist/build/happy/happy-tmp/Parser.hs
./dist/build/happy/happy-tmp/AttrGrammarParser.hs

I think it makes sense for alex to not specify build-tools: alex, and take that as a hint that there's a pre-generated .x source bundled in the source-tarball (however, the legacy pathname ./dist/build/alex/alex-tmp/Parser.hs is imho a bad choice, but we will have to support that for the sake of existing alex/happy releases). Otoh, if a package specifies build-tools: alex, I'd expect the .x files to be regenerated by cabal (and build alex if needed).

Maybe for the future, something interacting with the new autogen-modules field would make sense here for a more principled solution.

/cc @ezyang

@hvr hvr added this to the 2.0 milestone Oct 19, 2016
@hvr
Copy link
Member Author

hvr commented Oct 19, 2016

Turns out, there's a very similiar problem when using legacy sandboxes: haskell/alex#54

@hvr
Copy link
Member Author

hvr commented Oct 19, 2016

After talking to @dcoutts I was informed that this is an unintenteded regression as a side-effect of 8dc39db (#3975); moreover, #130 is somewhat related

@hvr hvr added the type: bug label Oct 19, 2016
@hvr
Copy link
Member Author

hvr commented Oct 19, 2016

The following partial revert of 8dc39db was suggested by @dcoutts:

diff --git a/cabal-install/Distribution/Client/ProjectBuilding.hs b/cabal-install/Distribution/Client/ProjectBuilding.hs
index 0f3ae1b..0a69c43 100644
--- a/cabal-install/Distribution/Client/ProjectBuilding.hs
+++ b/cabal-install/Distribution/Client/ProjectBuilding.hs
@@ -749,11 +749,11 @@ withTarballLocalDirectory verbosity distDirLayout@DistDirLayout{..}
         -- builddir under the tarball src dir to keep path name lengths down.
         BuildAndInstall ->
           let tmpdir = distTempDirectory in
-          withTempDirectory verbosity tmpdir "src"   $ \unpackdir ->
-          withTempDirectory verbosity tmpdir "build" $ \builddir -> do
+          withTempDirectory verbosity tmpdir "src"   $ \unpackdir -> do
             unpackPackageTarball verbosity tarball unpackdir
                                  pkgid pkgTextOverride
             let srcdir   = unpackdir </> display pkgid
+                builddir = srcdir </> "dist"
             buildPkg srcdir builddir

UPDATE: I've tested the patch above improves the situation

hvr added a commit that referenced this issue Oct 20, 2016
Packages like `alex` include pre-generated lexer/parser source
in their source tarball as e.g.

  dist/build/alex/alex-tmp/Scan.hs
  dist/build/alex/alex-tmp/Parser.hs

to avoid having to have alex already installed before building
alex...

Unfortunately, 8dc39db broke
packages relying on this accidental feature by changing where the
distdir points to. This patch partly reverts that commit and
therefore addresses the regression aspect of #4009.
hvr added a commit that referenced this issue Oct 21, 2016
Packages like `alex` include pre-generated lexer/parser source
in their source tarball as e.g.

    dist/build/alex/alex-tmp/Scan.hs
    dist/build/alex/alex-tmp/Parser.hs

to avoid having to have `alex` already installed before building
`alex`...

Unfortunately, 8dc39db broke
packages relying on this accidental feature by changing where the
distdir points to. This patch partly reverts that commit and
therefore addresses the regression aspect of #4009.

This fix was suggested by @dcoutts
@ezyang
Copy link
Contributor

ezyang commented Oct 21, 2016

OK fixed now.

@ezyang ezyang closed this as completed Oct 21, 2016
@hvr
Copy link
Member Author

hvr commented Oct 21, 2016

@ezyang the build-tools:-regression is fixed, but we still can't build a cabal unpack'ed alex/happy this way

@ezyang ezyang reopened this Oct 21, 2016
@ezyang
Copy link
Contributor

ezyang commented Oct 21, 2016

I can't easily test this but I'll take your word for it. Why doesn't it work for cabal unpack?

@23Skidoo 23Skidoo modified the milestones: 2.0.1, 2.0 Feb 17, 2017
@23Skidoo 23Skidoo modified the milestones: 2.0.1, 2.0.2 Sep 19, 2017
@23Skidoo 23Skidoo modified the milestones: 2.0.2, 2.4 Aug 29, 2018
@23Skidoo 23Skidoo modified the milestones: 2.4, 2.4.1 Sep 17, 2018
@23Skidoo 23Skidoo modified the milestones: 2.4.1.0, 2.4.2.0 Apr 26, 2019
@phadej phadej modified the milestones: 2.4.2.0, 3.4 Nov 27, 2019
@gbaz
Copy link
Collaborator

gbaz commented Sep 2, 2021

Just tested and this appears very much resolved at this point.

@gbaz gbaz closed this as completed Sep 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants