Skip to content

extra-dep is ignored when other version already in snapshot #5679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
andreasabel opened this issue Feb 15, 2022 · 11 comments
Closed

extra-dep is ignored when other version already in snapshot #5679

andreasabel opened this issue Feb 15, 2022 · 11 comments

Comments

@andreasabel
Copy link

TL;DR: Unclear meaning of extra-dep when another version is already in the snapshot. Should it error/warn? Improve documentation?!

The following stack file did not work under Windows:
https://github.com/hackage-trustees/hackage-cli/blob/3c0f18bec5cdcb7a6ed92220335b942307b458ae/stack-8.6.5.yaml

resolver: lts-14.27

extra-deps:
- http-io-streams-0.1.2.0
- brotli-streams-0.0.0.0
- brotli-0.0.0.0

The reason is that HsOpenSSL-0.11.4.17 (via http-io-streams) in the snapshot refers to C library eay32 which has been renamed to crypto.

I tried to force HsOpenSSL-0.11.6.1 by adding it to extra-deps. However, stack still used HsOpenSSL-0.11.4.17:
https://github.com/hackage-trustees/hackage-cli/runs/5199834098?check_suite_focus=true

I would have assumed that specifying a specific library version under extra-deps would have a similar effect than --constraint='HsOpenSSL==0.11.6.1' passed to cabal (or constraint: HsOpenSSL==0.11.6.1 in cabal.project).

The documentation isn't telling me what happens if an extra-dep is already in the snapshot:

### extra-deps
This field allows you to specify extra dependencies on top of what is
defined in your snapshot (specified in the `resolver` field mentioned
above). These dependencies may either come from a local file path or a
Pantry package location.

I think it should be more explicit about this case.

It seems that I need to drop HsOpenSSL first before I can request a different version:

# Force use of newer HsOpenSSL, used by http-iostreams
drop-packages:
- HsOpenSSL

extra-deps:
  # HSOpenSSL-0.11.6.1: Windows: Use libcrypto instead of eay; allow pkg-config.
- HsOpenSSL-0.11.6.1
  # Needed because of drop-packages: HSOpenSSL
- base64-bytestring-1.1.0.0
- ghc-byteorder-4.11.0.0.10
- xor-0.0.1.0
  # Missing from lts-14.27:
- brotli-0.0.0.0
- brotli-streams-0.0.0.0
- http-io-streams-0.1.6.0
@sjakobi
Copy link
Member

sjakobi commented Mar 7, 2022

I've definitely used extra-deps to override the version provided from the resolver. If that behaviour has changed or doesn't work reliably, I think that would be a bug in stack.

@sjakobi
Copy link
Member

sjakobi commented Mar 8, 2022

This FAQ entry suggests quite directly that extra-deps override the snapshot versions.

@andreasabel
Copy link
Author

I got another instance of this issue:

Maybe extra-deps that do not appear in the build-depends of the .cabal file are ignored?

This would mean that mediate dependencies cannot be updated unless they are needed due to a configuration error otherwise?!

@mpilgrem
Copy link
Member

On Windows 11, I can build hackage-cli with stack and resolver lts-14.27 (GHC 8.6.5), with one exception. I am using this stack-8.6.5.yaml:

resolver: lts-14.27
packages:
- .

extra-deps:
  # HSOpenSSL-0.11.6.1: Windows: Use libcrypto instead of eay; allow pkg-config.
- HsOpenSSL-0.11.6.1
  # Missing from lts-14.27:
- brotli-0.0.0.0
- brotli-streams-0.0.0.0
- http-io-streams-0.1.6.0@rev:0

- base64-bytestring-1.1.0.0@sha256:190264fef9e65d9085f00ccda419137096d1dc94777c58272bc96821dc7f37c3,2334
- Cabal-3.4.1.0@sha256:48e64c97688149fac15445803830177248f15a9b1a783389efed5e375d70d2d0,31402
- ghc-byteorder-4.11.0.0.10@sha256:e345720de7b28ba1bf434775d34d3b94da8e8dd5dc24469f008e1f82717c0352,1594
- xor-0.0.1.1@sha256:60929c5aa763534e1bcfe15d787c99a582ae17e3d9697fb86e53c3403d1e5b3b,2950

To do that, I need first to install some C libraries, namely:

❯ stack --stack-yaml=stack-8.6.5.yaml exec -- pacman -S mingw-w64-x86_64-brotli
❯ stack --stack-yaml=stack-8.6.5.yaml exec -- pacman -S mingw-w64-x86_64-openssl

The exception is that the registration of brotli-0.0.0.0 fails (extract):

brotli         > Registering library for brotli-0.0.0.0..
brotli         > Cabal-simple_Z6RU0evB_2.4.0.1_ghc-8.6.5.exe:
brotli         > 'C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-8.6.5\bin\ghc-pkg.exe'
brotli         > exited with an error:
brotli         > brotli-0.0.0.0: Warning: haddock-interfaces:
brotli         > C:\sr\snapshots\70e78788\doc\brotli-0.0.0.0\brotli.haddock doesn't exist or
brotli         > isn't a file
brotli         > brotli-0.0.0.0: library-dirs: /mingw64/lib is a relative path which makes no
brotli         > sense (as there is nothing for it to be relative to). You can make paths
brotli         > relative to the package database itself by using ${pkgroot}. (use --force to
brotli         > override)
brotli         > brotli-0.0.0.0: dynamic-library-dirs: /mingw64/lib is a relative path which
brotli         > makes no sense (as there is nothing for it to be relative to). You can make
brotli         > paths relative to the package database itself by using ${pkgroot}. (use
brotli         > --force to override)
brotli         >
Progress 1/4

--  While building package brotli-0.0.0.0 (scroll up to its section to see the error) using:
      C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_2.4.0.1_ghc-8.6.5.exe --builddir=.stack-work\dist\e626a42b register
    Process exited with code: ExitFailure 1

@mpilgrem
Copy link
Member

The problem I was having with brotli-0.0.0.0 (on Windows 11) stems from part of brotli.cabal, namely pkgconfig-depends: libbrotlienc, libbrotlidec. If I replace that in a local fork of the package with extra-libraries: libbrotlienc, libbrotlidec and replace the extra-deps: - brotli-0.0.0.0 with a reference to the local fork under packages:, then I can build hackage-cli with stack (version 2.7.5) and resolver lts-14.27 (GHC 8.6.5).

@mpilgrem
Copy link
Member

In summary, I can't reproduce the difficulty in using extra-deps: to override the version of HsOpenSS in the resolver.

@mpilgrem
Copy link
Member

@andreasabel, is this still an issue for you? I could not reproduce it.

@andreasabel
Copy link
Author

Ok, I will repeat the experiment. (Could also be related to caching so that it cannot be reproduced on a fresh instance.)

The documentation isn't telling me what happens if an extra-dep is already in the snapshot

I found the information here:

Shadowing semantics, described
[here](https://docs.haskellstack.org/en/v1.5.1/architecture/#shadowing), are
applied to your configuration. So, if you add a package to your `packages` list,
it will be used even if you're using a snapshot that specifies a particular
version. Similarly, `extra-deps` will shadow the version specified in the
resolver.

@andreasabel
Copy link
Author

Well, in the repetition of my experiment,

removing the drop-packages, just asking for HsOpenSSL-0.11.6.1 (https://github.com/hackage-trustees/hackage-cli/pull/43/files#:~:text=andreasabel-,now,-Weird%2C%20why%20does), I find that stack uses HsOpenSSL-0.11.7.2 instead.
So, previously it used a older version, now it uses a newer version ?? I thought if I ask for HsOpenSSL-0.11.6.1 in extra-deps, then this is the version I get.

https://github.com/hackage-trustees/hackage-cli/runs/6712301152?check_suite_focus=true#step:9:993

HsOpenSSL                        > configure
HsOpenSSL                        > [1 of 2] Compiling Main             ( C:\\Users\runneradmin\AppData\Local\Temp\stack-ae5ec2e32a3742ef\HsOpenSSL-0.11.7.2\Setup.hs, C:\\Users\runneradmin\AppData\Local\Temp\stack-ae5ec2e32a3742ef\HsOpenSSL-0.11.7.2\.stack-work\dist\274b403a\setup\Main.o )
data-default-instances-old-locale> copy/register
data-default-instances-old-locale> Installing library in C:\sr\snapshots\ca5389af\lib\x86_64-windows-ghc-8.10.7\data-default-instances-old-locale-0.0.1-7TyKZqwxsVt5wEA26i3Q11
data-default-instances-old-locale> Registering library for data-default-instances-old-locale-0.0.1..
HsOpenSSL                        > [2 of 2] Compiling StackSetupShim   ( C:\\sr\setup-exe-src\setup-shim-Z6RU0evB.hs, C:\\Users\runneradmin\AppData\Local\Temp\stack-ae5ec2e32a3742ef\HsOpenSSL-0.11.7.2\.stack-work\dist\274b403a\setup\StackSetupShim.o )
HsOpenSSL                        > Linking C:\\Users\\runneradmin\\AppData\\Local\\Temp\\stack-ae5ec2e32a3742ef\\HsOpenSSL-0.11.7.2\\.stack-work\\dist\\274b403a\\setup\\setup.exe ...
HsOpenSSL                        > Configuring HsOpenSSL-0.11.7.2...
HsOpenSSL                        > build

(Unrelated, out of curiousity: why does stack name a directory x86_64-windows-ghc-8.10.7 when we are building with GHC 8.6.5?)

@milgrem wrote:

On Windows 11, I can build hackage-cli with stack and resolver lts-14.27 (GHC 8.6.5), .... I am using this stack-8.6.5.yaml:

Did you check whether stack actually uses the version of HsOpenSSL you ask for, or simply another one like in my experiment?

@mpilgrem
Copy link
Member

Returning to this after a hiatus, with Stack 2.15.1 and a simple cross-platform example. A single-package project that implements the ansi-terminal example and has package.yaml:

name:                edTest
version:             0.1.0.0
dependencies:
- base >= 4.7 && < 5
- ansi-terminal

executables:
  edTest:
    main:                Main.hs
    source-dirs:         app
    ghc-options:
    - -threaded
    - -rtsopts
    - -with-rtsopts=-N

With stack.yaml:

snapshot: lts-22.12

stack ls dependencies includes, as expected, ansi-terminal-1.0.2 (being in the snapshot).

With stack.yaml:

snapshot: lts-22.12
extra-deps:
- ansi-terminal-1.1

stack ls dependencies now includes, as expected, ansi-terminal-1.1 (the extra-dep shadows the snapshot).

@mpilgrem
Copy link
Member

I am going to close this issue given the passage of time and because I cannot reproduce it, including on a Mac mini/M1 with Stack 2.13.1. If I have a simple one-package project with package.yaml (extract):

dependencies:
- base >= 4.7 && < 5
- HsOpenSSL

and stack.yaml:

snapshot: lts-20.26

then stack ls dependencies includes (as expected) HsOpenSSL-0.11.7.5 (being the package version in the snapshot).

If I change stack.yaml to:

snapshot: lts-20.26
extra-deps:
- HsOpenSSL-0.11.7.6

then stack ls dependencies includes (as expected) HsOpenSSL-0.11.7.6 (the extra-dep shadowing the package version in the snapshot),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants