Skip to content

build.zig.zon improve workflow to update dependency hash #16972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
marler8997 opened this issue Aug 26, 2023 · 8 comments · Fixed by #17363
Closed

build.zig.zon improve workflow to update dependency hash #16972

marler8997 opened this issue Aug 26, 2023 · 8 comments · Fixed by #17363
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. zig build system std.Build, the build runner, `zig build` subcommand, package management
Milestone

Comments

@marler8997
Copy link
Contributor

marler8997 commented Aug 26, 2023

Here's my typical process for synchronizing the hash field for a dependency in build.zig.zon:

  1. Set the hash field to a bogus value
  2. Run zig build
  3. Update build.zig.zon with the actual hash displayed in the error message
  4. Run zig build again to verify it worked

My initial proposal is to introduce a special value "?" that tells Zig to update build.zig.zon for us. The new flow becomes.

  1. Set the hash field to "?"
  2. Run zig build to update the hash/verify it works

There have since been other good suggestions:

from @zenMaya: add a command for zig to download and report the hash of a URL (i.e. zig download URL). This might be a good command to make available regardless of whether we need it for this specific use case?

from @mpfaff: put hashes into a separate "lock file" meant to be managed more "automatically" instead of manually. This isn't a full solution but could make other ideas easier.

from @Pyrolistical: add a flag (or maybe a separate command?) that tells zig to update hashes (instead of using a special hash value). I believe this would require zig to assume that all hashes could be invalid meaning it would need to download/recalculate all dependency hashes and update them accordingly, or, zig could maintain a known URL to HASH mapping and only update hashes where this mapping has not been calculated yet. In other words, if the user modifies a URL, zig will see that it hasn't calculated a hash for it yet.

@drunderscore
Copy link

I can't imagine ever wanting my build tools to modify or write to my build files.

Is the proper way of doing this not: being given a valid hash and URL from the package developer?

@zenMaya
Copy link

zenMaya commented Aug 26, 2023

there also is another way, which is something like: (taken from guix download)

zig download <package source>

This caches the package and prints its hash to be added to the zon. But does not modify the build file itself nor does it link the package into the project.

@marler8997
Copy link
Contributor Author

@zenMaya yeah that's also a reasonable workflow

@mpfaff
Copy link
Contributor

mpfaff commented Aug 26, 2023

Another option would be having some sort of lock file that keeps track of the hashes for any dependency without one set explicitly.

@Pyrolistical
Copy link
Contributor

@zenMaya I agree, but you can also support both by adding a --update flag.

zig download --update build.zig.zon <package source>

This way we need to explicitly opt into this feature

@AdamGoertz
Copy link
Contributor

I prefer a build flag like —update to setting some magic hash value, but I agree having a feature like this would be nice.

It will be particularly useful for vendored dependencies with #14603. If you’re working on multiple modules simultaneously it’s a major pain to repeatedly change the dependency hashes.

@marler8997 marler8997 changed the title build.zig.zon dependency hash deferral value build.zig.zon improve workflow to update dependency hash Aug 27, 2023
@marler8997
Copy link
Contributor Author

marler8997 commented Aug 27, 2023

Lot's of good ideas, I think I'd be fine with any of them. I've updated the title/description to summarize the various ideas.

@jayrod246
Copy link
Contributor

If we’re worried about the build system modifying build scripts, add a flag that just always fetches packages, ignores hash checking. Then you would fill in the hashes yourself when you’re ready to publish?

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. zig build system std.Build, the build runner, `zig build` subcommand, package management labels Oct 2, 2023
@andrewrk andrewrk added this to the 0.12.0 milestone Oct 2, 2023
andrewrk added a commit that referenced this issue Oct 2, 2023
zig fetch [options] <url>
zig fetch [options] <path>

Fetches a package which is found at <url> or <path> into the global
cache directory, printing the package hash to stdout.

Closes #16972
Related to #14280

Additionally, this commit:

* Adds uncompressed .tar support to package fetching
* Introduces symlink support to package fetching
andrewrk added a commit that referenced this issue Oct 3, 2023
zig fetch [options] <url>
zig fetch [options] <path>

Fetches a package which is found at <url> or <path> into the global
cache directory, printing the package hash to stdout.

Closes #16972
Related to #14280

Additionally, this commit:

* Adds uncompressed .tar support to package fetching
* Introduces symlink support to package fetching
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. zig build system std.Build, the build runner, `zig build` subcommand, package management
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants