Skip to content

x/build/cmd/relui: dynamically respond to nested modules in monthly golang.org/x tagging workflow #68873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dmitshur opened this issue Aug 14, 2024 · 6 comments
Closed

x/build/cmd/relui: dynamically respond to nested modules in monthly golang.org/x tagging workflow #68873

dmitshur opened this issue Aug 14, 2024 · 6 comments
Assignees
@dmitshur
Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Unreleased

Comments

@dmitshur
Copy link
Member

Issue #48523 was about tagging golang.org/x repos monthly, automatically.

It's been implemented for a while, but one area for improvement is that the relui workflow currently hard-codes extra directories that need to be tidied in some x/ repos because they have nested modules and/or replace directives (and in very very rare cases, there may be a workspace via a checked in go.work files). The relevant code:

https://cs.opensource.google/go/x/build/+/master:internal/task/tagx.go;l=393-404;drc=37c49bbfbc400bf05e0bc91b0042279a5220a2b9

This is the tracking issue to improve this to cause less overhead as golang.org/x repos evolve.

We already do the work of dynamically walking the repo tree to find nested modules e.g. when testing all golang.org/x repos (and their nested modules), see https://source.chromium.org/chromium/infra/infra_superproject/+/main:infra/go/src/infra/experimental/golangbuild/testmode.go;l=370-392;drc=d2899a13c5223f9c36cf236efae90ae0cecb684f. So that might come in handy.

CC @golang/release.
@dmitshur dmitshur added Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 14, 2024
@dmitshur dmitshur added this to the Unreleased milestone Aug 14, 2024
@gabyhelp
Copy link

Related Issues and Documentation

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/605316 mentions this issue: internal/task: add oscar submodules to tagx

gopherbot pushed a commit to golang/build that referenced this issue Aug 28, 2024
@rsc
internal/task: add oscar submodules to tagx
3a2d61c 
For golang/go#68873.

Change-Id: Ibcd81ec99314819dd5e2faf0c23ae8c33f8a02a7
Reviewed-on: https://go-review.googlesource.com/c/build/+/605316
Reviewed-by: Carlos Amedee <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
@dmitshur dmitshur self-assigned this Oct 3, 2024
@dmitshur dmitshur moved this to In Progress in Go Release Oct 3, 2024
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 3, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/617776 mentions this issue: internal/task: update tagging for x/oscar/internal/devtools nested module

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/617778 mentions this issue: internal/task: dynamically find nested modules to tidy during tagging

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/617777 mentions this issue: internal/gitfs: add a generated copy of x/website/internal/gitfs

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/622396 mentions this issue: internal/task: read nested module's own go.mod files

gopherbot pushed a commit to golang/build that referenced this issue Oct 24, 2024
@dmitshur @gopherbot
internal/task: update tagging for x/oscar/internal/devtools nested mo…
95fa777 
…dule

A fourth nested module was added to x/oscar in CL 615155. The rest of
the CL stack makes manually maintaining this list no longer necessary.

For golang/go#68873.

Change-Id: Ib775a2c84d269b5d9bbd85ba12ded1b78b711706
Reviewed-on: https://go-review.googlesource.com/c/build/+/617776
Auto-Submit: Dmitri Shuralyov <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>
Reviewed-by: Michael Knyszek <[email protected]>
gopherbot pushed a commit to golang/build that referenced this issue Oct 24, 2024
@dmitshur @gopherbot
internal/gitfs: add a generated copy of x/website/internal/gitfs
40c0547 
It's generated rather than copied manually, so that it is clear its
canonical source still lives at golang.org/x/website/internal/gitfs.

For golang/go#68873.

Change-Id: I2ec03384666d5c230e59c36db118e7f3969a8e11
Reviewed-on: https://go-review.googlesource.com/c/build/+/617777
Reviewed-by: Dmitri Shuralyov <[email protected]>
Auto-Submit: Dmitri Shuralyov <[email protected]>
Reviewed-by: Michael Knyszek <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
gopherbot pushed a commit to golang/build that referenced this issue Oct 24, 2024
@dmitshur @gopherbot
internal/task: dynamically find nested modules to tidy during tagging
fa508ab 
Fetch the git tree where go.mod is being updated, walk over its content
to find nested modules to tidy. This way even if new nested modules are
added, they'll be handled without needing additional attention here.

The existing test cases continue to pass (i.e., the x/tools/gopls module
is still being tidied). It required adding a minimal HTTP(S) git server
to FakeGerrit, to act like the real Gerrit server, or rather just enough
for gitfs to be able to successfully clone from it.

For golang/go#68873.

Change-Id: Ic571f2edc6345d25558591e531d58ecce70f1851
Reviewed-on: https://go-review.googlesource.com/c/build/+/617778
Reviewed-by: Michael Knyszek <[email protected]>
Reviewed-by: Dmitri Shuralyov <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Auto-Submit: Dmitri Shuralyov <[email protected]>
@github-project-automation github-project-automation bot moved this from In Progress to Done in Go Release Oct 24, 2024
@gabyhelp gabyhelp mentioned this issue Dec 3, 2024
Open
@gabyhelp gabyhelp mentioned this issue Apr 8, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dmitshur dmitshur

Labels
Builders x/build issues (builders, bots, dashboards) NeedsFix The path to resolution is known, but the work has not been done.
Projects
Go Release
Archived in project
Milestone
Unreleased
Development

No branches or pull requests
3 participants
@dmitshur @gopherbot @gabyhelp