cmd/go: go mod tidy ignores go.work file

[bozaro]

What version of Go are you using (go version)?

$ go version
go version go1.18beta1 linux/amd64

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bozaro/.cache/go-build"
GOENV="/home/bozaro/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/bozaro/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/bozaro/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.18"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.18/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18beta1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/bozaro/github/go-work-play/tools/go.mod"
GOWORK="/home/bozaro/github/go-work-play/go.work"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4292218461=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Minimal reproducing repository: https://github.com/bozaro/go-work-play/tree/go-mod-tidy

Full script:

$ git clone https://github.com/bozaro/go-work-play.git -b go-mod-tidy
$ cd go-work-play/tools
$ go mod tidy
go: finding module for package github.com/bozaro/go/go-work-play/shared/foo
github.com/bozaro/go/go-work-play/tools/hello imports
	github.com/bozaro/go/go-work-play/shared/foo: module github.com/bozaro/go@latest found (v0.0.0-20200925035954-2333c6299f34), but does not contain package github.com/bozaro/go/go-work-play/shared/foo

What did you expect to see?

I expect go.mod and go.sum updated with current working copy state.

What did you see instead?

I see that go mod tidy try to get modules for shared go modules from repository ignoring go.work content.

[bcmills]

Tidiness is a property of an individual module, not a workspace: if a module is tidy, then a downstream consumer of the module knows which versions to use for every dependency of every package in that module.

If you don't particularly care about downstream consumers having a package that is provided by the workspace, you can use go mod tidy -e to ignore the error from the missing package.

Otherwise, you either need to publish the workspace dependencies before running go mod tidy (so that they have well-defined upstream versions), or tell the go command what those versions will be (using a replace and require directive in the individual module).

[bcmills]

@matloob, for Go 1.19 I wonder if we should augment the go.work file to be able to declare explicit intended versions for the modules in the workspace. Then go mod tidy could use those versions instead of looking upstream. 🤔

[bozaro]

Currently with reproducing repository (https://github.com/bozaro/go-work-play/tree/go-mod-tidy).

I can run:

cd tools/hello
go run .

And go run works like workspace go modules declared as replace in current tools/go.mod.

But for go mod tidy I need add replace manually.

This behaviour looks like inconsistent: I expects that go run and go mod tidy would use same dependencies.

[liadmord]

What valid use cases are there for go.work if we are still required to fully implement the go.mod in order to work on multi module repo?
@bcmills

[AlmogBaku]

This is also happening for go get ./...

I agree with @liadmord, the only benefit I see get from the go.work is a better "understanding" of my IDE
Or - maybe I'm doing something wrong here?

[Jictyvoo]

I agree with @bozaro @liadmord and @AlmogBaku

I have a monorepo private project where I have multiple modules, and on each one I need to insert a lot of replace statements for each direct and indirect dependencies. With go workspaces I really want to be possible to run go mod download or go mod tidy without the needed to write a replace statement in every go.mod file in my workspace.

[maxsxu]

Encountering the same issue with go1.18.2.

Furthermore, some other go mod commands also failed. See sample repo: https://github.com/maxsxu/go-workspace

• go mod vendor

➜  server   go mod vendor 
atmax.io/go-workspace/server imports
        atmax.io/go-workspace/commons/pkg: no required module provides package atmax.io/go-workspace/commons/pkg; to add it:
        go get atmax.io/go-workspace/commons/pkg

• go mod verify

➜  server   go mod verify 
atmax.io/go-workspace/server : missing ziphash: open hash: no such file or directory

Expected behaviour: The go mod commands should not try to find modules which have been declared in go.work. which I think will make go workspace more meaningful.

[Gbps]

My expected functionality is that a go mod tidy on a go.work enabled workspace should feel like a bunch of phantom replace statements inside the go.mod without editing the file. If the module is at github.com/Gbps/mymodule and I tell my go.work that the module can be found in /path/to/mymodule, then go mod tidy should pull the version from my local repo instead of going to github at all.

If I don't commit my go.work, then anyone who uses my go.mod afterwards will have to go to GitHub, which is correct! Now I can edit my local copy and test changes, but when I'm ready to push, I can tidy the correct version into the go.mod and push both the mymodule local dev repo and my current project. This makes sense!

Once I must make an edit to go.mod to include a local path string for any feature of the go mod command line to work, all the purpose of go.work is lost on me. I should never have to taint go.mod with local paths, go mod tidy can find what commit I'm working with locally through go.work and update it accordingly. It should be no different than if I pushed a commit to github and ran go mod tidy to pull the newer version into my go.mod.

In other words, if I add this line to go.mod:

replace github.com/Gbps/mymodule => /path/to/mymodule/

Then go mod tidy works as expected. If I have use /path/to/mymodule/ in my go.work, it should accomplish the same thing. The benefit being I don't have to taint go.mod with my local paths, as that is going to need to be committed to remote whereas go.work isn't.

[opengs]

This is a very big issue for me in the case of creating Docker containers :( I cant use go mod tidy and copy the dependency graph to the Dockerfile. So every time I build a container I have to download downstream dependencies instead of doing go mod download.

Golang is considered the language for microservices and In the current state of the go work it's very hard to work with them in monorepo. Creating multiple-container environment is incredibly hard :(. Basically, every time I hit docker-build it has to download downstream dependencies. Currently, I have two ways to fix this:

1. Push packages to the repo. But if I have ~30 microservices and most of them are 1 file program - it's a very poor experience.
2. replace and require create too many relative dependencies and it's very hard to maintain.

I expect 4 things from the golang workspaces:

1. go mod to know about other modules and don't try to download the local package from the internet. Maybe traverse up in the folder hierarchy and check if go.work exists?
2. you can create a clear dependency graph so it can be used for caching in Docker containers
3. functions like go work tidy and go work download, because now I have to enter every package and type go mod commands. I just want to tidy and download everything from one place, and because we already have a path to the modules in go.work file - it should be trivial.
4. functionality to reduce the number of similar dependencies (download one specific version and force to use it in every module in workspace). This was done by for example turborepo, lerna, and other monorepo management tools.

So now, the only thing why I'm using go work is to have a better codding experience in VsCode. Most of the negative comments that I hear from colleagues about golang are about poor modules/workspaces management. I think, go work is a very important feature to have and it's a very important direction to go.

[HeCorr]

I've faced this issue yesterday but my case is a bit different.. My replace statements are in the go.work file, which seemed to solve go run errors, but when attempting go mod tidy on any of the modules, it is indeed ignored.

The solution is simply to move the replace statements to the go.mod files as explained by Gbps, but keeping track of what package uses what internal package is a bit of a hassle. Is this considered a bug also?

[bcmills]

go mod tidy is intended to update the module's self-contained dependencies. It doesn't use the workspace because in general one may work on multiple independently-maintained modules in the same workspace, and if you're preparing an upstream commit you definitely don't want that commit to rely on unpublished local modifications.

go work sync is intended to update the modules within a workspace to reflect the dependencies selected in that workspace. That is probably what you want if you are working on a set of modules that are all maintained as a single unit.

Folks who are commenting here (and please bear in mind https://go.dev/wiki/NoPlusOne!) — have you considered go work sync for your use-case? If so, what does go work sync do (or not do) that makes it unsuitable for your use-case?

[AlmogBaku]

@bcmills I'm not sure what the intended workflow is, but I think the feedback here is that the DX is really not intuitive.

[burdiyan]

Folks who are commenting here (and please bear in mind https://go.dev/wiki/NoPlusOne!) — have you considered go work sync for your use-case? If so, what does go work sync do (or not do) that makes it unsuitable for your use-case?

I wish there were go work tidy or something similar that would do the same as go mod tidy for each module in the workspace. Or maybe go work sync could have a flag for that, or even do it by default. This way one could easily manage a single repository with many modules, keeping everything in sync and tidy :) without having to switch to every single module directory to keep it tidy.

[mlaventure]

go work sync does not work for me in that scenario either unfortunately. I have a go.work for a monorepo containing yet to be published packages (the domain also does not exist yet). e.g.:

use (
    ./foo
    ./bar
    ./fubar
)

replace not-yet-created.com/foo/foo v0.0.0-00010101000000-000000000000 => ./foo

However that replace directive seems to be ignored by go work sync

$ go work sync
go: not-yet-created.com/foo/[email protected]: unrecognized import path "not-yet-created.com/foo/foo": https fetch: Get "https://not-yet-created/foo/foo?go-get=1": dial tcp: lookup not-yet-created.com on 127.0.0.53:53: server misbehaving

[li-jin-gou]

I've encountered a similar problem. the current solution is write a shell to go mod replace first and then go mod tidy and removing go mod replace. Hope GO Support go mod tidy Ignore go.work.

[EduartePaiva]

As someone who is starting with Go, and learning micro-services, I want to share my perspective, on this workspace thread.

Here's my user case that I was working on with workspace:

• created module micro-service-1.

• created module micro-service-2.

• created now a utils module, currently micro-services-1 uses it.

• created a gRPC module, it have generated code that allows ms-1 and ms-2 to talk to each other.

Now I want to build docker images for ms-1 and ms-2.

what would be my advantage for using workspace in this scenario?

• Faster download times for building images, for example, ms-2 don't use utils module so it don't need to copy it and download it's dependencies.
• Better mental model for external packages.
• from a newbe perspective this looked as the right way to do things.

now here's the problem that is emerging:

• having to manually tidy things
• now for tidy to work I have to do workaround with this: removing link style from module names, publishing as a package, using replace (learned about this now)

Too much trouble, probably I'll just stop using workspace and forget the possible advantages

My suggestion for a "go mod tidy --workspace", or "go work tidy".

It should recursively go to workspace modules and tidy them too.

For example, if inside ms-1 I run the supposed "go work tidy", it should see that it's using the utils module and recursively tidy utils too.

If this behavior is debatable put a flag for it e.g. "go work tidy --recursive".

[matloob]

@EduartePaiva I'm wondering why you didn't place all your packages in the same module? That is the happy path. And then no workspace is required.

To others on this issue: I would be like to speak to people about the requirements they have that push them towards multi-module repos with go.work files. I think if we could better understand those requirements we would be better equipped to see how to satisfy them.

Please let me know if you're interested in talking about your use case.

[EduartePaiva]

@matloob being honest I was following a microservice course and the teacher started using workspace to separate each microservice. At the time I didn't questioned why, just thought that it was the way to do things.

[nemith]

@matloob The main reason I was looking for a workspace solution is that a large module doesn't play well with creating docker images from both a caching perspective as well as a building perspective. Workspaces have some challenges here as well without jumping to a solution like Bazel to maintain build dependencies.

On top of not being able to independently version any libraries for external consumption.

Ultimately we did go with a common module and some complex docker contexts to make sure everything came together.

I am no longer at $company with a main repo. This plus things like github CI and the like really loves to make monorepos benefits start to dwindle a lot.

[jurienhamaker]

@matloob In my case I am rewriting several discord bots to go. However they have some shared code which I am putting in their own package & the services themselves.

You can check it out here

Now I got a dockerfile for each and I copy the mod file for the app, then the shared mod file and download them seperately etc.

[KevinFairise2]

Hi,
Working on https://github.com/DataDog/datadog-agent, we decided to split our repository in several modules to be able to import part of the agent code without importing the whole code base. It definitely worked but brought some challenge on the tooling to manage the multi module repository.
We recently introduced go.work and added it to the repository. It improves things, because we are now able to keep dependencies version synchronised between the modules and it simplifies running test in the different modules. But we have not been able yet to remove all the replace from the go.mod files because of we the things mentioned in this issue.

[GustavoStingelin]

My suggestion for a "go mod tidy --workspace", or "go work tidy".

It should recursively go to workspace modules and tidy them too.

For example, if inside ms-1 I run the supposed "go work tidy", it should see that it's using the utils module and recursively tidy utils too.

If this behavior is debatable put a flag for it e.g. "go work tidy --recursive".

I did this using a Makefile to handle the tidy

[matloob]

@EduartePaiva Ah, got it. Yeah, in general multi-module repositories should only be reached only if a single repo repository isn't working for your use case.

@nemith Do you have modules in your workspace that aren't go get-able? And if so, what's the constraint that prevents them from being go get-able? My understanding is that that is when this issue occurs- when there are modules in your workspace that can't function independently.

@jurienhamaker Are the bots in separate modules to ease deployment similar to the problems @nemith mentioned?

@KevinFairise2 What were the downsides of importing the whole codebase? Was it to restrict the size of dependencies that were brought in with the imports? Also, are each of the modules independently go-gettable? If you're able to do a call I'd definitely be interested in talking about your use case.

[jurienhamaker]

@matloob The bots itself can function independently as long as they are compiled together with the shared module. The shared module is their only dependency.

The dockerfiles in the repository show basically that, I don't add service y & z to the docker container of x.

[nemith]

@matloob I am going to go off on memory here (maybe I will try to reproduce it if i get time later). But as I remember everything worked fine until I added a new module that wasn't yet committed to the repo.

Despite being listed in go.work no other modules could try to use it as it would try to contact the repo (github in my case) to get information on the imported repo instead of just looking locally first.

I guess that makes it not go getable but that was just because of order of operations and not desired end state.

If i remember right there were workarounds but all of it added too much cognitive load to the team who's job was to write software and not understand the rube goldberg building machine as much as I could.

[majelbstoat]

@matloob one consideration is that Go is not necessarily the whole picture for any given project. We have a product with a front-end in Typescript, multiple backend services in Go, and an app in Swift. They all communicate using GRPC, and depend on a single common set of protobufs, from which we generate types, clients, and server stubs.

Keeping all the protos, services, apps, and frontends in one repo allows us to change protobufs, regenerate all the client and server stubs, and commit the consequences to the whole system all in one go. in particular, we can understand if anything is going to break before changes are merged to main. New versions of affected services and apps can also be automatically deployed based on changes to protos on which they depend.

All the go code being in one repo is a subset of that need, and similar to what others have said: common grpc (and other) code in lib/go/src and service code is services/service-one, services/service-two etc.

To make this work, we commit go.work to the repo (even though that's not its intended use). go.work and go.mod look like:

go.work:

use (
  ./lib/go/src
  ./services/service-one
  ./services/service-two
)

lib/go/src/go.mod:

module company.com/common

services/service-one/go.mod:

module company.com/service-one

Each service can then import "company.com/common/grpc" and everything resolves nicely. Building an application is simple too: go build company.com/service-one/app. We use go work sync to keep all the package dependencies consistent across services.

This all works very nicely. The only slight issue is that go mod tidy within a given service gives errors:

go: company.com/service-one/app imports
	company.com/common: cannot find module providing package company.com/common: unrecognized import path "company.com/common": https fetch: Get "https://company.com/common?go-get=1": dial tcp: lookup company.com: no such host

this is fine, because there's nothing to retrieve, but it would be ideal if it paid attention to the use directives of go.work and didn't bother to do the lookups.

[majelbstoat]

As for why the services aren't all in one module together - it's because they aren't all one module. They are applications that happen to be written in go, amongst others that are not. They are each deployed separately. They have different environment variables, they have different kubernetes configs. We choose to organize our code by logical service, not by implementation language.

This is service one:

├── app
│   ├── main.go
├── env
│   ├── dev.env
│   ├── production.env
│   └── test.env
├── go.mod
├── go.sum
└── k8s
    ├── dev.yaml
    ├── production.yaml
    └── test.yaml

This is service three:

├── app
│   ├── index.ts
├── env
│   ├── dev.env
│   ├── production.env
│   └── test.env
├── package.json
└── k8s
    ├── dev.yaml
    ├── production.yaml
    └── test.yaml

The enforced regularity of this layout greatly simplifies devops in a heterogeneous repo, and reduces cognitive load on developers as the system and team grows.

[matloob]

@majelbstoat I'd like to understand this better. Is the goal of placing the go.mod files in each of the service directories (rather than the top level) to make the versions of the dependencies of the services independent of each other?

[majelbstoat]

The idea is that each go.mod represents the minimal set of dependencies for each service/library. This helps optimize deployments, where we save the go build cache and results of go get for each service in a cache keyed by the go.sum of that service (and the common library go.sum).

.circleci.yaml:

restore_cache:
      keys:
          - go-mod-v3-{{ checksum "lib/go/src/go.sum" }}-{{ checksum "services/<< parameters.service-name >>/go.sum"}}

Practically speaking, that means if we make a change to the dependencies of service-one, we only have to expire the cache for service-one, not every go service. If they were all in one go.mod, we would have to redeploy every service, and blow every cache, for every dependency change to every service, even if the other services didn't depend on the new dependency. (Or else make our deployment decision-making process quite a bit more complex.)

We actually want common dependencies for each service to be at the same version. For example, it's important that the version of the google.golang.org/grpc package is the same for all services. go work sync does this for us - it ensures all go.mods that include google.golang.org/grpc include it at the same highest version. It gets run automatically pre-commit if any go.mod has changed.

Aesthetically too, each service imports packages from the common library. If they all were in the same go.mod, imports for a service from the common library would be like import "company.com/lib/go/src/common/grpc", which kinda ugly and a pain to change if we ever changed its relative location in the repo. It doesn't happen often, but it has happened. In the current setup, it requires one edit to one use line in go.work.

[fayep]

I think that I have a simpler reason for why this is incorrect behavior right now.
go.mod and go.sum are distributable artifacts. go.work is for local overriding of dependency modules for more streamlined development. Once development is done, I should not need to modify go.mod as I will have pushed my simultaneously developed dependency module as well as this one.

Expected behavior:
I have an as yet unpublished dependency which I have added to my workspace ../mymod. If I go get or go mod tidy it should not attempt to locate mymod upstream because it is defined in my workspace. It should continue to get other dependency modules not included in my workspace. On finishing development and publishing the dependency, I remove the workspace and go get to update the dependencies in go.mod and go.sum

Observed behavior:
go mod tidy and go get retrieve dependency modules but fail on trying to load my unpublished dependency. The newly retrieved modules are not added to go.mod and go.sum leaving the external dependencies unresolved.

Workaround:
add replace myrepo.myid.mymod => ../mymod which we used to do before workspaces. This is unsatisfactory because once development is done, you need to go back into go.mod and remove this override.

[matloob]

Thanks for the comments. We're still thinking about this. Let me know if you're interested in doing a call to talk about your use case.

It still seems to me that changing the behavior of go mod tidy isn't correct. In the common case with a workspace that multiple modules are participating in, the module isn't tied to a specific workspace. So tidy should be able to tidy the module without knowledge of the workspace. Similarly, go get by default should operate on the module alone without knowledge of the workspace. So we'll need to think of other ideas to address these use cases.

@fayep, It seems to me that a temporary replace directive is appropriate to use in this case. If the module doesn't exist upstream yet, you can't have a proper requirement on that module. The replace directive allows the module's build graph to be complete until that dependency module is released and can be required. This is especially important to tidy a module. It would be nice if we could reduce the friction in the go get use case though.

[mytechcn]

Thanks for the comments. We're still thinking about this. Let me know if you're interested in doing a call to talk about your use case.

It still seems to me that changing the behavior of go mod tidy isn't correct. In the common case with a workspace that multiple modules are participating in, the module isn't tied to a specific workspace. So tidy should be able to tidy the module without knowledge of the workspace. Similarly, go get by default should operate on the module alone without knowledge of the workspace. So we'll need to think of other ideas to address these use cases.

@fayep, It seems to me that a temporary replace directive is appropriate to use in this case. If the module doesn't exist upstream yet, you can't have a proper requirement on that module. The replace directive allows the module's build graph to be complete until that dependency module is released and can be required. This is especially important to tidy a module. It would be nice if we could reduce the friction in the go get use case though.

---

I have a common module that includes some common functions, which are referenced by all projects. For example, project A now references it in the workspace, which is private.
But now when A uses go mod tidy to manage project dependencies, it will report an error.
It is necessary for multiple modules to reference each other, which is a common basic feature in Java and C #. This bug should undoubtedly be resolved as soon as possible.
We have been dragging on for too long.

[spekary]

Since there are multiple situations where you may or may not want the behavior, why not add a flag for go mod tidy to pay attention to go.work?

[matloob]

@spekary As we currently define it, a tidy module is that it is one that includes the appropriate set of requirements to satisfy its imports. Changing or adding a concept of being tidy with respect to a workspace would complicate things further and modules can be fairly complex as is. We'd like to find a solution that wouldn't add to the complexity of the system.

In many of the cases listed on this issue the workspaces do logically function as a single module (for example they are all versioned together, and have the same set of dependency versions) but there are various blockers or concerns that prevent the use of a single module. And there are cases where the modules do function differently but there's friction in the process of making them available to the modules that depend on them via go get We'd like to address those issues for the users facing them. And for those users facing different issues, we'd like to talk to them about their use cases to try to better understand the problem space.

[spekary]

I suppose I am suggesting approaching the problem differently. When a developer adds a go.work file, they are saying that they are doing something special. I thought that was the idiom. They are composing a working environment that anticipates eventually removing the go.work file, running go mod tidy, and checking in the go.mod file. But for the time being, they need an environment that can work differently.

Before go.work, we would put "replace" statements into the go.mod file, but then we would have to remember not to check that in. go.work was supposed to solve that problem, but I guess it didn't, because go.mod still would need to behave differently because of the dependencies implied in the use of different packages, and so we still have to remember not to check in go.mod during development. I guess we need a go.mod.work file (and of course the ensuing go.mod.sum.work file).

So how about this: When go.work is present, go mod tidy creates a go.mod.work file. When a go.mod.work file is present, the various go tools use it instead of go.mod. Yeah, it could create some confusion if the developer doesn't realize that is what is happening. Perhaps the go tools could call that out while processing files, like pointing out that a go.mod.work file was found? In any case, the developer started the process by creating a go.work file, and the developer can fix the problem by removing it and the go.mod.work file.

The alternative I was suggesting previously is to use a flag to cause go mod tidy to behave this way. But as I think about it, that flag would have to be across all go tools to tell them to use go.mod.work if it exists. Not an easy problem to solve, but I prefer Go's general use of convention to solve these problems, rather than a proliferation of flags in the tools.

[matloob]

I suppose I am suggesting approaching the problem differently. When a developer adds a go.work file, they are saying that they are doing something special. I thought that was the idiom. They are composing a working environment that anticipates eventually removing the go.work file, running go mod tidy, and checking in the go.mod file. But for the time being, they need an environment that can work differently.

Yes, a go.work is saying that the user is doing something special.

Before go.work, we would put "replace" statements into the go.mod file, but then we would have to remember not to check that in. go.work was supposed to solve that problem, but I guess it didn't, because go.mod still would need to behave differently because of the dependencies implied in the use of different packages, and so we still have to remember not to check in go.mod during development. I guess we need a go.mod.work file (and of course the ensuing go.mod.sum.work file).

Are you seeing issues in the case where the dependency module already exists? If you are I'd like to hear more about the specifics. There is definitely more friction in the case where the dependency module doesn't yet exist. We may have to think more about those cases but they should only happen "once" when the dependency module is being introduced.

So how about this: When go.work is present, go mod tidy creates a go.mod.work file. When a go.mod.work file is present, the various go tools use it instead of go.mod. Yeah, it could create some confusion if the developer doesn't realize that is what is happening. Perhaps the go tools could call that out while processing files, like pointing out that a go.mod.work file was found? In any case, the developer started the process by creating a go.work file, and the developer can fix the problem by removing it and the go.mod.work file.

The alternative I was suggesting previously is to use a flag to cause go mod tidy to behave this way. But as I think about it, that flag would have to be across all go tools to tell them to use go.mod.work if it exists. Not an easy problem to solve, but I prefer Go's general use of convention to solve these problems, rather than a proliferation of flags in the tools.

I'd like to hear more about your use case. In the case where you're using go.work to do something special (and the dependency module already exists at a previous version in a go gettable state) you should be able to work on both modules, go getting dependencies as needed. You may not be able to tidy until you release the dependency module if the changes made compared to the previous version affect the dependency graph, but tidying the module shouldn't be necessary until you release. Are you seeing issues running go get?

[nemith]

for example they are all versioned together

This would not be true for my use case and was the sole reason why we wanted to to use a go.work solution inside a monorepo was to version shared libraries for consumption outside of the monorepo separately than services in the monorepo which follow more of a trunk based/v0.0.0/unversioned model.

The "monorepo" was really just a shared team repo with other teams doing their own multi or monorepos.

We ended up dropped this requirement and following trunk based versioning for the libraries as well but only has a workaround. Combined with also poor monorepo support in docker and github actions it was all not worth it.

[matloob]

@nemith Yeah the use case you mentioned where you're adding a new module that hasn't been released yet has some friction currently. Just to make sure I understand your use case, you weren't trying to do module-mode operations like go get of a dependency or go mod tidy but instead couldn't even do a go build in workspace mode, right?

In that case, currently we'd recommend adding a replace directive until there's a module version that could be properly required. Though I can see how it might get confusing to have to use a replace together with a workspace. We definitely need to improve our documentation around that. And try to see if there are ways to make that use case better.

[nemith]

@matloob The most friction was when adding a new module with inter-mod dependencies. A replace could have fixed it but additional additional load of updating docker files to include sources for dependent modules, using a path-filter with GHA to fix CI on dependent changes, on top of adding replace directives everywhere just wasn't worth it.

Being naive and not knowing better i tried go.work as a way to make modules "aware" of each other, the obviously isn't the case. Also I am not even sure what the valid use cases for go.work is anymore if replace is still a valid and promoted option. Confusion is the right word.

It's sad as a monorepo approach solves a lot of other issues of dependency testing and integration.

[ohir]

@matloob

adding a new module that hasn't been released yet has some friction currently.

It is not a "friction" - for the bootstrap and air-gapped environments it is entirely counterproductive. Even in open source scenarios it makes the bootstraping phase a pain. Frankly, I can not see where and how workspaces current functionality might give less work. I personally am back to the mod replaces managed by a script. Making a workspace just muds the water, makes project brittle, and doubles the maintenance time.

[crazy-matt]

@matloob If you care about downstream consumers, then don't forget about just all the other devs who do not see why go workspaces can't address that issue and go mod tidy handle it if one forces it to do so. Bootstrapping and CI/CD are valid use cases where it's currently painful with replace directives management.
So why not just adding a flag to go mod tidy to make it workspace aware? That way, the current implementation is still valid and not breaking any downstream consumer implementation. If one wants to use that flag then assume they know what they're doing, if they don't, they will learn quick. This issue has been raise 3 years ago.