Description
What version of Go are you using (go version)?
$ go version go version go1.15.6 linux/amd64
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env GO111MODULE="" GOARCH="amd64" GOBIN="/usr/local/go/bin/" GOCACHE="/root/.cache/go-build" GOENV="/root/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/ua/go/pkg/mod" GONOPROXY="github.ibm.com/*" GONOSUMDB="github.ibm.com/*" GOOS="linux" GOPATH="/home/ua/go" GOPRIVATE="github.ibm.com/*" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64" GCCGO="gccgo" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build393178232=/tmp/go-build -gno-record-gcc-switches"
What did you do?
We have an app using a lot of Golang plugins developed by many teams. It works very well with Golang Dep.(https://github.com/golang/dep) because you do not need to care any compatibility issues caused by the new packages of the plugins and all those in the vendor directory because they are all independent.
While when we migrate to Go Modules, there are various compatibility issues. For most scenarios, we need to refine the shared projects and then we have to change all plugins developed by many teams. The workload is huge.
Though finally, we can fix the problem with various ways, the cost is too big so that we have to go back to Golang Dep.
What did you expect to see?
Like with Golang Dep, if we can build the GoLang plugin successfully, then we can load the plugin with the main app and all plugins can work together without any issue.
What did you see instead?
Right now, when we successfully build a GoLang plugin, it does not mean the plugin can be loaded and it does not mean it can work with other plugins. There will be various "different version of package" issues.
Metadata
Metadata
Assignees
Labels
Type
Projects
Relationships
Development
Participants
Activity
mvdan commentedon Jan 17, 2021
This reads more like a question or a request for help than a bug report, since there isn't a way to reproduce the issue or any clear details pointing at a bug that can be fixed. Have you seen https://golang.org/wiki/Questions?
liurui-1 commentedon Jan 18, 2021
Thanks @mvdan .
I am not sure if this is just a question instead of a bug. Since after Go 1.14, the GO Modules is becoming the only active tool for dependency management. While it is good for building common standalone binaries, but the effort is too big to maintain app with many independent plugins.
If we do not care about the effort, we can resolve all dependency issue for a new plugin developed. But it is not a sustainable solution. When there is a new plugin developed, we may need to revise the main app and all other 100 plugins.
For example, we have following project as the shared "library".
go.mod
go.sum
It is very likely for some new plugin to use different packages from the above "go.sum". It will lead to the problem for example the main binary cannot load the plugin because of "different version of package" or different plugins cannot work together. The problem is not for one case that we cannot resolve with new revised configurations. The problem is that we may have to upgrade the main binary and all 100+ existing plugins to adapt to a new developed plugin.
And according to the statement in Go 1.14 release notes at "https://golang.org/doc/go1.14", I need to open an issue:
Module support in the go command is now ready for production use, and we encourage all users to migrate to Go modules for dependency management. If you are unable to migrate due to a problem in the Go toolchain, please ensure that the problem has an open issue filed. (If the issue is not on the Go1.15 milestone, please let us know why it prevents you from migrating so that we can prioritize it appropriately.)seankhliao commentedon Jan 18, 2021
how is this better with
depthen? don't you need to keep the versions in sync with each other withdepas well? or did it just mask the issue because no version information is available?liurui-1 commentedon Jan 19, 2021
@seankhliao
Till now, dep works very well for our scenarios. We have a git repo as the major "shared library". It is very stable and we do not change all its packages unless a big upgrade (actually there is no compatibility issue to add new packages to the "shared library"). The main binary code and all 100+ plugins are all based on this "shared library".
The "shared library" has its dependent packages in its own vendor directory like all plugins. GoLang considers packages in different vendor directories as different packages. This prevent converting unless using unsafe.Pointer (we do not suggest to use it). This is a big limit of functionality. While the advantage is much bigger. The plugin owners (our team and many partners) do not need to care about compatibility. The plugins can always run well as long as it can be built.
We do need Golang to provide something like a lightweight Java OSGi does.
seankhliao commentedon Feb 22, 2021
So same root cause as #18827 but the desired result is in the opposite direction
omeid commentedon Jun 8, 2021
I think it is more of a request for supporting vendoring outside main module too. Currently,
go moddoes not support it.https://golang.org/ref/mod#vendoring
seankhliao commentedon Jun 15, 2022
Does building everything under a shared workspace help?
All dependencies should be resolved to the same versions within a single workspace
liurui-1 commentedon Jun 16, 2022
We do not have shared build server with our partners. It is not feasible.
[-]Go Modules is not good for apps using a lot of Go plugins[/-][+]plugin: doesn't work well with modules, requires coordinating dependency versions[/+]titpetric commentedon Feb 4, 2023
@seankhliao if you were asking about go workspaces, then no - plugins have a
mainmodule, alas, where you load the plugin you also have amainmodule, and the dependencies to load the plugin need to be version compatible too - the plugin refuses to build and complains about only wanting one main module.