Description
What version of Go are Windows users using (go version
)?
$ go version go version * windows/*
Does this issue reproduce with the latest release?
Yes, it is due to architectural differences between Windows and other operating systems.
What did Windows people do?
Nothing.
We include regular end users of Go programs, including system administrators and developers who use software written in Go like Docker or software which is partially written in or makes use of Go components like Visual Studio and Git.
What did you expect to see?
That software written in go continues to work after disaster recovery or changing a sub directory from being physically present to being a reparse point to a large fresh volume, without hunting down configuration issues with go software because it persisted paths after calling EvalSymLinks
, assuming that the path would be more stable than the path configured explicitly by the system administrator after calling EvalSymLinks
.
Why should EvalSymLinks no-op on Windows?
On Windows, the path that EvalSymLinks
returns is not more stable, rather it is in very common situations random due to the GPT volume identifier being random and is hence the path returned by this API is also not under control of the administrator. The administrator might be forced to change reparse points or hard links due to disk failure, disaster recovery, resolving disk full problems, and similar. Hence, the use of EvalSymLinks
for the purposes it is designed for is wrong on Windows, even if its use is justified in the same software on other operating systems.
What did you see instead?
Docker services stop being able to start, even though they should have run and should continue to run from C:\Docker
, regardless of the fact that this directory resolves now to a Windows path on a different volume with a different GPT unique identifer and hence a different volume path in the form \\?\Volume{guid}
.
Activity
networkimprov commentedon Jul 7, 2020
Thanks for raising this. It's a proposal (I think) so doesn't have to follow the standard issue template. It's also rather long and a bit grumpy.
Could you thin it down to a list of specific problems (with examples) in the current API, and possible solutions?
Note that the Windows port has long depended on contributions from outside the team at Google.
ericwj commentedon Jul 7, 2020
Well sorry for being grumpy, but leaving Windows end users like me with a myriad of problems that they should wholeheartedly own as stewards of go is ... more grumpiness. Notice that I was explicitly mentioning Google and not the well-meaning community, but they are limited to playing whack-a-mole and naturally everybody cares about just their own little issues.
As I said, I have no experience with Go. I just counted them and I only ever wrote 54 lines in 2 files and just 20 lines that actually do something.
So, although I might shorten this post and make it less grumpy, making this specific I feel first of all doesn't truly reflect the fact I find issues in about every function in path_windows.go and file_windows.go and probably other
*_windows.go
files. I mean from palmface issues like theVolumeName
API not returning or intending even to return any kind of volume name but a root path to hacking strings and finding ':' where you might also in fact be looking for '\\?\Volume{GUID}' to stripping '\\?\' from a path after 'GetFinalPathNameByHandle' returns that, breaking long paths and special paths in the process, to suspecting that happens in perhaps every file API in go by looking at the issues in this repository, on Stack Overflow and in the docker repository, etc.I mean how did they get that name so wrong? Was that API proposal not reviewed or did they like the confusion that would result but which they could hope would go unnoticed? Is go a joke?
networkimprov commentedon Jul 7, 2020
What Go app has been giving you trouble, Docker? Have you filed an issue there? If so, can you point it out?
It's the app's responsibility to work correctly on platforms it supports. I patch the Go stdlib on Windows where needed.
ianlancetaylor commentedon Jul 7, 2020
@ericwj I'm sorry that you are frustrated but this issue doesn't seem to give us a useful way forward.
I doubt this will make you feel any better but pretty much all the Windows path support was written and contributed by Windows experts who do not work at Google.
It seems to me that
filepath.EvalSymlinks
is well defined, and it is clearly useful on Unix systems. Perhaps programs that run on Windows shouldn't be calling it, but that doesn't mean that we should remove it.CC @alexbrainman @mattn
[-]Consider obsoleting or no-op'ing `EvalSymLinks` and review all Go path handling on Windows[/-][+]path/filepath: consider obsoleting or no-op'ing `EvalSymLinks` and review all Go path handling on Windows[/+]ericwj commentedon Jul 7, 2020
Are you saying Google isn't involved? They are at least reviewing every single change - or they appear to be.
If you personally wrote a new language, I would agree. Imho Google has a larger responsibility. But hey, the fact the first thing Microsoft does contributing to chromium is to add features for people with disabilities speaks to the company culture at Google refusing to do anything of the sort for 12 whole years.
@ianlancetaylor I am not asking for the API to be removed if it is 'normal' to use that on *nix, although I think overruling an administrator is about always wrong by principle regardless of the operating system. But go knows whether you are on Windows or cross building for it and if it has infrastructure to give warnings or suggest fixes for this API on Windows then that would be step 1.
I didn't personally file issues with Docker, but the are legion links to be found in their issues and on Stack Overflow with people stumbling on error messages like
unable to prepare context: unable to evaluate symlinks in context path
and similar.I did file issues with Microsoft about this Visual Studio behavior, but since these errors do not reside in actual Microsoft code they de-prioritized.
About Git I am honestly just unaware of the components that make up the system. Visual Studio Code works quite allright, but Visual Studio has some go somewhere in its Git extension or I don't know where such that these problems magically appear in recent versions of Visual Studio.
ianlancetaylor commentedon Jul 7, 2020
Google is a huge company. I'm not sure what it means to say that Google is involved. Certainly Google employees are heavily involved with the development of Go. But it's not accurate to say that Google employees are reviewing every single change. Go is an open source project. Many changes to Go are written, reviewed, and submitted by people who are not Google employees, without anybody employed by Google being involved at all.
networkimprov commentedon Jul 7, 2020
Well, Eric, since you learned Go, maybe you could fix some of those Docker issues (perhaps by replacing EvalSymlinks in their codebase), and then file an issue or two here about how Go could support them better :-)
But we need specific bug reports to make progress.
ericwj commentedon Jul 8, 2020
So is .NET. And .NET actually works even with Windows-isms on a long list of *nix variants, because Microsoft as primary steward of the ecosystem is taking its proper responsibility. The downside is they have so much tests that it is hard to find any actual code if you look for a definition of something in /dotnet/runtime say.
Look, let's talk solutions. I am fed up hitting brick walls every time someone chooses go over another language and then shoves it onto my system as part of some software that I use - that's why I'm here. I identified very concrete issues and I can probably write code from plain primitives that uses the Windows API properly to fill in go API surface -- I just can't realistically write it in go. Would I help writing plain C# implementing some of the broken API's in go?
I could perhaps even write tests that will show how to break go, but I'd have to do that mostly in PowerShell and C#.
ericwj commentedon Jul 8, 2020
PowerShell to create .vhdx files and mount/unmount them - system administration stuff, which I don't think go can do today at all.
49 remaining items
alexbrainman commentedon Jul 11, 2020
Sorry, but I cannot find them. Perhaps this thread is too long. Why don't you create new issue for every bug you found? Please, do not ignore the issue template, but fill in as much details as you can.
Please, be aware that Go issue tracker is only used to track real bugs / problems, and submit change proposals.
Thank you.
Alex
ericwj commentedon Jul 11, 2020
Yes just leave this one to be about
no-op ing EvalSymLinks
and nothing else.Even the way to do that without breaking anyone is a separate discussion about alternatives and what they look like.
Perhaps this issue could be about changing documentation, or at least adding a few lines of text to make people aware of why this API could bite them on Windows and about expectations by Windows foaks that they shall not default to using it.
networkimprov commentedon Jul 11, 2020
You can just submit a CL with documentation fix clarifying what happens for certain paths.
BTW English folk is spelled like the German volk :-)
ericwj commentedon Jul 11, 2020
From Old Frisian *foks, from Proto-Germanic *fuhsaz. More at English fox.
Hmm, what did I call myself? ;)
ericwj commentedon Jul 12, 2020
I think messed this up. This patch was supposed to be just against
ericwj/go
for a new pull request. Did it get added to #40095? How do I fix that?networkimprov commentedon Jul 12, 2020
It was added to the CL, yes. Just revert. Then make a new branch, and cherrypick the docs commits to that, and file a new PR.
Also, it should describe EvalSymlinks problems on Windows, not make blanket statements.
ericwj commentedon Jul 12, 2020
That is explained in the commit message, but I'm not sure that is visible. Just revert? How? Commit after getting the old file from history?
networkimprov commentedon Jul 12, 2020
git revert <commit>
is a common op; it creates new commits. Alternatively make a new commit by hand that removes the docs change.networkimprov commentedon Jul 12, 2020
Best to close this issue, and make a new one for the docs change. Should post the new docs there for discussion before filing PR/CL.