Skip to content

path/filepath: no-op path.filepath.EvalSymLinks on Windows #40104

Closed
@ericwj

Description

@ericwj

What version of Go are Windows users using (go version)?

$ go version
go version * windows/*

Does this issue reproduce with the latest release?

Yes, it is due to architectural differences between Windows and other operating systems.

What did Windows people do?

Nothing.

We include regular end users of Go programs, including system administrators and developers who use software written in Go like Docker or software which is partially written in or makes use of Go components like Visual Studio and Git.

What did you expect to see?

That software written in go continues to work after disaster recovery or changing a sub directory from being physically present to being a reparse point to a large fresh volume, without hunting down configuration issues with go software because it persisted paths after calling EvalSymLinks, assuming that the path would be more stable than the path configured explicitly by the system administrator after calling EvalSymLinks.

Why should EvalSymLinks no-op on Windows?

On Windows, the path that EvalSymLinks returns is not more stable, rather it is in very common situations random due to the GPT volume identifier being random and is hence the path returned by this API is also not under control of the administrator. The administrator might be forced to change reparse points or hard links due to disk failure, disaster recovery, resolving disk full problems, and similar. Hence, the use of EvalSymLinks for the purposes it is designed for is wrong on Windows, even if its use is justified in the same software on other operating systems.

What did you see instead?

Docker services stop being able to start, even though they should have run and should continue to run from C:\Docker, regardless of the fact that this directory resolves now to a Windows path on a different volume with a different GPT unique identifer and hence a different volume path in the form \\?\Volume{guid}.

Activity

networkimprov

networkimprov commented on Jul 7, 2020

@networkimprov

Thanks for raising this. It's a proposal (I think) so doesn't have to follow the standard issue template. It's also rather long and a bit grumpy.

Could you thin it down to a list of specific problems (with examples) in the current API, and possible solutions?

Note that the Windows port has long depended on contributions from outside the team at Google.

ericwj

ericwj commented on Jul 7, 2020

@ericwj
Author

Well sorry for being grumpy, but leaving Windows end users like me with a myriad of problems that they should wholeheartedly own as stewards of go is ... more grumpiness. Notice that I was explicitly mentioning Google and not the well-meaning community, but they are limited to playing whack-a-mole and naturally everybody cares about just their own little issues.

As I said, I have no experience with Go. I just counted them and I only ever wrote 54 lines in 2 files and just 20 lines that actually do something.

So, although I might shorten this post and make it less grumpy, making this specific I feel first of all doesn't truly reflect the fact I find issues in about every function in path_windows.go and file_windows.go and probably other *_windows.go files. I mean from palmface issues like the VolumeName API not returning or intending even to return any kind of volume name but a root path to hacking strings and finding ':' where you might also in fact be looking for '\\?\Volume{GUID}' to stripping '\\?\' from a path after 'GetFinalPathNameByHandle' returns that, breaking long paths and special paths in the process, to suspecting that happens in perhaps every file API in go by looking at the issues in this repository, on Stack Overflow and in the docker repository, etc.

I mean how did they get that name so wrong? Was that API proposal not reviewed or did they like the confusion that would result but which they could hope would go unnoticed? Is go a joke?

networkimprov

networkimprov commented on Jul 7, 2020

@networkimprov

What Go app has been giving you trouble, Docker? Have you filed an issue there? If so, can you point it out?

It's the app's responsibility to work correctly on platforms it supports. I patch the Go stdlib on Windows where needed.

ianlancetaylor

ianlancetaylor commented on Jul 7, 2020

@ianlancetaylor
Contributor

@ericwj I'm sorry that you are frustrated but this issue doesn't seem to give us a useful way forward.

I doubt this will make you feel any better but pretty much all the Windows path support was written and contributed by Windows experts who do not work at Google.

It seems to me that filepath.EvalSymlinks is well defined, and it is clearly useful on Unix systems. Perhaps programs that run on Windows shouldn't be calling it, but that doesn't mean that we should remove it.

CC @alexbrainman @mattn

changed the title [-]Consider obsoleting or no-op'ing `EvalSymLinks` and review all Go path handling on Windows[/-] [+]path/filepath: consider obsoleting or no-op'ing `EvalSymLinks` and review all Go path handling on Windows[/+] on Jul 7, 2020
added
NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.
on Jul 7, 2020
added this to the Unplanned milestone on Jul 7, 2020
ericwj

ericwj commented on Jul 7, 2020

@ericwj
Author

Are you saying Google isn't involved? They are at least reviewing every single change - or they appear to be.

It's the app's responsibility to work correctly on platforms it supports.

If you personally wrote a new language, I would agree. Imho Google has a larger responsibility. But hey, the fact the first thing Microsoft does contributing to chromium is to add features for people with disabilities speaks to the company culture at Google refusing to do anything of the sort for 12 whole years.

@ianlancetaylor I am not asking for the API to be removed if it is 'normal' to use that on *nix, although I think overruling an administrator is about always wrong by principle regardless of the operating system. But go knows whether you are on Windows or cross building for it and if it has infrastructure to give warnings or suggest fixes for this API on Windows then that would be step 1.

I didn't personally file issues with Docker, but the are legion links to be found in their issues and on Stack Overflow with people stumbling on error messages like unable to prepare context: unable to evaluate symlinks in context path and similar.

I did file issues with Microsoft about this Visual Studio behavior, but since these errors do not reside in actual Microsoft code they de-prioritized.

About Git I am honestly just unaware of the components that make up the system. Visual Studio Code works quite allright, but Visual Studio has some go somewhere in its Git extension or I don't know where such that these problems magically appear in recent versions of Visual Studio.

ianlancetaylor

ianlancetaylor commented on Jul 7, 2020

@ianlancetaylor
Contributor

Are you saying Google isn't involved? They are at least reviewing every single change - or they appear to be.

Google is a huge company. I'm not sure what it means to say that Google is involved. Certainly Google employees are heavily involved with the development of Go. But it's not accurate to say that Google employees are reviewing every single change. Go is an open source project. Many changes to Go are written, reviewed, and submitted by people who are not Google employees, without anybody employed by Google being involved at all.

networkimprov

networkimprov commented on Jul 7, 2020

@networkimprov

Well, Eric, since you learned Go, maybe you could fix some of those Docker issues (perhaps by replacing EvalSymlinks in their codebase), and then file an issue or two here about how Go could support them better :-)

But we need specific bug reports to make progress.

ericwj

ericwj commented on Jul 8, 2020

@ericwj
Author

Go is an open source project.

So is .NET. And .NET actually works even with Windows-isms on a long list of *nix variants, because Microsoft as primary steward of the ecosystem is taking its proper responsibility. The downside is they have so much tests that it is hard to find any actual code if you look for a definition of something in /dotnet/runtime say.

Look, let's talk solutions. I am fed up hitting brick walls every time someone chooses go over another language and then shoves it onto my system as part of some software that I use - that's why I'm here. I identified very concrete issues and I can probably write code from plain primitives that uses the Windows API properly to fill in go API surface -- I just can't realistically write it in go. Would I help writing plain C# implementing some of the broken API's in go?

I could perhaps even write tests that will show how to break go, but I'd have to do that mostly in PowerShell and C#.

ericwj

ericwj commented on Jul 8, 2020

@ericwj
Author

PowerShell to create .vhdx files and mount/unmount them - system administration stuff, which I don't think go can do today at all.

49 remaining items

alexbrainman

alexbrainman commented on Jul 11, 2020

@alexbrainman
Member

@alexbrainman I found a number of issues, some of which are mentioned in this thread.

Sorry, but I cannot find them. Perhaps this thread is too long. Why don't you create new issue for every bug you found? Please, do not ignore the issue template, but fill in as much details as you can.

Please, be aware that Go issue tracker is only used to track real bugs / problems, and submit change proposals.

Thank you.

Alex

ericwj

ericwj commented on Jul 11, 2020

@ericwj
Author

Yes just leave this one to be about no-op ing EvalSymLinks and nothing else.

Even the way to do that without breaking anyone is a separate discussion about alternatives and what they look like.

Perhaps this issue could be about changing documentation, or at least adding a few lines of text to make people aware of why this API could bite them on Windows and about expectations by Windows foaks that they shall not default to using it.

networkimprov

networkimprov commented on Jul 11, 2020

@networkimprov

You can just submit a CL with documentation fix clarifying what happens for certain paths.

BTW English folk is spelled like the German volk :-)

ericwj

ericwj commented on Jul 11, 2020

@ericwj
Author
ericwj

ericwj commented on Jul 12, 2020

@ericwj
Author

I think messed this up. This patch was supposed to be just against ericwj/go for a new pull request. Did it get added to #40095? How do I fix that?

networkimprov

networkimprov commented on Jul 12, 2020

@networkimprov

It was added to the CL, yes. Just revert. Then make a new branch, and cherrypick the docs commits to that, and file a new PR.

Also, it should describe EvalSymlinks problems on Windows, not make blanket statements.

ericwj

ericwj commented on Jul 12, 2020

@ericwj
Author

That is explained in the commit message, but I'm not sure that is visible. Just revert? How? Commit after getting the old file from history?

networkimprov

networkimprov commented on Jul 12, 2020

@networkimprov

git revert <commit> is a common op; it creates new commits. Alternatively make a new commit by hand that removes the docs change.

networkimprov

networkimprov commented on Jul 12, 2020

@networkimprov

Best to close this issue, and make a new one for the docs change. Should post the new docs there for discussion before filing PR/CL.

locked and limited conversation to collaborators on Jul 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.OS-Windows

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @mattn@networkimprov@ianlancetaylor@gopherbot@ericwj

        Issue actions

          path/filepath: no-op path.filepath.EvalSymLinks on Windows · Issue #40104 · golang/go