Skip to content

proposal: path/filepath: add Resolve, replacing EvalSymlinks #37113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
bk2204 opened this issue Feb 7, 2020 · 78 comments
Closed

proposal: path/filepath: add Resolve, replacing EvalSymlinks #37113

bk2204 opened this issue Feb 7, 2020 · 78 comments

Comments

@bk2204
Copy link

bk2204 commented Feb 7, 2020

What version of Go are you using (go version)?

$ go version
go version go1.12.7 windows/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

Applies to all OSes

What did you do?

  1. Mounted a UNC path as a drive letter.
  2. In CMD, switched the current working directory to that drive.
  3. Called filepath.Abs on a relative path.
  4. Called filepath.EvalSymlinks on the result of that function.

What did you expect to see?

The same results as calling GetFinalPathNameByHandle: a UNC path.

What did you see instead?

A path using the drive letter instead of the UNC path.

Notes

This affects any attempt to canonicalize paths using the output of Git in such a situation. Git produces some paths as absolute and some paths as relative, and uses GetFinalPathNameByHandle for canonicalizing absolute paths. However, Go lacks a function to canonicalize paths in a standard way, so it isn't possible to produce results equivalent to a C program and still write code that works portably across systems.

Go should add a function that is explicitly defined to canonicalize paths in a way equivalent to the underlying operating system, since using filepath.Abs and filepath.EvalSymlinks doesn't work correctly on Windows. It does work fine on Unix, but Unix paths are much simpler and easier to reason about.

It was determined in #17084 that filepath.Abs and filepath.EvalSymlinks were sufficient in this case, but that doesn't appear to be true. I expect there are other cases in which those don't work on Windows, but I am insufficiently versed in Windows paths to know what those are.

This was originally reported to the Git LFS project in git-lfs/git-lfs#4012.

@dmitshur dmitshur changed the title Add a function to canonicalize paths path/filepath: add a function to canonicalize paths Feb 7, 2020
@dmitshur dmitshur added FeatureRequest Issues asking for a new feature that does not need a proposal. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 7, 2020
@dmitshur dmitshur added this to the Backlog milestone Feb 7, 2020
@dmitshur
Copy link
Member

dmitshur commented Feb 7, 2020

/cc @robpike @rsc per owners.

@alexbrainman
Copy link
Member

What did you expect to see?

The same results as calling GetFinalPathNameByHandle: a UNC path.

I did not try it, but, I suspect, UNC paths wouldn't work in some situations. For example, can you pass UNC path to os.Chdir?

Alex

@ianlancetaylor ianlancetaylor changed the title path/filepath: add a function to canonicalize paths proposal: path/filepath: add a function to canonicalize paths Feb 7, 2020
@bk2204
Copy link
Author

bk2204 commented Feb 7, 2020

I don't know for certain, but judging by a quick Google search, it appears to be possible in Ruby, so I assume one can do that in C-based languages.

I'm not a Windows developer, so I'm not a good person to ask about the capabilities of Windows. I'm just a Unix developer trying to make general-purpose software not be terrible on Windows.

@alexbrainman
Copy link
Member

I don't know for certain, but judging by a quick Google search, it appears to be possible in Ruby, so I assume one can do that in C-based languages.

You are correct. I was wrong. os.Chdir does work with UNC paths.

Alex

@bk2204
Copy link
Author

bk2204 commented Jun 25, 2020

It is also the case that filepath.EvalSymlinks fails to work when canonicalizing paths where there's a junction to a volume that lacks a drive letter (a OneDrive mount is a good example of this). For example, if C:\Users\User\OneDrive\Vault is a junction pointing to a OneDrive mount and we try to call filepath.EvalSymlinks("C:/Users/Users/OneDrive/Vault/home.git"), that will fail with readlink C:\Users\User\OneDrive\Vault: The system cannot find the path specified.

This also works with C-based programs.

@bk2204
Copy link
Author

bk2204 commented Sep 8, 2020

Hey,

Is there any interest in fixing this? Right now, there is no cross-platform way to canonicalize a path in Go. We keep running up against additional cases where the existing behavior doesn't canonicalize paths properly, leading to incompatibility with other programs on the system (notably Git). This necessarily limits the portability of using Go as a cross-platform language.

@networkimprov
Copy link

By "function to canonicalize paths" do you mean a variation of EvalSymlinks that works on Windows? If so, note that EvalSymlinks is not recommended: #40180 (and probably can't be fixed).

Go on Windows has a variety of long-standing filesystem bugs. I suggest using x/sys/windows to call the WinAPI if that solves your problem.

@bk2204
Copy link
Author

bk2204 commented Sep 8, 2020

I mean a function, when given a path, that returns a canonicalized version of that path. In other words, the equivalent to realpath(3) on Unix or GetFinalPathNameByHandle on Windows, and the equivalent to Rust's std::path::canonicalize.

It isn't helpful to me to call the Windows API because (a) I'm not a Windows programmer and have no clue how to use it, (b) it isn't cross-platform, and (c) this is a function that is generally provided by the standard library.

@dmitshur dmitshur modified the milestones: Backlog, Proposal Sep 8, 2020
@dmitshur dmitshur removed FeatureRequest Issues asking for a new feature that does not need a proposal. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Sep 8, 2020
@networkimprov
Copy link

Go has gaps on Windows; I plug them in my code. You've seen the interest this issue evoked :-p

What you need isn't hard. Create a file named yourpkg_windows.go, import "golang.org/x/sys/windows", define GetCanonicalPath() to call CreateFile("yourfile") (to get a handle) then GetFinalPathNameByHandle.

Create a file yourpkg_unix.go with a // +build directive for your unix platforms. Define GetCanonicalPath() with the solution for unix you already know.

@bk2204
Copy link
Author

bk2204 commented Sep 9, 2020

First of all, I appreciate that you're trying to help. However, I do feel firmly that this functionality should be in the standard library, since it is in almost every other language, and it is in POSIX. I don't want to carry a lot of platform-specific code in a program because it's difficult to maintain and test, especially when I don't typically develop on Windows.

If Go is known to have known defects on Windows, those should be promptly fixed or clearly documented. For many purposes, it's fine if code doesn't run or run well on Windows, but there are some cases where it does. The documentation should clearly and prominently list any limitations with using Go on Windows so that folks can make an informed decision. Last I checked, the filepath documentation didn't indicate such limitations, and hasn't for some time.

Normally, when I find a bug or missing feature, I would send a patch to implement that functionality. However, Go has a CLA, and I don't sign CLAs, so any patch I might submit wouldn't be accepted. If that changes, I'm happy to send a patch to implement this properly if nobody gets to it before me.

@ianlancetaylor
Copy link
Contributor

On Unix systems I think the proposed function is the same as filepath.EvalSymlinks.

@bk2204
Copy link
Author

bk2204 commented Sep 9, 2020

Yes, I believe that they are identical. filepath.EvalSymlinks is, as far as I'm aware, equivalent to realpath(3) on Unix and has the semantics I'm looking for.

@networkimprov
Copy link

networkimprov commented Sep 9, 2020

This proposal should probably also deprecate EvalSymlinks, which is seriously broken on Windows, see #40180 (comment)

@rsc
Copy link
Contributor

rsc commented Sep 16, 2020

What does "canonical" mean, precisely?

@bk2204
Copy link
Author

bk2204 commented Sep 16, 2020

If there are multiple ways to refer to a filename, the canonical path is the absolute filename which uses no indirections and uses the canonical case (that is, the path component as written to the file system) if the system permits case folding. On Unix, that's the one that contains no symlinks (and, on macOS, uses canonical case and composition). On Windows, there are many ways to have indirection in a path: symlinks, junctions, SUBST, etc. (I don't actually know all of the possible ways, since I almost never use Windows). The canonical form uses none of those indirections and uses the canonical case.

Another way to say this is that assuming no hardlinks exist, a file on Unix should have exactly one canonical name whose components are either directories or non-symlink, non-directory (but possibly special) files.

@rsc
Copy link
Contributor

rsc commented Sep 17, 2020

@networkimprov, you make assertions without being specific about them. I am confused about three of the things you've said related to this issue.

Thanks.

@networkimprov
Copy link

networkimprov commented Sep 17, 2020

I mentioned #40180 in #37113 (comment) to suggest that the issue author reconsider canonicalization of paths. I didn't link it again later, but it documents a long list of problems with EvalSymlinks on Windows (which I've now linked).

Re path length bugs, other instances of those have been left alone, see #21782 & #36375. And here's a list of Windows bugs that mention "filepath" https://github.com/golang/go/issues?q=is%3Aopen+is%3Aissue+label%3AOS-Windows+filepath

@alexbrainman
Copy link
Member

Based on the discussion above, adding Resolve with the semantics described seems like a likely accept.

Russ,

I disagree. I don't think we should add filepath.Resolve function that behaves like GetFinalPathNameByHandle Windows API.

If someone wants to use this API, we can add golang.org/x/sys/windows.GetFinalPathNameByHandle and they can use it there.

I still don't understand what proposed filepath.Resolve function will do. It will be even harder for others, who are just learning this package, to understand the difference between filepath.Resolve and filepath.EvaluateSymlinks. I would like to see documentation for that function before this proposal is even considered.

filepath.Resolve is not a good name (like utils). It tells you nothing about what the function does. To me it is a good indication that you yourself do not know what this function does.

And what problem will this new function solve? Is filepath.Resolve is supposed to be a replacement (good version) for filepath.EvaluateSymlinks? If yes, let's have someone actually try and replace filepath.EvaluateSymlinks in the current Go tree and see if it still works. If that does not work, then we need to think even harder why we need filepath.Resolve, and not just golang.org/x/sys/windows.GetFinalPathNameByHandle.

Thank you for consideration.

Alex

PS: Sorry I did not comment earlier. But I have been busy with other staff, and missed this thread altogether.

@networkimprov
Copy link

@alexbrainman all of the questions you raised have been addressed in the comments above. Re missing the thread, you last commented here 7 days ago.

@ianlancetaylor
Copy link
Contributor

@ericwj is unable to comment on this issue. As far as we can tell, this is a GitHub bug. It is not due to any intentional action by the Go team.

This comment is by @ericwj, sent via e-mail:

GetFinalPathNameByHandle does not canonicalize. I just ran it, trying all flags. The result is different depending on how I configure my system, depending on which path I use to access the file and depending on which flag I provide in each situation. This means canonicalization will break if the machine configuration changes, which can also happen during Git operations, and the result may change depending on which path is used to access the same file or directory.

The simplicity of EvalSymlinks is one of its problems. Any function string Foo(string) will have some of the same fundamental, unfixable problems that I documented for EvalSymlinks. Issue #40180 only proposes docs fixes because outright no-op'ing EvalSymlinks on Windows was refused in #40104, but I think the conclusion of the analysis in #40180 is that #40104 was refused because I did not argue the case effectively yet. And this proposal is just too similar, apart from perhaps thread-safety and eventually perhaps looking more often like it works.

Even suppose something like Resolve could be perfect, Go cannot handle the return value. The unfortunate truth is that Go can only handle paths with drive letters and ‘normal’ UNC paths in the form \\server\share (without prefix). I have evaluated about half of all path/filepath API’s and the prefixes GetFinalPathNameByHandle yields break about every one of them. You cannot depend on stripping the prefixes, either.

This proposal goes wrong the same way path/filepath has, which imho is thinking *nix can be ported unmodified to a fundamentally different operating system without proper attention, expertise involved, or testing. And for letting fundamentals slide for the benefit of a bit of simplicity, offering no alternative besides P/Invoke in cases where this makes things fall apart.

I suggest writing the fundamentals, especially well for Windows, testing and reviewing meticulously, while iterating towards an API suitable for the general public on top of those fundamentals, before proposing inclusion in any standard library. If the problem is just UNC mappings, much more appropriate WNet* functions exist to translate DOS device names. I think object identifiers are the more general solution for Git LFS, in favour of path strings. Wrapped in a suitable, portable abstraction to obtain and compare them.

@rasky
Copy link
Member

rasky commented Oct 20, 2020

@rasky, I see two problems:

  1. EvalSymlinks has Unix-specific behavior as far as returning absolute vs relative paths. If you have a symlink x->y and you EvalSymlinks("x"), you get "y", but if you have a symlink x->/tmp/y, you get "/tmp/y". This is correct enough for Unix but much more difficult on Windows, and on both systems sometimes you just want a canonical absolute path. Depending on the symlink is surprising, even on Unix. Note that realpath(3) always returns an absolute path. That's what Resolve will do too.

Yes, I agree that we cannot exactly reproduce this if we switch EvalSymlinks to GetFinalPathNameByHandle. Though I'm not fully sure that the clients of EvalSymlinks really do behave on this implementation detail (that is, unfortunately, documented). What if drop that sentence from the documentation, and just say that EvalSymlinks can either return a relative or an absolute path? Do you have an idea on whether clients really do rely on this specific behavior, even on Linux?

  1. On Windows, we defined "with a drive letter" as "absolute" but even that's not correct. Drive letters are not absolute in the same way as the Unix root. Instead, Windows has introduced \\ paths that are a better definition of absolute there.
    Concretely, supposing that M: and N: map to the same network drive (\\host\share\), then EvalSymlinks(`M:\`) = `M:\` and EvalSymlinks(`N:\`) , so that EvalSymlinks(`M:\`) != EvalSymlinks(`N:\`) even though they are actually the same.
    In contrast, in that case, Resolve(`M:\`) == Resolve(`N:\`) == `\\host\share` .

Notice that realpath on Linux doesn't follow mountpoints as well:

# mkdir a
# mkdir b
# mount --bind a b
# cd a
# touch foo
# cd ..
# realpath a/foo
/tmp/a/foo
# realpath b/foo
/tmp/b/foo

Is realpath(3) on Linux different than filepath.Abs(filepath.EvalSymlinks(path))? I might be missing something here.

In both cases, EvalSymlinks has behavior that is difficult to change but is less than ideal. Adding Resolve lets us introduce a different operation that behaves like realpath(3) and GetFinalPathNameByHandle instead of trying to shoehorn those into EvalSymlinks. EvalSymlinks still does exactly what it says - it evaluates symlinks - and people who want exactly that behavior can still use it for that. But people also sometimes want "get me a fully resolved, canonical equivalent of this path". It makes sense to provide that separately.

So my understanding is that if this proposal is accepted, we end up with:

  • Linux: Resolve and EvalSymlinks are basically the same function, with the only difference is that Resolve always returns an absolute path, while EvalSymlinks returns a path constructed following the symlinks, so it might be absolute or relative depending on how the symlinks were created. Neither of them see through mountpoints in any way.
  • Windows: EvalSymlinks (after bugfixing) would possibly go through all kind of NTFS links (with the same relative/absolute handling than Linux), and it would not follow mountpoints. Resolve instead would also resolve mountpoints.

It looks like the biggest gain here is that we somehow exposes GetFinalPathNameByHandle which is a "common" function used in other frameworks / languages to canonicalize paths. Since we're missing it, we face interoperability problems. In fact, semantically speaking, we're going to have a Resolve function which behaves differently between Windows and Linux wrt mountpoints. And BTW we're also getting into another corner: if somebody puts in the effort of adding mountpoint resolution to Resolve on Linux, we end up with interoperability problems again, because the rest of the world might still just be calling realpath(3) on Linux.

A slightly different proposal is to add filepath.SameFile(path1, path2 string). This would use GetFinalPathNameByHandle on Windows, and realpath on Linux (but can be further improved on Linux by looking at inode number, etc.). It would solve the real-world problem of many softwares without having to define what a canonical path name is, and would also relax the software engineering concern of preferring to keep around the original input path rather than a resolved one.

@bk2204
Copy link
Author

bk2204 commented Oct 21, 2020

So my understanding is that if this proposal is accepted, we end up with:

* Linux: `Resolve` and `EvalSymlinks` are basically the same function, with the only difference is that Resolve always returns an absolute path, while `EvalSymlinks` returns a path constructed following the symlinks, so it might be absolute or relative depending on how the symlinks were created. Neither of them see through mountpoints in any way.

Yes, because bind mounts on Linux are not identical. One directory may be read-only while the other may be read-write. Bind mounts also, by default, don't include submounts and are therefore not necessarily equivalent.

* Windows: `EvalSymlinks` (after bugfixing) would possibly go through all kind of NTFS links (with the same relative/absolute handling than Linux), and it would not follow mountpoints. `Resolve` instead would also resolve mountpoints.

As far as I understand it, these mountpoints are functionally identical. They are more like opaque symlinks rather than the typical Unix mountpoints with different mount options.

It looks like the biggest gain here is that we somehow exposes GetFinalPathNameByHandle which is a "common" function used in other frameworks / languages to canonicalize paths. Since we're missing it, we face interoperability problems. In fact, semantically speaking, we're going to have a Resolve function which behaves differently between Windows and Linux wrt mountpoints. And BTW we're also getting into another corner: if somebody puts in the effort of adding mountpoint resolution to Resolve on Linux, we end up with interoperability problems again, because the rest of the world might still just be calling realpath(3) on Linux.

This is exactly the behavior that's desired: canonicalizing the path in the way the operating system does that. Languages that use the system C library don't have this problem because they use the system C library functions for this purpose and implement those functions' behavior. I would literally define the expected behavior in terms of the system's C library, since that's what other languages (e.g., Rust) do.

A slightly different proposal is to add filepath.SameFile(path1, path2 string). This would use GetFinalPathNameByHandle on Windows, and realpath on Linux (but can be further improved on Linux by looking at inode number, etc.). It would solve the real-world problem of many softwares without having to define what a canonical path name is, and would also relax the software engineering concern of preferring to keep around the original input path rather than a resolved one.

That doesn't really assist in my use case of finding out whether one path is inside another.

@rasky
Copy link
Member

rasky commented Oct 21, 2020

Even suppose something like Resolve could be perfect, Go cannot handle the return value. The unfortunate truth is that Go can only handle paths with drive letters and ‘normal’ UNC paths in the form \server\share (without prefix). I have evaluated about half of all path/filepath API’s and the prefixes GetFinalPathNameByHandle yields break about every one of them. You cannot depend on stripping the prefixes, either.

This should mostly be fixed by my CL (https://go-review.googlesource.com/c/go/+/263538). Please test it and let me know.

It also rectifies some EvalSymlinks issues on Windows reported in your #40180 table. I have a followup CL (still not mailed) that fixes another set of issues in EvalSymlinks. NOTE: I'm not touching the semantic of the function as it is implemented now:
I'm just trying to fix implementation bugs so that we can see a clearer picture of where we are now.

@rsc
Copy link
Contributor

rsc commented Oct 21, 2020

There's clearly new, substantive discussion here. Moving back to Active.

@alexbrainman
Copy link
Member

all of the questions you raised have been addressed in the comments above. Re missing the thread, you last commented here 7 days ago.

@networkimprov I cannot find them. Can you, please, provide references to me? Thank you.

Alex

@rasky
Copy link
Member

rasky commented Oct 22, 2020

So my understanding is that if this proposal is accepted, we end up with:

* Linux: `Resolve` and `EvalSymlinks` are basically the same function, with the only difference is that Resolve always returns an absolute path, while `EvalSymlinks` returns a path constructed following the symlinks, so it might be absolute or relative depending on how the symlinks were created. Neither of them see through mountpoints in any way.

Yes, because bind mounts on Linux are not identical. One directory may be read-only while the other may be read-write. Bind mounts also, by default, don't include submounts and are therefore not necessarily equivalent.

To be honest, I disagree on this. The two underlying files are indeed the same and in fact os.SameFile returns true. It sounds absolutely surprising that os.SameFile(a,b) differs from filepath.Resolve(a) == filepath.Resolve(b).

Moreover, the same situation happens on Windows. You can easily have C:\FOO and \\LOCALHOST\BAR\FOO being the same file with different permissions. I'm not sure why on Windows it should be necessary to have filepath.Resolve returning true in that situation, while on Linux it should return false on bind mounts (or any other kind of mount really).

I think the only correct answer here is that the request is not to establish a cross-platform semantic, but just add a function that calls realpath on Linux and GetFinalPathNameByHandle on Windows because this is what most open source projects have ended up doing. I sympathize with the goal of interoperability, but I'm just hoping that we can find a better solution.

A slightly different proposal is to add filepath.SameFile(path1, path2 string). This would use GetFinalPathNameByHandle on Windows, and realpath on Linux (but can be further improved on Linux by looking at inode number, etc.). It would solve the real-world problem of many softwares without having to define what a canonical path name is, and would also relax the software engineering concern of preferring to keep around the original input path rather than a resolved one.

That doesn't really assist in my use case of finding out whether one path is inside another.

Looking at git-lfs/git-lfs#4012, it looked like filepath.SameFile would be enough, to compare if the git root actually matches. From your answer, I understand that you actually need whether a specific file of the repo is within the git repo (with the git root path being read through git and thus canonicalized through GetFinalPathNameByHandle)? Can you please elaborate on the use case you are mentioning?

I have one question on this. Say that we have \\network\repo mounted as N:, just like in the referenced bug. Within the repo, there's a committed symlink N:\mylink that points to a file on a local disk C:\file. Would you expect filepath.Resolve("N:\\mylink") to return \\network\repo\mylink or C:\file? If I followed you correctly, you would need the former, but GetFinalPathNameByHandle obviously returns the latter.

@bk2204
Copy link
Author

bk2204 commented Oct 22, 2020

I think the only correct answer here is that the request is not to establish a cross-platform semantic, but just add a function that calls realpath on Linux and GetFinalPathNameByHandle on Windows because this is what most open source projects have ended up doing. I sympathize with the goal of interoperability, but I'm just hoping that we can find a better solution.

This is what Git does and what other projects do, and it's the semantics we need. It is the same semantics that other programming languages provide. I am interested in Go working with other software that is also on the system in a compatible way.

You may argue that Windows and Unix should provide the same semantics for path canonicalization, and perhaps they should. However, Microsoft was fully aware of Unix path semantics and behavior, and clearly chose deliberately to do things in an incompatible and much more complicated way. I feel that was a mistake, but we're stuck with it now.

Looking at git-lfs/git-lfs#4012, it looked like filepath.SameFile would be enough, to compare if the git root actually matches. From your answer, I understand that you actually need whether a specific file of the repo is within the git repo (with the git root path being read through git and thus canonicalized through GetFinalPathNameByHandle)? Can you please elaborate on the use case you are mentioning?

I need to change the directory to the root of the repository to make GIT_WORK_TREE work. I also need to be able to resolve any other Git environment variables to their canonical paths so I can set them appropriately before invoking Git, and all of those need to be canonicalized appropriately and compatibly with Git. This is also the behavior that anyone else working with Git will need to implement.

I also need to be able to determine if a path the user has provided is within the repository and fail if it is not. I can do that by removing the trailing file component, canonicalizing the directory, and appending the file path. That works to tell me if the component is in the repository, whether or not the file itself is a symlink. Repositories are not allowed to span file systems.

I have one question on this. Say that we have \\network\repo mounted as N:, just like in the referenced bug. Within the repo, there's a committed symlink N:\mylink that points to a file on a local disk C:\file. Would you expect filepath.Resolve("N:\\mylink") to return \\network\repo\mylink or C:\file? If I followed you correctly, you would need the former, but GetFinalPathNameByHandle obviously returns the latter.

I don't need to canonicalize file names, only directory names. However, I do need the standard GetFinalPathNameByHandle semantics for that. Other people may have different use cases and will also need the standard semantics.

I want to be clear that I'm not asking for this just for Git LFS. I'm asking for this because these are the standard platform semantics and they should be available in a cross-platform way. I fully realize that they are inconsistent across platforms. I want the same behavior out of Go that I get out of languages that use the system C library. I'm not looking for special functionality to handle my special case, I'm looking for generic tools that make Go programs compatible with non-Go programs.

It's fine that Go does not wrap the system C library, but if it chooses not to do that, it needs to be compatible with the semantics of the platform as if it did, and not provide behavior that is materially different. Trying to sell me on a behavior that is consistent across platforms when that is not practically achievable in a useful way isn't helpful. That's what we have now with EvalSymlinks, and everyone agrees that it's not the right decision.

@bk2204
Copy link
Author

bk2204 commented Oct 23, 2020

I want to be clear that while there may be other proposals, such as filepath.SameFile, those are separate and independent proposals and are outside of the scope of this proposal, which is to implement a function which canonicalizes paths. I don't dispute that they may be valuable and important proposals, but they are not this one, and as such, they should probably be discussed in their own issues, not this one.

I also want to be clear that I'm an experienced developer who understands his problem space well and am confident I know what's best for it, so it's unhelpful (and frankly, insulting) to imply that some other solution would be better for my needs or that I should adopt a different design. If people disagree with the Git project's decisions to adopt this technique (with which I am implementing compatibility) and feel that they were unwise, those concerns should be addressed to the project's mailing list, although I suspect that in this case such a message will be about as welcome as I have found it.

As I've mentioned, my goal here is to expose a cross-platform way to canonicalize paths as the operating system defines that. Whether the operating system designers made a provident design decision or whether the operating system has provided desirable semantics is outside the scope of this proposal. I assume for the purposes of this proposal that the fact that the operating system is successful means that its developers were competent and implemented desirable, useful, and well-thought-out features. The facts remain that other common, successful languages implement functionally equivalent designs, and that while similar concerns have been expressed during the design of those implementations, nobody has yet proposed a better or more universally desirable solution. Moreover, Go attempted to provide a solution for this problem in EvalSymlinks which demonstrates that this is a problem that people need solved. Additionally, canonicalizing paths is considered useful for security purposes by Carnegie Mellon's Software Engineering Institute, which is considered a reputable source of secure coding advice. Finally, this functionality is implemented in the popular realpath utility, which is widely shipped for scripting across platforms (including Windows).

For these reasons, I believe that the proposal as it stands should be implemented. I take no position on proposals for other functions, as they are outside my scope.

@networkimprov
Copy link

Brian, the Windows maintainer has objected to this proposal, raising Q's in #37113 (comment). It might help to point out where Russ or Ian have previously addressed those Q's.

@saracen
Copy link
Contributor

saracen commented Oct 23, 2020

Looking at how this problem is solved elsewhere, if a new function is to be added, perhaps a better name would be Canonical(). Rust has canonicalize and C++'s filesystem library has canonical, which Microsoft's Standard C++ Library implements using GetFinalPathNameByHandle.

Canonical was not immediately obvious to me before I looked at this issue, and I think that's a fair concern. However, looking at modern solutions to this problem, once you know of it, the naming familiarity between languages might be a good thing.

@ericwj is unable to comment on this issue. As far as we can tell, this is a GitHub bug. It is not due to any intentional action by the Go team.

@ianlancetaylor The original author of this issue has blocked @ericwj because they felt insulted by the help they were providing. When you block a user on GitHub, they're unable to comment on issues/pull requests you're the owner of.

@ianlancetaylor
Copy link
Contributor

@saracen Thanks for the explanation.

@bk2204 Would you be willing to unblock @ericwj? I think they have useful information to contribute for moving this proposal forward. Thanks.

@bk2204
Copy link
Author

bk2204 commented Oct 24, 2020

I do believe people have the right to decide who they'd like to block, assuming they are not in a government position, and that asking people to unblock others is inappropriate. That's often an approach people use to engage in continued harassment of women and other folks who tend to be underrepresented in tech. That's not what's happening here, and I'm sure you had only good intent to move the discussion forward in a productive way, but since we often lack all the context to know people's reasons for blocking others, it's generally best not to ask.

For the limited purpose of advancing this proposal, I have unblocked @ericwj for the moment.

I'm fine with Canonical() or Canonicalize() as a name since it's used elsewhere. That may be more intuitive than Resolve(), but since we'll have sane documentation either way, I'm not very picky about the name.

@golang golang deleted a comment from ericwj Oct 24, 2020
@ianlancetaylor
Copy link
Contributor

@bk2204 My sincere apologies. I was in the wrong, and I've done you harm. I'm sorry.

@ericwj Your comment violates the Go Community Code of Conduct (https://golang.org/conduct), and I've deleted it.

@alexbrainman
Copy link
Member

the Windows maintainer has objected to this proposa

@networkimprov I assume you refer to me as the Windows maintainer. That is not my title. Please stop inventing things. You give false impression to others.

Thank you.

Alex

@networkimprov
Copy link

Alex, sorry, I had gathered that you're recognized as the principle Windows maintainer. I didn't mean to invent a title.

@ericwj
Copy link

ericwj commented Oct 24, 2020

Well, I'm sorry for having been slightly edgy due to the total irrelevance of that statement with which he accompanies the news that he unblocked me.

I care to repeat that Brian should argue his case from a position of hands on, mature knowledge rather than having us collect that for him and demanding his proposal be accepted unmodified and that none of us is in a better position than to test and evaluate the suitability of including the proposal of porting canonicalize unmodified into the go standard library for Windows than he himself and the company he works for. Further I can only repeat my previous arguments and fix the oversight of not mentioning the obvious - that EvalSymlinks is implemented with GetFinalPathNameByHandle and that some of the problems even mentioned in this issue will be exactly the same with canonicalize for this reason.

@bk2204
Copy link
Author

bk2204 commented Oct 24, 2020

I'm going to close this issue and I'd like to ask the core team to lock it at their earliest convenience. I don't feel this discussion is going in a productive way and I don't wish to pursue this issue or proposal further. If other folks think this a similar proposal will be valuable, I'm happy for them to drive it without my involvement.

@bk2204 bk2204 closed this as completed Oct 24, 2020
@alexbrainman
Copy link
Member

Alex, sorry,

No worries. You did not hurt my feelings or anything.

I just did not want other people to get impression that they need to convince me more than others. Any proposal here should be judged purely on its merits, not on authority of proposal author or judges.

I had gathered that you're recognized as the principle Windows maintainer.

I am Go contributor, like many others. I gathered bunch of experience over the years. But that does not make me principle Windows maintainer. I am not responsible for Windows port - Go Team is.

I didn't mean to invent a title.

No worries. No harm done.

Alex

@ianlancetaylor
Copy link
Contributor

I have written a new proposal, #42201, to replace this one.

I will now lock this issue.

@golang golang locked as too heated and limited conversation to collaborators Oct 25, 2020
@rsc rsc moved this to Declined in Proposals Aug 10, 2022
@rsc rsc added this to Proposals Aug 10, 2022
@rsc rsc removed this from Proposals Oct 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

10 participants