path/filepath: Clean() incorrect trunc for volume-guid paths on Windows

[lowenna]

What version of Go are you using (go version)?

$ go version
go version go1.12rc1 windows/amd64

Does this issue reproduce with the latest release?

Yes, and is a change in behaviour from 1.11.x

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

PS E:\docker\build\golang1.12> go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Administrator\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=e:\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\ADMINI~1\AppData\Local\Temp\1\go-build584421990=/tmp/go-build -gno-record-gcc-switches
PS E:\docker\build\golang1.12>

@alexbrainman

This is the root cause of failures on Windows for upgrading github.com/moby/moby to golang 1.12. (See moby/moby#38404).

The cause appears to be the fix for #27791: d1f7470

In the MSDN documentation for "volume-guid"-style naming, Windows should have a trailing backslash (https://docs.microsoft.com/en-us/windows/desktop/FileIO/naming-a-volume). The specific quote is

To solve this problem, the operating system uses volume GUID paths to identify volumes. These are strings of this form:
"\\?\Volume{GUID}\"
where GUID is a globally unique identifier (GUID) that identifies the volume.

To simplify demonstration of the issue, run this test code on golang 1.11.5 and 1.12RC1 (or 1.12Beta)

package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	p := `\\?\Volume{7650f9bc-33f8-11e9-9bf2-e06fe09706b2}\`
	fmt.Println(filepath.Clean(p))
}

On go 1.11.5

E:\docker\build\golang1.12> .\golang1.12.exe
\\?\Volume{7650f9bc-33f8-11e9-9bf2-e06fe09706b2}\ <nil>
E:\docker\build\golang1.12>

On go 1.12RC1

PS E:\docker\build\golang1.12> .\golang1.12.exe
\\?\Volume{7650f9bc-33f8-11e9-9bf2-e06fe09706b2} <nil>
PS E:\docker\build\golang1.12>

Specifically, the trailing backslash is now omitted making an invalid filepath following the MSDN documentation.

[thaJeztah]

@bcmills @mikioh can this be added to the 1.12 milestone (as it caused a regression)?

[mikioh]

I added the requested label and NeedsInvestigation temporarily.
@ianlancetaylor, can you please triage this issue?

[ianlancetaylor]

CC @QtRoS @alexbrainman

This does seem to have been introduced by https://golang.org/cl/137055 for #27991. I don't understand the fix. The issue says that Clean(\\somepath\dir\) should return \\somepath\dir, which seems reasonable although I don't know if it's actually correct. And I'm not sure what to make of the MSDN docs, which use a trailing slash for everything.

Given that this broke something, we should probably just roll it back and consider trying again for 1.13.

[gopherbot]

Change https://golang.org/cl/163077 mentions this issue: path/filepath: revert "fix Windows-specific Clean bug"

[ianlancetaylor]

I sent https://golang.org/cl/163077 but I don't really know if it's right. It is, at least, likely to be safe.

[lowenna]

Thanks @ianlancetaylor. I'm happy to try and figure out the right fix in conjunction with yourself and @alexbrainman for 1.13 if you would like. However, at RC1 of 1.12, I tend to agree that reverting the previous fix is probably the safest bet at this stage rather than trying to put a potentially faulty fix in.

I believe you are correct that a "traditional" UNC-style path Clean("\\server\share\") should return "\\server\share" based on the description of filepath.Clean(). Specifically Clean returns the shortest path name equivalent to path...

For volume-guid paths, the rule is, somewhat annoyingly, but hey - I didn't create this API!, slightly different. It stems from this sentence in https://docs.microsoft.com/en-us/windows/desktop/FileIO/naming-a-volume: All volume and mounted folder functions that take a volume GUID path as an input parameter require the trailing backslash. In other words, (excluding the CreateFile API exception), a volume-guid style path requires the trailing backslash when there's nothing following it.

So I believe (although haven't done full validation of this), the following should be the expected behaviour when calling Clean on volume-GUID style paths to get the shortest (valid) path name:

• \\?\Volume{GUID}\ --> \\?\Volume{GUID}\ (backslash remains otherwise invalid)
• \\?\Volume{GUID}\folder\ --> \\?\Volume{GUID}\folder (backslash stripped)*
• \\?\Volume{GUID}\folder\..\folder2\file.txt --> \\?\Volume{GUID}\folder2\file.txt

The one marked * is where I'm not entirely sure as there's some ambiguity reading the documentation, but is certainly what golang 1.11 does, and does not appear to have any issues I am aware of. The way I read the docs is that it's referring to when it's "just" a volume-guid style path, without any further relative paths added - ie just the \\?\Volume{GUID}\, not \\?\Volume{GUID}\something\else\here.txt

[gopherbot]

Change https://golang.org/cl/163078 mentions this issue: [release-branch.go1.12] path/filepath: revert "fix Windows-specific Clean bug"