cmd/go: document that gofiles on command line must be in same directory

[dmitshur]

What version of Go are you using (go version)?

go version go1.9rc2 darwin/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/Dmitri/Dropbox/Work/2013/GoLanding:/Users/Dmitri/Dropbox/Work/2013/GoLand"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/tw/kgz4v2kn4n7d7ryg5k_z3dk40000gn/T/go-build182434083=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

I read https://golang.org/cmd/go/#hdr-Compile_and_run_Go_program, it said:

Usage:

go run [build flags] [-exec xprog] gofiles... [arguments...]

Run compiles and runs the main package comprising the named Go source files. A Go source file is defined to be a file ending in a literal ".go" suffix.

[more details that are not relevant]

Notice there are no restrictions on the prefix of the file, what directory it's in, or what build constraints the file has.

Then I ran:

mkdir -p $GOPATH/src/new/package
cd $GOPATH/src/new/package
echo 'package main; import "fmt"; func main() { fmt.Println("go") }' > main.go
go run main.go
mv main.go 123.go
go run 123.go
mv 123.go _underscore.go
go run _underscore.go
mv _underscore.go .dot.go
go run .dot.go
mv .dot.go thisis.notgo
go run thisis.notgo
rm thisis.notgo
go run

(Credit goes to @natefinch for discovering this at https://twitter.com/NateTheFinch/status/898585298111787009.)

What did you expect to see?

go
go
go
go
go run: no go files listed
go run: no go files listed

What did you see instead?

go
go
package main: no Go files in /Users/Gopher/go/src/new/package
package main: no Go files in /Users/Gopher/go/src/new/package
go run: no go files listed
go run: no go files listed

It looks like the .go file starting with _ or . gets ignored, probably related to the logic go/build has to ignore files beginning with _ and . in normal Go packages.

Edit: Upon further reading, I think this section from https://golang.org/cmd/go/#hdr-Description_of_package_lists applies and explains the behavior:

As a special case, if the package list is a list of .go files from a single directory, the command is applied to a single synthesized package made up of exactly those files, ignoring any build constraints in those files and ignoring any other files in the directory.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

[davecheney]

I can see the inconsistency and i don't think it should be fixed. I want go developers to stop leaning on go run, and if this (unlikely, see below) situation encourages them to do so, all the better in my mind. I'm guessing this occurred because the OP wanted to hide some utility program, possibly a generator, from go build/install. If that's the case then build tags are the better choice in my opinion.

…

On Sat, 19 Aug 2017, 10:17 Dmitri Shuralyov ***@***.***> wrote: What version of Go are you using (go version)? go version go1.9rc2 darwin/amd64 What operating system and processor architecture are you using (go env)? GOARCH="amd64" GOBIN="" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOOS="darwin" GOPATH="/Users/Dmitri/Dropbox/Work/2013/GoLanding:/Users/Dmitri/Dropbox/Work/2013/GoLand" GORACE="" GOROOT="/usr/local/go" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GCCGO="gccgo" CC="clang" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/tw/kgz4v2kn4n7d7ryg5k_z3dk40000gn/T/go-build182434083=/tmp/go-build -gno-record-gcc-switches -fno-common" CXX="clang++" CGO_ENABLED="1" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" What did you do? I read https://golang.org/cmd/go/#hdr-Compile_and_run_Go_program, it said: Usage: go run [build flags] [-exec xprog] gofiles... [arguments...] Run compiles and runs the main package comprising the named Go source files. *A Go source file is defined to be a file ending in a literal ".go" suffix.* [more details that are not relevant] Notice there are no restrictions on the prefix of the file, what directory it's in, or what build constraints the file has. Then I ran: mkdir -p $GOPATH/src/new/packagecd $GOPATH/src/new/packageecho 'package main; import "fmt"; func main() { fmt.Println("go") }' > main.go go run main.go mv main.go 123.go go run 123.go mv 123.go _underscore.go go run _underscore.go mv _underscore.go .dot.go go run .dot.go mv .dot.go thisis.notgo go run thisis.notgo rm thisis.notgo go run (Credit goes to @natefinch <https://github.com/natefinch> for discovering this at https://twitter.com/NateTheFinch/status/898585298111787009.) What did you expect to see? go go go go go run: no go files listed go run: no go files listed What did you see instead? go go package main: no Go files in /Users/Gopher/go/src/new/package package main: no Go files in /Users/Gopher/go/src/new/package go run: no go files listed go run: no go files listed It looks like the .go file starting with _ or . gets incorrectly ignored, probably related to the logic go/build has to ignore files beginning with _ and . in normal Go packages. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#21529>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA8BVw0Gnfuma8YbwXynklUEktcY2ks5sZimMgaJpZM4O8KYJ> .

[kashav]

From https://golang.org/cmd/go/#hdr-Description_of_package_lists:

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

[natefinch]

@davecheney I know you have a personal dislike for go run, but it is an incredibly useful command, and it is a part of the go toolchain that isn't going away (afaik), and thus we should try to make it work in the best way possible. At the very least it should not be misleading and frustrating.

The message should be made consistent with the rest of the examples (and perhaps docs updated). But ideally, I'd like it to just run the files I give it if they're .go files.

[dmitshur]

I tried with go1.8.3. The error message is slightly different, but it's the same idea:

package main: no buildable Go source files in /Users/Gopher/go/src/new/package

I want go developers to stop leaning on go run,

Me too, but we shouldn't conflate bugs/inconsistencies with our desires to demote some feature that Go officially supports.

@kshvmdn, I am extremely familiar with that, but my understanding was that it applied for normal Go packages specified by import paths only. go run operates on .go files provided to it, and does not support Go packages specified via import paths.

If you look at the topic of the section where what you quoted is, it starts with:

Many commands apply to a set of packages:

go action [packages]

Usually, [packages] is a list of import paths.

And what follows sounds like it only apples to those commands.

However, upon closer look, I noticed the paragraph preceding what you quoted:

As a special case, if the package list is a list of .go files from a single directory, the command is applied to a single synthesized package made up of exactly those files, ignoring any build constraints in those files and ignoring any other files in the directory.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

I think that means the current go run behavior is correct/as expected.

What we need to consider next is whether the documentation and the error message text is optimal. I'm going to look into it more before I make further comments.

[dmitshur]

I've confirmed that go run has the same behavior as described in that section regarding .go files being in the same directory. If they are not, you get this error:

$ touch another.go
$ open another.go 
$ go run main.go another.go 
another
go
$ mkdir dir
$ mv another.go dir/another.go
$ go run main.go dir/another.go
named files must all be in one directory; have ./ and dir/
$ go build main.go dir/another.go
named files must all be in one directory; have ./ and dir/
$ go install main.go dir/another.go
named files must all be in one directory; have ./ and dir/

This is definitely not a behavior bug. However, I suspect the current go run documentation could be improved. Its wording seems to imply it has its own behavior, but it turns out that it works with ad-hoc packages no different than go build and go install.

[davecheney]

My point was, is there another way to solve the problem you had without making go run even more special? I'm not arguing that this isn't a bug. What I am I suggesting is rather than the easy option--remove the prohibition on . and _--we should do the harder thing, encourage people to use build tags to exclude source files in directories, not os specific (cute) hacks like . and _. I think the latter will have a more profound experience on the developer experience.

…

On Sat, 19 Aug 2017, 10:31 Nate Finch ***@***.***> wrote: @davecheney <https://github.com/davecheney> I know you have a personal dislike for go run, but it is an incredibly useful command, and it is a part of the go toolchain that isn't going away (afaik), and thus we should try to make it work in the best way possible. At the very least it should not be misleading and frustrating. The message should be made consistent with the rest of the examples (and perhaps docs updated). But ideally, I'd like it to just run the files I give it if they're .go files. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21529 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA5X7kAgFLuwMouizBzHIiskBZwjoks5sZizqgaJpZM4O8KYJ> .

[dmitshur]

@davecheney Did you see my comments above? I found out that this is not a bug, there is an explanation for the current go run behavior. It behaves identically as go build and go install (and go test) do on ad-hoc packages (packages specified as a list of .go files in a single directory).

The only potential issue is that the documentation of this behavior is far away from the docs of go run command, and the phrasing in go run command makes it sound like go run behavior is unique, when in reality it isn't.

is there another way to solve the problem you had without making go run even more special?

I'm not sure which "you" you're referring to. If it's me, I don't have a problem that I was solving with go run and I'm not suggesting we change it's behavior to make it special. I just want to see if it's possible to improve the docs so its current correct behavior is more clear.

[davecheney]

I was replying to Nate.

…

On Sat, 19 Aug 2017, 11:48 Dmitri Shuralyov ***@***.***> wrote: @davecheney <https://github.com/davecheney> Did you see my comments above? I found out that this is not a bug, there is an explanation for the current go run behavior. It behaves identically as go build and go install (and go test) do on ad-hoc packages (packages specified as a list of .go files in a single directory). The only potential issue is that the documentation of this behavior is far away from the docs of go run command, and the phrasing in go run command makes it sound like go run behavior is unique, when in reality it isn't. is there another way to solve the problem you had without making go run even more special? I'm not sure which "you" you're referring to. If it's me, I don't have a problem that I was solving with go run and I'm not suggesting we change it's behavior to make it special. I just want to see if it's possible to improve the docs so its current correct behavior is more clear. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21529 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcAzPgLmLuYq9Up1pHZnIxtU49Kuw6ks5sZj7ngaJpZM4O8KYJ> .

[natefinch]

I'd like the error to be more clear. Saying there's no go files in the directory is confusing, since the tool isn't running on a directory. Fwiw I wasn't intentionally using underscore to make the file get missed by go build, it was a generated file that just happened to start with an underscore for other reasons... But the error message was so confusing I couldn't tell what was wrong, and since go run ignores build constraints, I assumed it would ignore the file name restrictions as well.

…

On Fri, Aug 18, 2017, 9:49 PM Dmitri Shuralyov ***@***.***> wrote: @davecheney <https://github.com/davecheney> Did you see my comments above? I found out that this is not a bug, there is an explanation for the current go run behavior. It behaves identically as go build and go install (and go test) do on ad-hoc packages (packages specified as a list of .go files in a single directory). The only potential issue is that the documentation of this behavior is far away from the docs of go run command, and the phrasing in go run command makes it sound like go run behavior is unique, when in reality it isn't. is there another way to solve the problem you had without making go run even more special? I'm not sure which "you" you're referring to. If it's me, I don't have a problem that I was solving with go run and I'm not suggesting we change it's behavior to make it special. I just want to see if it's possible to improve the docs so its current correct behavior is more clear. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21529 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADCcyA5kQcCKvgbUgvJEafYM8CWUiM5lks5sZj8TgaJpZM4O8KYJ> .

[cznic]

run command skips .go files starting with _ or . characters

IINM, this has nothing to do with the 'run' command. dotfiles and _* are ignored by the whole toolchain. I believe it is already documented somewhere, but I was not able to now find where.

[dmitshur]

There are 2 separate changes I want to consider that I think will resolve this issue. The first one is improving cmd/go docs, the second is improving the error message when doing go (run|build|install|test) _underscore.go (if it's viable to do so cleanly).

The main one is making thecmd/go documentation more accurate about what go run does. When someone reads the following part of the Description of package lists section:

As a special case, if the package list is a list of .go files from a single directory, the command is applied to a single synthesized package made up of exactly those files, ignoring any build constraints in those files and ignoring any other files in the directory.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

I want users to be able to understand that applies to go run and it's precisely the only input go run accepts.

Currently, go run docs say something that imply that it has similar but different behavior:

Run compiles and runs the main package comprising the named Go source files. A Go source file is defined to be a file ending in a literal ".go" suffix.

Here's a first stab/draft at a more accurate description of go run behavior, but also one that should hopefully communicate that it behaves the same as go build, go install, etc., do on a list of .go files.

Run compiles and runs a single synthesized main package comprising the named
.go files from a single directory, ignoring any build constraints
in those files and ignoring any other files in the directory.
File names that begin with "." or "_" are also ignored.

That seems accurate and easy to recognize the similarity, but I don't like the first sentence. It jumps into details and doesn't do a great job communicating the overview of what the command does. The original is much better:

Run compiles and runs the main package comprising the named Go source files.

Maybe there's a way to keep that as is, but modify the rest of the text to follow... Suggestions appreciated.

[rsc]

See also #21045. It may be a bug to be skipping those files if named explicitly.

[dmitshur]

I think CL 109341 (for issue #22726) has helped this issue somewhat (at least, it implemented a part of what I described in my comment from Aug 20 above). /cc @rsc

[rsc]

If you run go run (or build or test or ...) with a list of .go file names, it should use that list. That's #21045.

This issue can be to document that the file names must all be in the same directory.

[gopherbot]

Change https://golang.org/cl/149797 mentions this issue: cmd/go/internal/{run,work}: document file path location requirement