x/playground: support third-party imports

[bradfitz]

It's time for the playground to support importing third-party packages.

We have https://proxy.golang.org/ now which is the hard piece.

It might have to assume @latest for now, unless we let people write a go.mod file somehow (magic comments either one per module version, or magic comments separating the textarea into N logical files, ala mime/multipart by easier to type?)

/cc @bcmills @dmitshur @ysmolsky

[bradfitz]

e.g. @neild posted this snippet recently: https://play.golang.org/p/1rR97pPojOd ... it'd be nice if that actually worked.

[bradfitz]

Well, that was easy...

bradfitz@go:~/src/golang.org/x/playground$ git di
diff --git a/sandbox.go b/sandbox.go
index 7493213..bf7a421 100644
--- a/sandbox.go
+++ b/sandbox.go
@@ -325,7 +325,7 @@ func compileAndRun(req *request) (*response, error) {
        exe := filepath.Join(tmpDir, "a.out")
        goCache := filepath.Join(tmpDir, "gocache")
        cmd := exec.Command("go", "build", "-o", exe, in)
-       cmd.Env = []string{"GOOS=nacl", "GOARCH=amd64p32", "GOPATH=" + os.Getenv("GOPATH"), "GOCACHE=" + goCache}
+       cmd.Env = []string{"GOOS=nacl", "GOARCH=amd64p32", "GO111MODULE=on", "GOPROXY=https://proxy.golang.org", "GOPATH=" + os.Getenv("GOPATH"), "GOCACHE=" + goCache}
        if out, err := cmd.CombinedOutput(); err != nil {
                if _, ok := err.(*exec.ExitError); ok {
                        // Return compile errors to the user.

There's a minor complication with supporting the legacy code.google.com/p/go-tour/pic/* packages in module mode. We currently pre-install them in the Docker container's $GOPATH, but module mode doesn't use that.

[dmitshur]

Agreed this would be nice, and is easier now than before.

There are no additional security concerns with this feature request. It's always been possible to accomplish this in a round-about way: by re-writing the snippet such that all third-party imports are inlined into the main program (similar to what bundle does). If the snippet does too much and takes too long to run, it'll still time out, etc.

ala mime/multipart by easier to type?

txtar format comes to mind.

One can also imagine a more featureful UI that allows adding extra files. For example, https://gist.github.com/ has an "Add file" button that adds another text box.

Maybe we can just have the playground run gists. *flashbacks to 2016* 😱

There's also some overlap with https://github.com/dave/jsgo#how-it-works and perhaps some other playground-like projects (https://goplay.space/, etc.).

It might have to assume @latest for now

As long as we fix #31889, using @latest would at least return consistent results.

[bcmills]

One easy way to support non-latest versions would be to add comments to import statements:

import (
	"fmt"

	"gopkg.in/errgo.v2/errors"  // v2.1.0+incompatible
)

But that doesn't describe all of the transitive dependencies, and thus isn't particularly reproducible.
Might be better to write and store an explicit go.mod file; we could show it in a second column (perhaps initially hidden in a tray).

[bradfitz]

@bcmills, or how about this: if the user hits "play" or "share" on a package that uses a third-party import, we append the auto-generated go.mod to the end of the textarea buffer (using whatever magic delimiter syntax we come up with), so it's included in shares and reproducible.

e.g. user writes:

package main

import "github.com/davecgh/go-spew/spew"

func main() {
        spew.Dump(map[string]string{"foo": "bar"})
}

Then hits "Run" or "Share" and we first change it to:

package main

import "github.com/davecgh/go-spew/spew"

func main() {
        spew.Dump(map[string]string{"foo": "bar"})
}

# File: go.mod
module play
go 1.13
require github.com/davecgh/go-spew v1.1.1

[ysmolski]

I love the idea. And it really comes to this: support multiple files in playground. I like the idea of using delimeters per file (txtar). I really wanted to implement support for multiple files, but I hate the idea of adding a box per each file - it does not feel right for the current UI - I tried multiple options there.

I support the idea of having multiple files specified by txtar, also main.go does not have to be prefixed by any file marker.

[gopherbot]

Change https://golang.org/cl/176317 mentions this issue: playground: support third-party imports (off by default for now)

[bradfitz]

The basic functionality (without any of the txtar etc multi-file splitting) is now done in CL 176317. It's off by default for now with an env flag (ALLOW_PLAY_MODULE_DOWNLOADS) set to false in the Dockerfile. That gives us a quick knob to turn it off later.

We also want to do some cleaning probably, so the disk doesn't fill. We'll probably want to hold a RWMutex while cleaning to avoid #31948

[bradfitz]

Actually, CL 176317 now also cleans up after itself (any downloaded modules) after each build.

[bcmills]

I would still rather we use a separate textarea instead of tacking the files together in the same buffer.
I don't really care whether that textarea goes below the Go source file or beside it.

I find it really useful to be able to press ⌃a ⌃c and paste the result directly into cat, and having to trim the file becomes particularly annoying as the files get longer.

[bradfitz]

@bcmills, we can start with a single buffer because it's a ton easier. We can add UI later, once somebody has time for that.

I find it really useful to be able to press ⌃a ⌃c and paste the result directly into cat, and having to trim the file becomes particularly annoying as the files get longer.

a) maybe playground snippets should only be snippets :)
b) I'll write you a splitter CLI tool you can pipe into. :)

[rsc]

Yes, please lets start with a single txtar buffer. We can worry about fancier UI later.

[Sajmani]

A discussion around whether to support multiple files in the playground is in issue #3806.