Skip to content

go mod doesn't work for github.com/gomarkdown/markdown/html #27565

New issue
New issue
Closed
Closed
go mod doesn't work for github.com/gomarkdown/markdown/html#27565
Labels
FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.WaitingForInfoIssue is not actionable because of missing required information, which needs to be provided.modules
Milestone
 
Go1.12
@kjk

Description

@kjk
kjk
opened on Sep 8, 2018 · edited by kjk

What version of Go are you using (go version)?

$ go version
go version go1.11 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

OS X 10.13.6

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/kjk/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kjk/src/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/2k/p2_4052s70vd5_cfdm0k0l740000gn/T/go-build479826685=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

  • make sure you're OUTSIDE OF GOPATH
  • git clone https://github.com/kjk/modtest.git
  • cd modtest
  • go mod init github.com/kjk/modtest
  • go build

What did you expect to see?

This should build.

What did you see instead?

$ go build
go: finding github.com/gomarkdown/markdown/ast latest
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
main.go:8:2: unknown import path "github.com/gomarkdown/markdown/html": cannot find module providing package github.com/gomarkdown/markdown/html

For some reason go mod has issue with github.com/gomarkdown/markdown/html even though it works for github.com/gomarkdown/markdown and github.com/gomarkdown/markdown/ast.

The source of the problem seems to be:

$ go get -v github.com/gomarkdown/markdown/html
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
Fetching https://github.com?go-get=1
Parsing meta tags from https://github.com?go-get=1 (status code 200)
go get github.com/gomarkdown/markdown/html: no matching versions for query "latest"

There is no problem when running same go get when in GOPATH.

Activity

josephlr
added
modules
on Sep 8, 2018
josephlr

josephlr commented on Sep 8, 2018

@josephlr
josephlr
on Sep 8, 2018
Member

@kjk I just looked into this, I was able to get your example working. I had to make a tiny modification as htmlHighlight is not defined, but after commenting out that line, I was able to run:

> git clone "https://github.com/kjk/modtest.git"
> cd modtest
> go mod init # To create a blank go.mod
> go mod tidy # To fill in the contents of go.mod
> go build
> ./modtest
Hello
josephlr

josephlr commented on Sep 8, 2018

@josephlr
josephlr
on Sep 8, 2018
Member

Full output:

➜ git clone https://github.com/kjk/modtest.git
Cloning into 'modtest'...
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 7 (delta 0), reused 4 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), done.
➜ cd modtest
➜ go build
main.go:7:2: cannot find package "github.com/gomarkdown/markdown/ast" in any of:
        /snap/go/2635/src/github.com/gomarkdown/markdown/ast (from $GOROOT)
        /home/joe/dev/go/src/github.com/gomarkdown/markdown/ast (from $GOPATH)
main.go:8:2: cannot find package "github.com/gomarkdown/markdown/html" in any of:
        /snap/go/2635/src/github.com/gomarkdown/markdown/html (from $GOROOT)
        /home/joe/dev/go/src/github.com/gomarkdown/markdown/html (from $GOPATH)
➜ vim main.go
➜ go mod init
go: creating new go.mod: module github.com/kjk/modtest
➜ go build
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown/ast latest
go: finding github.com/gomarkdown/markdown latest
go: downloading github.com/gomarkdown/markdown v0.0.0-20180907191918-6fda95a9e93f
➜ ./modtest
Hello
kjk

kjk commented on Sep 8, 2018

@kjk
kjk
on Sep 8, 2018 · edited by kjk
Author

@josephlr I fixed the compilation error in the code.

However, I believe you did this inside GOPATH (as indicated by (from $GOROOT) messages).

As I've indicated, you have to checkout the code outside of GOPATH.

Try repro steps after e.g. cd $HOME.

josephlr

josephlr commented on Sep 8, 2018

@josephlr
josephlr
on Sep 8, 2018
Member

@kjk Nope this was done outside of my GOPATH. The go build errors were just what happens if no go.mod file is present.

josephlr

josephlr commented on Sep 8, 2018

@josephlr
josephlr
on Sep 8, 2018
Member

Just tried this on darwin/amd64, got the same result. go mod init && go build && ./modtest produced no errors for me (building outside of GOPATH).

kjk

kjk commented on Sep 8, 2018

@kjk
kjk
on Sep 8, 2018
Author

@josephlr Are you using go 1.11 or master? Did you set GO11MODULE env variable to something? (I didn't so it's auto).

I just double-checked and am still getting an error:

Krzysztofs-MacBook-Pro:src kjk$ git clone https://github.com/kjk/modtest.git
Cloning into 'modtest'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 10 (delta 1), reused 7 (delta 1), pack-reused 0
Unpacking objects: 100% (10/10), done.
Krzysztofs-MacBook-Pro:src kjk$ cd modtest
Krzysztofs-MacBook-Pro:modtest kjk$ go mod init
go: creating new go.mod: module github.com/kjk/modtest
Krzysztofs-MacBook-Pro:modtest kjk$ go build
go: finding github.com/gomarkdown/markdown/ast latest
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
main.go:8:2: unknown import path "github.com/gomarkdown/markdown/html": cannot find module providing package github.com/gomarkdown/markdown/html
Krzysztofs-MacBook-Pro:modtest kjk$ go mod tidy
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
Krzysztofs-MacBook-Pro:modtest kjk$ go build
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
main.go:8:2: unknown import path "github.com/gomarkdown/markdown/html": cannot find module providing package github.com/gomarkdown/markdown/html
Krzysztofs-MacBook-Pro:modtest kjk$ go get -v github.com/gomarkdown/markdown/html
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
Fetching https://github.com?go-get=1
Parsing meta tags from https://github.com?go-get=1 (status code 200)
go get github.com/gomarkdown/markdown/html: no matching versions for query "latest"
kjk

kjk commented on Sep 8, 2018

@kjk
kjk
on Sep 8, 2018
Author

Also, what does go get -v github.com/gomarkdown/markdown/html do?

$ go get -v github.com/gomarkdown/markdown/ast
go: finding github.com/gomarkdown/markdown/ast latest
go: finding github.com/gomarkdown/markdown latest

$ go get -v github.com/gomarkdown/markdown/html
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
Fetching https://github.com?go-get=1
Parsing meta tags from https://github.com?go-get=1 (status code 200)
go get github.com/gomarkdown/markdown/html: no matching versions for query "latest"

https://github.com?go-get=1 is suspicious and only happens for html sub-package, not for ast sub-package.

kjk

kjk commented on Sep 8, 2018

@kjk
kjk
on Sep 8, 2018 · edited by kjk
Author

And just to be sure, do GO111MODULE=on go get -u -v github.com/gomarkdown/markdown/html.

It works with GO111MODULE=off and fails with on or auto if not in GOPATH.

modtest kjk$ GO111MODULE=off go get -u -v github.com/gomarkdown/markdown/html
github.com/gomarkdown/markdown (download)

modtest kjk$ GO111MODULE=on go get -u -v github.com/gomarkdown/markdown/html
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
Fetching https://github.com?go-get=1
Parsing meta tags from https://github.com?go-get=1 (status code 200)
go get github.com/gomarkdown/markdown/html: no matching versions for query "latest"
josephlr

josephlr commented on Sep 8, 2018

@josephlr
josephlr
on Sep 8, 2018 · edited by josephlr
Member

@kjk Still no luck reproducing your bug on darwin/amd64

I'm outside of GOPATH, no Go environment variables set (except GOPATH), using go 1.11 installed from Homebrew latest. I even went an manually deleted the entire package cache. Here's my full output:

joe@Lust ➜  git clone https://github.com/kjk/modtest.git
Cloning into 'modtest'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 10 (delta 1), reused 7 (delta 1), pack-reused 0
Unpacking objects: 100% (10/10), done.
joe@Lust ➜  cd modtest
joe@Lust ➜  bash
bash-3.2$ exit
exit
joe@Lust ➜  GO111MODULE=on go get -u -v github.com/gomarkdown/markdown/html
go: creating new go.mod: module github.com/kjk/modtest
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
go: downloading github.com/gomarkdown/markdown v0.0.0-20180907191918-6fda95a9e93f
joe@Lust ➜  go build
joe@Lust ➜  ./modtest
Hello
joe@Lust ➜  pwd
/Users/joe/Documents/programming/funstuff/modtest
joe@Lust ➜  echo $GOPATH
/Users/joe/Documents/programming/go

Other versions of software that could help:

git --version
git version 2.16.2

wget -V
GNU Wget 1.19.4 built on darwin17.3.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls
+ntlm +opie -psl +ssl/openssl

Wgetrc:
    /Users/joe/.wgetrc (user)
    /usr/local/etc/wgetrc (system)
Locale:
    /usr/local/Cellar/wget/1.19.4_1/share/locale
Compile:
    clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc"
    -DLOCALEDIR="/usr/local/Cellar/wget/1.19.4_1/share/locale" -I.
    -I../lib -I../lib -I/usr/local/opt/openssl/include -DNDEBUG
Link:
    clang -DNDEBUG -lidn2 -L/usr/local/opt/openssl/lib -lssl -lcrypto
    -ldl -lz ftp-opie.o openssl.o http-ntlm.o ../lib/libgnu.a -liconv
    -lintl -Wl,-framework -Wl,CoreFoundation -lunistring

Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.
kjk

kjk commented on Sep 8, 2018

@kjk
kjk
on Sep 8, 2018 · edited by kjk
Author

@josephlr Interesting. I tried to reproduce on another mac machine and couldn't.

So it's something specific to a setup on one machine but I have no ideas on how to investigate this further.

They both have similar setup, all programs from homebrew, kept up-to-date.

agnivade

agnivade commented on Sep 9, 2018

@agnivade
agnivade
on Sep 9, 2018
Contributor

@bcmills

myitcv

myitcv commented on Sep 9, 2018

@myitcv
myitcv
on Sep 9, 2018
Member

@kjk please can you try the following:

export GOPATH=$(mktemp -d)
cd $(mktemp -d)
git clone https://github.com/kjk/modtest.git
cd modtest/
go mod init
go build
kjk

kjk commented on Sep 10, 2018

@kjk
kjk
on Sep 10, 2018
Author

That works, but I already said it works inside GOPATH.

Krzysztofs-MacBook-Pro:src kjk$ export GOPATH=$(mktemp -d)
Krzysztofs-MacBook-Pro:src kjk$ cd $(mktemp -d)
Krzysztofs-MacBook-Pro:tmp.uw4hgRPe kjk$ git clone https://github.com/kjk/modtest.git
Cloning into 'modtest'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 10 (delta 1), reused 7 (delta 1), pack-reused 0
Unpacking objects: 100% (10/10), done.
Krzysztofs-MacBook-Pro:tmp.uw4hgRPe kjk$ cd modtest/
Krzysztofs-MacBook-Pro:modtest kjk$ go mod init
go: creating new go.mod: module github.com/kjk/modtest
Krzysztofs-MacBook-Pro:modtest kjk$ go build
go: finding github.com/gomarkdown/markdown/ast latest
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown latest
go: downloading github.com/gomarkdown/markdown v0.0.0-20180907191918-6fda95a9e93f
myitcv

myitcv commented on Sep 10, 2018

@myitcv
myitcv
on Sep 10, 2018
Member

@kjk

That works, but I already said it works inside GOPATH.

But this test was outside of GOPATH; notice GOPATH and your working directory are two different temporary directories, so by definition you are outside of GOPATH.

This therefore sounds like an issue with your build/module cache (most likely the latter).

Perhaps take a backup of $GOPATH/pkg/mod (in case we want to investigate further) and then run go clean -modcache and see whether your original problem persists.

bcmills

bcmills commented on Sep 10, 2018

@bcmills
bcmills
on Sep 10, 2018
Contributor

I'm not able to reproduce the failure either.

I'm curious what happens if you run:

go get github.com/gomarkdown/markdown@v0.0.0-20180907191918-6fda95a9e93f
go mod verify

~$ cd $(mktemp -d)

$ git clone https://github.com/kjk/modtest.git
Cloning into 'modtest'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 10 (delta 1), reused 7 (delta 1), pack-reused 0
Unpacking objects: 100% (10/10), done.

$ cd modtest

modtest$ go mod init github.com/kjk/modtest
go: creating new go.mod: module github.com/kjk/modtest

modtest$ go build
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown/ast latest
go: finding github.com/gomarkdown/markdown latest
go: downloading github.com/gomarkdown/markdown v0.0.0-20180907191918-6fda95a9e93f

modtest$ go version
go version devel +b0a1c5df98 Wed Aug 8 12:50:33 2018 -0400 linux/amd64
bcmills
added
WaitingForInfoIssue is not actionable because of missing required information, which needs to be provided.
NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.
on Sep 10, 2018
bcmills
added this to the Go1.12 milestone on Sep 10, 2018
kjk

kjk commented on Sep 11, 2018

@kjk
kjk
on Sep 11, 2018
Author

go clean -modcache fixed this.

So the cache was busted to the point of messing things up. Knowing what I know now, most likely /Users/kjk/go/pkg/mod/github.com/gomarkdown/markdown\@v0.0.0-20180907191918-6fda95a9e93f and/or corresponding /Users/kjk/go/pkg/mod/cache/vcs/${sha1} and/or /Users/kjk/Library/Caches/go-build was corrupted.

Unfortunately by removing the cache I also lost all the info needed to investigate it.

A hint from earlier debug tries:

modtest kjk$ go get -v
go: finding github.com/gomarkdown/markdown/html latest
go: finding github.com/gomarkdown/markdown/ast latest
go: finding github.com/gomarkdown/markdown latest
Fetching https://github.com?go-get=1
Parsing meta tags from https://github.com?go-get=1 (status code 200)
github.com/gomarkdown/markdown/html
github.com/gomarkdown/markdown/ast
go build github.com/gomarkdown/markdown/html: no Go files in

no Go files in indicates that the git repository was corrupted because there are .go files in https://github.com/gomarkdown/markdown/tree/master/html

Also the error message is truncated or just badly phrased.

Some other thoughts.

-n flag for -modcache seems broken

$ go clean -n -modcache
cd /Users/kjk/src/modtest
rm -f modtest modtest.exe modtest.test modtest.test.exe main main.exe

I'm pretty sure go clean -modcache deletes more than that.

debugging tools for this area are not adequate

-v doesn't provide enough information.

GODEBUG allows printing a bit more information but not enough and it's not documented anywhere in full (as in: relevant to go toolchain as a whole; it's scattered in runtime docs etc.)

In retrospect it looks like local copy of github.com/gomarkdown/markdown in /Users/kjk/go/pkg/mod/cache/vcs/ was busted but I only figured it out after running strace (well, dtruss) on go get step (after I fixed it by cleaning the cache).

To properly debug this there should be a switch that makes go tool print all the files it's inspecting, all git commands its executing etc.

Realistically this bug can be closed because I don't see a way of making progress on debugging this further.

As a data point:

  • it's possible to corrupt modules cache in a way that is not detected
  • earlier I've been running 1.11 from master post beta 2 so corruption might have been introduced by that version and subsequently the root cause of such corruption might have been fixed before release
bcmills

bcmills commented on Sep 11, 2018

@bcmills
bcmills
on Sep 11, 2018
Contributor

I'm pretty sure go clean -modcache deletes more than that.

Yeah, go clean needs quite a bit of work for 1.12. (See also #27469, #27458, #27310, and #26991.)

earlier I've been running 1.11 from master post beta 2 so corruption might have been introduced by that version and subsequently the root cause of such corruption might have been fixed before release

That could certainly do it. go1.11beta2 was a pretty rough build for modules (sorry about that!).

bcmills
closed this as completedon Sep 11, 2018
thepudds
mentioned this on Apr 28, 2019
golang
locked and limited conversation to collaborators on Sep 11, 2019
gopherbot
added
FrozenDueToAge
on Sep 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.WaitingForInfoIssue is not actionable because of missing required information, which needs to be provided.modules

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @kjk@agnivade@myitcv@bcmills@josephlr

        Issue actions
          go mod doesn't work for github.com/gomarkdown/markdown/html · Issue #27565 · golang/go