Skip to content

runtime: OpenReader: Invalid heap pointer in both 1.4rc2 and 1.4rc1 #9191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gopherbot opened this issue Dec 2, 2014 · 6 comments
Closed
Milestone

Comments

@gopherbot
Copy link
Contributor

by pruest:

Works in all releases prior to 1.4rc1.

Code base installs, but fails at run time (test or regular execution).
The code triggering the problem is attempting to open a binary looking
for an embedded zip file. In this case there is no zip file in the binary. 
In the past an error would have been returned.

    reader, err := zip.OpenReader(programName)  // load.go line 102 in trace below.

Excerpt from full trace below:
--------------------------------
goroutine 5 [garbage collection]:
runtime.switchtoM()
    /usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc20805e8b8 sp=0xc20805e8b0
runtime.gogc(0x0)
    /usr/local/go/src/runtime/malloc.go:469 +0x1cf fp=0xc20805e8f0 sp=0xc20805e8b8
runtime.mallocgc(0x12000, 0x4158140, 0x1, 0x403dff3)
    /usr/local/go/src/runtime/malloc.go:341 +0x391 fp=0xc20805e9a0 sp=0xc20805e8f0
runtime.newarray(0x4158140, 0x10400, 0x400)
    /usr/local/go/src/runtime/malloc.go:365 +0xc1 fp=0xc20805e9d8 sp=0xc20805e9a0
runtime.makeslice(0x414f980, 0x10400, 0x10400, 0x0, 0x0, 0x0)
    /usr/local/go/src/runtime/slice.go:32 +0x15c fp=0xc20805ea20 sp=0xc20805e9d8
archive/zip.readDirectoryEnd(0x43a4d08, 0xc20802a090, 0x30d1c8, 0x0, 0x0, 0x0)
    /usr/local/go/src/archive/zip/reader.go:330 +0x10c fp=0xc20805eb68 sp=0xc20805ea20
archive/zip.(*Reader).init(0xc20802e588, 0x43a4d08, 0xc20802a090, 0x30d1c8, 0x0, 0x0)
    /usr/local/go/src/archive/zip/reader.go:76 +0x65 fp=0xc20805ec58 sp=0xc20805eb68
archive/zip.OpenReader(0x7fff5fbffa58, 0x5c, 0x0, 0x0, 0x0)
    /usr/local/go/src/archive/zip/reader.go:57 +0x1ae fp=0xc20805ecf0 sp=0xc20805ec58
vu/load.newLoader(0x19)
    /Users/rust/code/vu/src/vu/load/load.go:102 +0x28c fp=0xc20805edd0 sp=0xc20805ecf0


------------------------------------------------------------------------------------------------------
Full trace from 1.4rc2
------------------------------------------------------------------------------------------------------
runtime: garbage collector found invalid heap pointer *(0xc20805ee48+0x80)=0x18 s=nil
fatal error: invalid heap pointer

runtime stack:
runtime.throw(0x42c2f63)
    /usr/local/go/src/runtime/panic.go:491 +0xad fp=0x7fff5fbff310 sp=0x7fff5fbff2e0
scanblock(0xc20805ee48, 0x128, 0x422e178)
    /usr/local/go/src/runtime/mgc0.c:378 +0x551 fp=0x7fff5fbff450 sp=0x7fff5fbff310
scanframe(0x7fff5fbff558, 0x0, 0x101)
    /usr/local/go/src/runtime/mgc0.c:719 +0x164 fp=0x7fff5fbff4c0 sp=0x7fff5fbff450
runtime.gentraceback(0x4039df0, 0xc20805e8b0, 0x0, 0xc208000b40, 0x0, 0x0, 0x7fffffff,
0x7fff5fbff608, 0x0, 0x0, ...)
    /usr/local/go/src/runtime/traceback.go:311 +0x7a8 fp=0x7fff5fbff5b0 sp=0x7fff5fbff4c0
scanstack(0xc208000b40)
    /usr/local/go/src/runtime/mgc0.c:777 +0x21c fp=0x7fff5fbff620 sp=0x7fff5fbff5b0
markroot(0xc208010000, 0x9)
    /usr/local/go/src/runtime/mgc0.c:553 +0xe7 fp=0x7fff5fbff680 sp=0x7fff5fbff620
runtime.parfordo(0xc208010000)
    /usr/local/go/src/runtime/parfor.c:76 +0xb2 fp=0x7fff5fbff700 sp=0x7fff5fbff680
gc(0x7fff5fbff838)
    /usr/local/go/src/runtime/mgc0.c:1439 +0x25e fp=0x7fff5fbff818 sp=0x7fff5fbff700
runtime.gc_m()
    /usr/local/go/src/runtime/mgc0.c:1368 +0xe0 fp=0x7fff5fbff850 sp=0x7fff5fbff818
runtime.onM(0x42e1010)
    /usr/local/go/src/runtime/asm_amd64.s:257 +0x68 fp=0x7fff5fbff858 sp=0x7fff5fbff850
runtime.mstart()
    /usr/local/go/src/runtime/proc.c:818 fp=0x7fff5fbff860 sp=0x7fff5fbff858

goroutine 5 [garbage collection]:
runtime.switchtoM()
    /usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc20805e8b8 sp=0xc20805e8b0
runtime.gogc(0x0)
    /usr/local/go/src/runtime/malloc.go:469 +0x1cf fp=0xc20805e8f0 sp=0xc20805e8b8
runtime.mallocgc(0x12000, 0x4158140, 0x1, 0x403dff3)
    /usr/local/go/src/runtime/malloc.go:341 +0x391 fp=0xc20805e9a0 sp=0xc20805e8f0
runtime.newarray(0x4158140, 0x10400, 0x400)
    /usr/local/go/src/runtime/malloc.go:365 +0xc1 fp=0xc20805e9d8 sp=0xc20805e9a0
runtime.makeslice(0x414f980, 0x10400, 0x10400, 0x0, 0x0, 0x0)
    /usr/local/go/src/runtime/slice.go:32 +0x15c fp=0xc20805ea20 sp=0xc20805e9d8
archive/zip.readDirectoryEnd(0x43a4d08, 0xc20802a090, 0x30d1c8, 0x0, 0x0, 0x0)
    /usr/local/go/src/archive/zip/reader.go:330 +0x10c fp=0xc20805eb68 sp=0xc20805ea20
archive/zip.(*Reader).init(0xc20802e588, 0x43a4d08, 0xc20802a090, 0x30d1c8, 0x0, 0x0)
    /usr/local/go/src/archive/zip/reader.go:76 +0x65 fp=0xc20805ec58 sp=0xc20805eb68
archive/zip.OpenReader(0x7fff5fbffa58, 0x5c, 0x0, 0x0, 0x0)
    /usr/local/go/src/archive/zip/reader.go:57 +0x1ae fp=0xc20805ecf0 sp=0xc20805ec58
vu/load.newLoader(0x19)
    /Users/rust/code/vu/src/vu/load/load.go:102 +0x28c fp=0xc20805edd0 sp=0xc20805ecf0
vu/load.NewLoader(0x0, 0x0)
    /Users/rust/code/vu/src/vu/load/load.go:78 +0x31 fp=0xc20805ee00 sp=0xc20805edd0
vu/audio.TestAudio(0xc208062000)
    /Users/rust/code/vu/src/vu/audio/audio_test.go:15 +0x67 fp=0xc20805ef78 sp=0xc20805ee00
testing.tRunner(0xc208062000, 0x42c73a0)
    /usr/local/go/src/testing/testing.go:447 +0xbf fp=0xc20805efd0 sp=0xc20805ef78
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20805efd8 sp=0xc20805efd0
created by testing.RunTests
    /usr/local/go/src/testing/testing.go:555 +0xa8b

goroutine 1 [chan receive]:
runtime.gopark(0x4032190, 0xc208056238, 0x41ed330, 0xc)
    /usr/local/go/src/runtime/proc.go:130 +0x105 fp=0xc208061ba8 sp=0xc208061b78
runtime.goparkunlock(0xc208056238, 0x41ed330, 0xc)
    /usr/local/go/src/runtime/proc.go:136 +0x48 fp=0xc208061bd0 sp=0xc208061ba8
runtime.chanrecv(0x4150040, 0xc2080561e0, 0xc208061dc0, 0x4058a01, 0x0)
    /usr/local/go/src/runtime/chan.go:410 +0x452 fp=0xc208061c70 sp=0xc208061bd0
runtime.chanrecv1(0x4150040, 0xc2080561e0, 0xc208061dc0)
    /usr/local/go/src/runtime/chan.go:311 +0x2b fp=0xc208061ca0 sp=0xc208061c70
testing.RunTests(0x4234e10, 0x42c73a0, 0x1, 0x1, 0xee99d9ca48babd01)
    /usr/local/go/src/testing/testing.go:556 +0xad6 fp=0xc208061e48 sp=0xc208061ca0
testing.(*M).Run(0xc2080300f0, 0x42cc8e0)
    /usr/local/go/src/testing/testing.go:485 +0x6c fp=0xc208061ed0 sp=0xc208061e48
main.main()
    vu/audio/_test/_testmain.go:52 +0x1d5 fp=0xc208061f98 sp=0xc208061ed0
runtime.main()
    /usr/local/go/src/runtime/proc.go:63 +0xf3 fp=0xc208061fe0 sp=0xc208061f98
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208061fe8 sp=0xc208061fe0

goroutine 2 [force gc (idle)]:
runtime.gopark(0x4032190, 0x42e0bb0, 0x41ee7f0, 0xf)
    /usr/local/go/src/runtime/proc.go:130 +0x105 fp=0xc20801a798 sp=0xc20801a768
runtime.goparkunlock(0x42e0bb0, 0x41ee7f0, 0xf)
    /usr/local/go/src/runtime/proc.go:136 +0x48 fp=0xc20801a7c0 sp=0xc20801a798
runtime.forcegchelper()
    /usr/local/go/src/runtime/proc.go:99 +0xce fp=0xc20801a7e0 sp=0xc20801a7c0
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20801a7e8 sp=0xc20801a7e0
created by runtime.init·4
    /usr/local/go/src/runtime/proc.go:87 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x4032190, 0x42cccb8, 0x41e8210, 0xd)
    /usr/local/go/src/runtime/proc.go:130 +0x105 fp=0xc20801df98 sp=0xc20801df68
runtime.goparkunlock(0x42cccb8, 0x41e8210, 0xd)
    /usr/local/go/src/runtime/proc.go:136 +0x48 fp=0xc20801dfc0 sp=0xc20801df98
runtime.bgsweep()
    /usr/local/go/src/runtime/mgc0.go:98 +0xbc fp=0xc20801dfe0 sp=0xc20801dfc0
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20801dfe8 sp=0xc20801dfe0
created by gc
    /usr/local/go/src/runtime/mgc0.c:1383

goroutine 4 [finalizer wait]:
runtime.gopark(0x4032190, 0x42cccb0, 0x41ee0b0, 0xe)
    /usr/local/go/src/runtime/proc.go:130 +0x105 fp=0xc208018f30 sp=0xc208018f00
runtime.goparkunlock(0x42cccb0, 0x41ee0b0, 0xe)
    /usr/local/go/src/runtime/proc.go:136 +0x48 fp=0xc208018f58 sp=0xc208018f30
runtime.runfinq()
    /usr/local/go/src/runtime/malloc.go:727 +0xba fp=0xc208018fe0 sp=0xc208018f58
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208018fe8 sp=0xc208018fe0
created by runtime.createfing
    /usr/local/go/src/runtime/malloc.go:707 +0x5e

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc20805dfe8 sp=0xc20805dfe0
@ianlancetaylor
Copy link
Contributor

Comment 1:

How can we recreate the problem?
What system are you running on?
Does your program use cgo?

Labels changed: added release-go1.4maybe, repo-main.

@gopherbot
Copy link
Contributor Author

Comment 2 by pruest:

OS X 10.10.1.  Cgo is involved.  Refinement of the problem in the attached file using
standard OpenAL on OS X and the OpenAL go bindings provided provided in the zip. Here is
the program from the zip.
package main
import (
    "al" // from https://github.com/gazed/vu/tree/master/src/vu/audio/al
    "archive/zip"
    "os"
)
// Issue: 9191. All 4 cgo calls and zip.OpenReader are necessary to cause issue.
func main() {
    al.Init()                                 // cgo
    if dev := al.OpenDevice(""); dev != nil { // cgo
        if ctx := al.CreateContext(dev, nil); ctx != nil { // cgo
            defer al.DestroyContext(ctx) // cgo
            programName := os.Args[0]
            zip.OpenReader(programName) // crashes?
        }
    }
}

Attachments:

  1. issue9191.zip (10332 bytes)

@ianlancetaylor
Copy link
Contributor

Comment 3:

I doubt this has anything to do with zip.  You would probably get the same crash calling
runtime.GC() instead of zip.OpenReader.
The problem is that something in the Go heap has a pointer type but contains the value
0x18, which is invalid.
I can't recreate the problem on GNU/Linux.  I don't see anything in your code that would
cause it.  This will have to be debugged by somebody with access to a Darwin system.

@gopherbot
Copy link
Contributor Author

Comment 4 by pruest:

Verified that problem can't be recreated on Windows 8.1 using OpenAL.dll from
http://kcat.strangesoft.net/openal.html.

@minux
Copy link
Member

minux commented Dec 3, 2014

Comment 5:

Reproduced on OS X, but it's not a Go problem.
the *al.Device returned from al.OpenDevice("") is 0x18, which is obviously
an invalid pointer.
If al.OpenDevice could return invalid pointers, the value shouldn't use a
Go pointer type to store.
Because regardless of the platform, the value returned from al.OpenDevice
is not a Go pointer (it can be a C pointer, or even an regular integer), so
Go's GC won't touch it anyway,  you can just use a uintptr to store it.
This situation is similar to Window's HANDLE.
Arguably, this is a bug of OS X OpenAL implementation, but as ALCdevice
is an opaque struct, whether alcOpenDevice returns a real pointer or not
doesn't matter that much to a C programmer.
For Go's new fully precise GC, not only passing Go heap pointer to C poses
problem, even storing a C value (pointer or struct) on Go stack/heap is also
potentially problematic as there might be a field that has a pointer type but
actually stores an integer. Perhaps we should do something here.

Status changed to WorkingAsIntended.

@gopherbot
Copy link
Contributor Author

Comment 6 by pruest:

Fantastic! Thank you!
Both al.OpenDevice (0x18) and al.CreateContext (0x19) return invalid pointer values on
OS X and regular pointers on Windows. Using uintptr to store them, as you recommend,
solves the problem.

This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants