cmd/cgo: do not let Go pointers end up in C

[rsc]

We need some way to prevent Go pointers from crossing the boundary into C.

People write code like (calling a C function that takes a T* to fill in).

var x C.T
C.f(&x)

One option is to disallow this - a Go pointer &x is being passed to C.

Another option is to have cgo's wrapper for C.f do a shallow copy into an allocated C
pointer, then run f, then copy the data back into the original Go data, then free the
allocated C pointer, then return.

If the second option handles 99% of the pointers being passed, then we probably should
do it. On the other hand, the free might create a dangling pointer that could confuse
people. At least it's a dangling pointer into the C heap and not the Go heap. :-)

[griesemer]

Comment 2:

Just for reference, and not meant as implementation suggestion: Java's JNI passes
pointers as "handles": A JNI-wrapper (a little stub, generated on-the fly for each call
signature) pushes the arguments on the stack again in C (reverse) order. For each Java
object, the stack address of the corresponding Java pointer is pushed. In C, that stack
address (which is immutable for the duration of the C call) acts as the handle via which
the underlying Java object is accessed. This permits GC to move objects around, but
comes at the cost of an extra (hopefully inlined) function call and indirection for each
Java data structure access from within C. It also requires the respective C code to be
written specifically with JNI in mind.

[ianlancetaylor]

Comment 3:

I think JNI is a good example of something we don't want.

[randall77]

Comment 4:

Another option is to pin pointers that are passed into C.f for the duration of C.f. 
This allows the C code to use raw pointers, but does not allow the C code to squirrel
the pointer away past the immediate call.
There's no way to enforce the no-squirrel rule, unfortunately.
This requires a mechanism to pin objects in the heap.  If/when we have a copying
collector this will be a pain, but might be necessary for other reasons (runtime
internals?).

[rsc]

Comment 5:

There's no way to enforce the no-squirrel rule, and also if we have any kind of write
barrier on pointer updates then the C code would not respect that, which could be
problematic. I think it will be much simpler in the long run if we keep C from ever
seeing a Go pointer.

[dvyukov]

Comment 6:

Updating Go objects from C code never worked. C code runs concurrently with GC. GC needs
to see consistent picture of the world. If C updates Go objects, GC corrupts heap.

[rsc]

Comment 7:

All the more reason to not let Go pointers end up in C.

[dominikh]

Comment 8:

Does this issue affect all Go pointers? The following is a somewhat common pattern in
cgo land: `x := make([]byte, 4096); C.foo(&x[0], len(x))`

[minux]

Comment 9:

Re #8, it will affect *all* Go pointers crossing language boundary.

[bryanturley]

Comment 10:

I am confused, Is this for an upcoming copying collector?
Would the basic read/write syscalls need to be double buffered?
I think the problem isn't stated clearly.

[ianlancetaylor]

Comment 11:

One goal is to make it possible to write a copying collector.  Another goal is to avoid
the possibility of the Go garbage collector collecting a pointer that has been passed to
C code.

[gopherbot]

Comment 12:

Please keep in mind leveldb, sqlite etc where one might actually want to store binary
blobs of data that e.g. came from a network socket. There has to be a better way than an
extra copy round for everything, e.g. for when you know the data pointed to does not
contain further pointers.

[andlabs]

Comment 13:

The copying proposal would be bad for code like my ui package, which wants to store a Go
pointer on the C side for the purposes of giving it back to Go when an event comes in.
dominikh says he also has callback functions which work similarly.
I don't know how you intend to deal with this,  unless it's something through
unsafe.Pointer, but then you would need to use unsafe.Pointer for C-side types allocated
in Go, which is also ugly.

[alexbrainman]

Comment 14:

@pietro10, windows standard packages pass pointers to Go objects into Windows kernel in
a few places too. So the situation is known.
Alex

[RLH]

Comment 15:

Passing a Go heap pointer to C can break today's 1.3 GC which does not scan
C stacks / variables.

[ianlancetaylor]

Comment 16:

In the long run there just isn't any way that we can support passing a Go pointer into
C.  We want to make it possible to write a moving garbage collector, but that means that
the garbage collector has to be able to change all pointers.  The GC won't be able to
see pointers stored in C code.  So we can't permit that.
To pass a pointer into C and get it back you will have to do something like use a
map[uintptr]*X.  When you want to pass the pointer down, convert to uintptr and store
the pointer in the map.  When you get it back, use the map to get the real Go pointer.