proposal: add a builtin function for making an accurate copy of any value

[feyeleanor]

Problem

Currently there seems to be no general method for making a shallow but complete copy of an existing value of arbitrary type in a Go program. This is particularly true when a value is a struct with unexported fields.

Even using reflection to perform a direct binary copy is a minefield of segfaults, unaddressable values and faked []byte using reflection which makes such code difficult to reason about.

Proposal

There are cases where making a full and accurate copy of an existing value - even a struct value with unexported fields - is a perfectly reasonable thing to do so I propose two possible additions to Go which would support this without compromising the key premise of unexported fields: that they cannot be modified except by functions and methods in the package where they're defined.

(Making an exact copy is not conceptually the same as changing the field contents, the problem is an artefact of Go's reflection system being implemented as a package rather than with some other API).

A new builtin function with signature duplicate(v Type) Type would return a complete and faithful shallow copy of any value. At the byte level it would be binary identical to the value it's created from and all the usual rules would apply to it, so unexported fields would still only be changeable by functions defined in its source package.

Usage

A snippet of how code would look using this function:

a := struct{ x, y int } { 3, 4 }
b := duplicate(x)
fmt.Printf("a.x: %v, a.y: %v\n", a.x, a.y)
fmt.Printf("b.x: %v, b.y: %v\n", b.x, b.y)

in both cases fmt.Printf() will print out the same values for the x and y fields.

Problems

In the case of reference types such as maps, slices, interfaces, pointers, channels, etc. a decision would have to be made as to whether to make a copy of the reference (slice header, etc.) or a copy of the underlying values.

• The former is inexpensive but possibly too shallow;
• the latter may not make sense for all reference types;
• perhaps reference types shouldn't duplicate at all;
• perhaps duplicate() is dangerous enough to be part of unsafe.

[minux]

There is already a way to do shadow copy of a value, just use assignment: a := struct{ x, y int } { 3, 4 } b := a fmt.Printf("a.x: %v, a.y: %v\n", a.x, a.y) fmt.Printf("b.x: %v, b.y: %v\n", b.x, b.y) What value does the proposed "duplicate" function add?

[bradfitz]

Even using reflection to perform a direct binary copy is a minefield of segfaults, unaddressable values and faked []byte using reflection which makes such code difficult to reason about.

So, put it in a library, rather than asking each user to do it by hand? Actually, it exists already: https://godoc.org/?q=deepcopy

Why does it need to be part of the language?

And if it were part of the language, I'm sure it would surprise people to learn that a function named duplicate only makes a shallow and not a deep clone of its argument.

[minux]

Perhaps a reflect.DeepCopy (to mirror reflect.DeepEqual) is better? However, there is already a reflect.Copy, and it does something unrelated (it relates the copy builtin function.) But still, I think needing to do deep copy on a value suggests an API design problem.

[rogpeppe]

As others have already pointed out, making an shallow copy of a type in Go is already straightforward.

Implementing a general deep copy is problematic. What should
happen when there's a diamond dependency? For the general case, if two
objects both point to the same object, they should continue to do so after copying.
But this is hard to do efficiently. Here's an example:

type A struct {
     SliceV []A
     SliceP []*A
 }
 x := A{SliceV: make([]A, 3)}
 y := A{SliceV: x.SliceV[1:]}
 z = A{SliceV: make([]A, 2)}
 x.SliceP = []*A{&x.SliceV[0], &x.SliceV[1], &z.SliceV[0]}
 y.SliceP = []*A{&x.SliceV[1]}
 w := deepcopy(&y)

If we were to copy this correctly, then all the slice and pointer references would
be maintained. This requires quite a bit of information to be retained
when copying. I had a go at doing this (https://godoc.org/github.com/rogpeppe/rog-go/exp/deepcopy) ages ago, but the code no longer compiles and I've never
come across a case where this general functionality is actually needed.

The only time I've actually needed a general deep copy, json.Marshal
followed by json.Unmarshal was sufficient.

func deepCopy(x interface{}) (interface{}, error) {
	data, err := json.Marshal(x)
	if err != nil {
		return nil, err
	}
	destv := reflect.New(reflect.TypeOf(x))
	err = json.Unmarshal(data, destv.Interface())
	if err != nil {
		return nil, err
	}
	return destv.Elem().Interface(), nil
}

It would be useful to have some specific cases where you see a more
general version of DeepCopy being useful.

But still, I think needing to do deep copy on a value suggests
an API design problem.

I tend to agree with this.

[feyeleanor]

This is not a proposal for a DeepCopy mechanism.

This is a method for making an exact duplicate of a value, including its unexported fields.

[minux]

Make a shadow copy is as simple as a regular assignment, why do we need a new built-in function? What am I missing here?

[bradfitz]

And even if you need to use reflect, it's just:

func duplicate(v interface{}) interface{} {
	rv := reflect.New(reflect.TypeOf(v)).Elem()
	rv.Set(reflect.ValueOf(v))
	return rv.Interface()
}

[feyeleanor]

Not if it has unexported fields.

[cespare]

@feyeleanor minux's first reply shows that assignment does indeed copy the unexported struct fields.

[minux]

And even if you have to use reflect, you can still copy unexported fields: (based on @bradfitz's code above): https://play.golang.org/p/YcJEcfUMXK

[bradfitz]

I'm going to close this, since it's unclear what benefit there is having this more built-in than it already is. Feel free to reopen if you have a more formal proposal.

[feyeleanor]

In which case I've misunderstood the documentation for reflect.Value.Set() and reflect.Value.CanSet().

If @bradfitz's code is robust in all circumstances (in addition to @minux's example I've tried it with an unexported struct type containing unexported fields from one of my own packages & that works) then it would still make sense to either have a convenience method on reflect.Value:

func Value.Duplicate() Value {
	rv := New(TypeOf(v)).Elem()
	rv.Set(ValueOf(v))
	return rv
}

or to include an example in the documentation for reflect.

[bradfitz]

https://blog.golang.org/laws-of-reflection is worth reading.

[feyeleanor]

Unfortunately Laws of Reflection doesn't cover this and the Third Law coverage which is relevant is obscured by using an example where addressability is the basis for settability.

An example in the reflect documentation would leave no room for misunderstanding.

[rogpeppe]

@feyeleanor Assuming you actually mean this:

func (v Value) Duplicate() Value {
        rv := New(v.Type())
        rv.Set(v)
        return rv.Elem()
}

but even if this trivial function was provided, what does this
give you that v.Interface() does not?

The documentation of Value.Set and Value.CanSet seems right
to me. "A Value can be changed only if it is addressable and was not obtained by the use of unexported struct fields." This doesn't say anything about the fields inside the
value. "x's value must be assignable to v's type." This implies the usual Go
assignment rules which do not prohibit assignment of unexported fields
as part of a larger value.

BTW the description of this issue says "add a builtin function for making an accurate copy of any value". None of the reflect package functions are built in. The builtin "function" for making an accurate copy of any value is the assignment operator.