proposal: Go 2: builtin add func resize(v Type, size)

[elichai]

The make builtin allow you to specify a capacity for maps,slices and channels, but once you specify you cannot currently change that.
With slices you could make a new slice, copy over and assign to the current slice, but with maps/channels it is harder.
Maps will auto-scale with inserts but sometimes you know you're going to add X elements to the map so you can tell it to reallocate/resize beforehand.
Channels are even worse, you currently are stuck with the buffer you choose when you created the channel, sometimes a single channel is used to communicate with multiple go routines, and you want to resize the buffer if you launch more routines s.t. no routine will block on the channel.

I think a way to resize/realloc a type can be very useful, both for performance and usability,
an alternative for a new builtin can be to add this support to the make builtin

[davecheney]

Thank you for raising this proposal.

I don’t think there is much value in resizing slices as they grow geometrical when appended to. If this is not good enough making a new slice and copying the contents is two lines.

Maps grow by doubling (I think) and provide a size hint. Perhaps this is good enough, if not, some benchmarks showing that map rehashing is dominating insert performance would be valuable for this proposal.

I cannot see how a slice could be resized safely without additional locking. There are probably some unanswered questions when a channel is resized from unbuffered to buffered.

[martisch]

With slices you could make a new slice, copy over and assign to the current slice, but with maps/channels it is harder.
Is it really that much harder for maps to warrant a new language addition?

new := make(map[X]Y, capacity)
for key, value := range old {
    new[key] = value
}
old = new

If performance is a concern that can solved without a language change:
#26951

With generics a generic map copy function can be defined not requiring more typing than a builtin and no additional language change.

Can you pleas spell out in more detail what resize does and where it is useful and how often that comes up?
e.g. Does it an inplace replace of a slice header like a = append(a, ...)? Are existing pointers/references to the old backing array of the slice effected? Is a size lower then the current size allowed? What happens if the size is smaller then the number of elements the map currently holds?...

[martisch]

For language change proposals, please fill out the template at https://go.googlesource.com/proposal/+/refs/heads/master/go2-language-changes.md .

When you are done, please reply to the issue with @gopherbot please remove label WaitingForInfo.

Thanks!

[mvdan]

Heh, @martisch, your instructions to tell the bot to remove the label just removed the label. I don't think it knows how to tell that your comment came before the label was added.

[martisch]

Heh, @martisch, your instructions to tell the bot to remove the label just removed the label. I don't think it knows how to tell that your comment came before the label was added.

Ack. ordering error on my part ...

[elichai]

I don’t think there is much value in resizing slices as they grow geometrical when appended to. If this is not good enough making a new slice and copying the contents is two lines.

I agree that slices have the least value in this proposal

Maps grow by doubling (I think) and provide a size hint. Perhaps this is good enough, if not, some benchmarks showing that map rehashing is dominating insert performance would be valuable for this proposal.

I can redo the benchmarks but I did see runtime.mapassign very high in my app's benchmarks.

I cannot see how a slice could be resized safely without additional locking. There are probably some unanswered questions when a channel is resized from unbuffered to buffered.

well no slice operation is thread safe.
about channels, I really don't know how their internals are implemented, so I can't quite comment on how to resize them safely

new := make(map[X]Y, capacity)
for key, value := range old {
    new[key] = value
}
old = new

If performance is a concern that can solved without a language change:
#26951

A. I did not know this is possible :).
B. the performance of this is definitely a concern.

Can you pleas spell out in more detail what resize does and where it is useful and how often that comes up?

the map scenerio comes a lot in functions like
it comes a lot in code like:

func (c myCollection) AddMultiple(objs []Objects) {
    for _, obj := range objs {
        c[obj.key()] = obj.value()
    }
}

or even:

func (c myCollection) Merge(other myCollection) {
    for key, value := range other {
        c[key] = value
    }
}

e.g. Does it an inplace replace of a slice header like a = append(a, ...)? Are existing pointers/references to the old backing array of the slice effected? Is a size lower then the current size allowed? What happens if the size is smaller then the number of elements the map currently holds?...

obviously just like with make the size is just a hint that the compiler can ignore, about the sizes we can either ignore, or decide that the caller provides the additional capacity not the resulting capacity. (that's how rust's Vec/HashMap works, on the contrary C++'s std::vector/map is given the resulting capacity, and it is ignore if equal or less than the current capacity)

about pointers/references it should work just like a natural resizing caused by appending/inserting (so yeah it's kinda weird in slices)

EDIT: @martisch Should I write a comment here/edit the OP with answers to all the questions in the template?

[martisch]

The map examples given just show adding elements. There would need to be an additional resize call that is trying to more then double the capacity. How would that look like in the code above and what capacity would it target?

If its not more than doubling the internal resize mechanism of maps would do the same as the resize on the fly already. A resize will need to also copy all elements to a new map allocation in the current design of maps to grow the map.
Spending alot of time in mapassign is not necessarily a sign of map growth but can entirely be generated by just adding elements to a big enough map. runtime.evacuate_* functions are work done for grow work. As discussed note that this work will not entirely go away. each resize would need to do similar work to the amount of exisitng elements in the map.

If performance is lost on alot of loops adding elements to a map an option would also be for the compiler to automatically initiate a growth to more than 2 times the map size if the two be added elements in that loop are more than double the size of the map. This wouldnt need an explicit hint by the programmer.

More details can be added in an extra comment or in the top comment.