proposal: Go 2: add &range in for statements

[pcostanza]

When using the for statement to loop over a slice, it is sometimes useful or necessary to take the address of each slice entry rather than a value copy. However, doing this is a bit inconvenient and could be made more readable.

Loop over a slice by value:

for _, v := range s {
   ...
}

Loop over a slice by address:

for i := range s {
  p := &s[i]
  ...
}

Proposed syntactic extension to loop over a slice by address:

for _, p := &range s {
  ...
}

I believe the suggested extension is more readable and makes the intent clearer.

[faiface]

Why do you need to take an address? What's wrong with this?

for i := range a {
    a[i].DoSomething()
}

[mvdan]

@pcostanza avoiding writing a few characters or a line (in this case, p := &s[i]) is very unlikely to be argument enough to warrant a language change that adds complexity to the spec. Do you have any other argument in its favor?

It's also worth noting that Go always passes by value - if you want to modify something, you pass its pointer. Adding &range reminds me a bit of & arguments in C++. So you would also be breaking that rule, which I believe is a part of the language's simplicity.

[pcostanza]

avoiding writing a few characters or a line (in this case, p := &s[i]) is very unlikely to be argument enough to warrant a language change that adds complexity to the spec. Do you have any other argument in its favor?

There are situations where you need to refactor your code, and one potential refactoring is to store structs as values in slices rather than pointers to structs. If then, in a corresponding for loop, you need to refer to the single entries by address rather than by value, you would have to move from (1) to (2):

// (1)
for _, p := range s {
   ...
}

// (2)
for i := range s {
   p := &s[i]
   ...
}

With the proposed change, such refactorings would be easier and result in more elegant code IMHO. The alternative you're suggesting is to replace each occurrence of p with s[i], which is more work and more error-prone for larger pieces of code. (That's a case that actually happened to me, which is why I'm proposing this change.)

It's also worth noting that Go always passes by value - if you want to modify something, you pass its pointer. Adding &range reminds me a bit of & arguments in C++. So you would also be breaking that rule, which I believe is a part of the language's simplicity.

I don't particularly care about the concrete syntax. The idea of &range is to suggest that the address of each successive slice element is taken, and the & operator already exists in Go, so this seems a straightforward syntactic change to me, without having to refer to C++.

Other possible syntaxes could be *range, or, for example:

for _, *p := range s {
   ...
}

I'd guess that's probably more problematic though from a language specification point of view, though. Again, I don't care about the particular syntax, but i think the feature could improve the language and make it a bit more orthogonal.

(The for _, v := range s {... v ...} form is also not necessary and could have been reduced to for i := range s {... s[i] ...} as well, but it seems to me that whoever designed the for statement thought the first form is aesthetically more pleasing. Maybe I'm wrong.)

[mvdan]

Note that my point about passing by value has little to do with syntax. With the current state of things, you have to explicitly take the address via &s[i]. With this change, it would be more subtle as you would introduce a language feature that would implicitly take addresses of slice/array elements. This is what I meant by "breaking the passing by value rule".

I see your point about the extra verbosity, but I'm still not (personally) convinced.

Could you elaborate on your error-prone point?

[cespare]

Related: #8346

[cespare]

I really like this suggestion. I've worked on quite a bit of code that uses []T rather than []*T for performance reasons and it's awkward to need to explicitly index into the slice rather than being able to use an iteration variable. I also agree that it's error-prone: when you're using such slices it's easy to accidentally write for i, v := range s { and make copies unintentionally.

[pcostanza]

Note that my point about passing by value has little to do with syntax. With the current state of things, you have to explicitly take the address via &s[i]. With this change, it would be more subtle as you would introduce a language feature that would implicitly take addresses of slice/array elements. This is what I meant by "breaking the passing by value rule".

OK, now I see what you mean. I still disagree with you, though. In C++, you indeed regularly encounter situations where the address of an object is implicitly taken, just by virtue of calling a function that takes reference parameters. There is no syntactic clue whatsoever at the call site that this is happening, and I fully understand that you want to avoid adding features that could lead to similar situations.

However, my proposal doesn't do this: The expression that specifies which values addresses are taken of is right next to the &range form, which is very explicit (and not more implicit than what the already existing range form does). I don't think this gives rise to any surprises when reading source code.

Could you elaborate on your error-prone point?

Nothing special here: The more code you need to touch in a refactoring, the more opportunities there are for making mistakes, like typing j instead of i; the i accidentally being shadowed by another "more local" i; and so on. Cespare also has a good point, which I admit I haven't had in mind myself.

[griesemer]

I have sentiment for the general idea; the situation where one wants to iterate over the addresses rather than the elements of an array/slice does occur fairly often, and it's unsatisfying that that code has to be quite asymmetric compared to the case where one iterates over the elements. A couple of observations:

1. The alternative suggested notation

for _, *p := range s {
   ...
}

(suggested in #21537 (comment)) would be problematic because the same notation using a = rather than := would be a currently valid iteration over the elements of s that stores the elements in *p. Having the meaning so different simply because there's an extra : seems problematic.

2. The proposal falls apart for iterations over maps because it's not permitted to take the address of a map element (even outside any iteration). Thus, one would have to exclude maps from this mechanism, which is unsatisfying.

I don't know if 2) would be a show stopper, but at the very least it's going to be a concern that speaks against the proposal.

[pcostanza]

The proposal falls apart for iterations over maps because it's not permitted to take the address of a map element (even outside any iteration). Thus, one would have to exclude maps from this mechanism, which is unsatisfying.

I believe this is only an issue when refactoring code from using a slice representation to a map representation, or vice versa, for a particular use case. However, such a refactoring requires much deeper changes than adapting a few for loops, so there is no need to create the illusion it's an easy change in the for loops. (It would be nicer, indeed, but won't be the major refactoring challenge.)

Moving from []*someType to []someType or vice versa is more common, and doesn't require the deep changes that moving between maps and slices require.

[MichaelTJones]

I like the idea and spoke of it in the mailing list a few years ago. one of the proposals from another person on that thread (Gri?) was a three-value range, with index, value, and pointer to value as the result. In this scenario, the existing two-value case is extended but the third value could be generally ignored. When the pointer was wanted, "i,_,p := range ..." gives it to you.

that seemed ok to me.