proposal: spec: immutable type qualifier

[romshark]

This issue describes a language feature proposal to Immutable Types. It targets the current Go 1.x (> 1.11) language specification and doesn't violate the Go 1 compatibility promise. It also describes an even better approach to immutability for a hypothetical, backward-incompatible Go 2 language specification.

The linked Design Document describes the entire proposal in full detail, including the current problems, the benefits, the proposed changes, code examples and the FAQ.

Updates

• October 7th: This proposal is approaching its second revision addressing major flaws such as const-poisoning, verbosity, const-keyword overloading and others.

Introduction

Immutability is a technique used to prevent mutable shared state, which is a very common source of bugs, especially in concurrent environments, and can be achieved through the concept of immutable types.

Bugs caused by mutable shared state are not only hard to find and fix, but they're also hard to even identify. Such kind of problems can be avoided by systematically limiting the mutability of certain objects in the code. But a Go 1.x developer's current approach to immutability is manual copying, which lowers runtime performance, code readability, and safety. Copying-based immutability makes code verbose, imprecise and ambiguous because the intentions of the code author are never clear. Documentation can be rather misleading and doesn't solve the problems either.

Immutable Types in Go 1.x

Immutable types can help achieve this goal more elegantly improving the safety, readability, and expressiveness of the code. They're based on 5 fundamental rules:

• I. Each and every type has an immutable counterpart.
• II. Assignments to objects of an immutable type are illegal.
• III. Calls to mutating methods (methods with a mutable receiver type) on objects of an immutable type are illegal.
• IV. Mutable types can be cast to their immutable counterparts, but not the other way around.
• V. Immutable interface methods must be implemented by a method with an immutable receiver type.

These rules can be enforced by making the compiler scan all objects of immutable types for illegal modification attempts, such as assignments and calls to mutating methods and fail the compilation. The compiler would also need to check, whether types correctly implement immutable interface methods.

To prevent breaking Go 1.x compatibility this document describes a backward-compatible approach to adding support for immutable types by overloading the const keyword (see here for more details) to act as an immutable type qualifier.

Immutable types can be used for:

• immutable fields
• immutable methods
• immutable arguments
• immutable return values
• immutable variables
• immutable interfaces
• immutable reference types and containers
• immutable package-scope variables

Immutable Types in Go 2.x

Ideally, a safe programming language should enforce immutability by default where all types are immutable unless they're explicitly qualified as mutable because forgetting to make an object immutable is easier, than accidentally making it mutable. But this concept would require significant,
backward-incompatible language changes breaking existing Go 1.x code. Thus such an approach to immutability would only be possible in a new backward-incompatible Go 2.x language specification.

Related Proposals

This proposal is somewhat related to:

• proposal: spec: add read-only slices and maps as function arguments #20443
• proposal: spec: read-only types #22876
• proposal: frozen slices / arrays / maps and derivatives (immutable subscript-able objects) #22048

Detailed comparisons to other proposals are described in the design document, section 5..

---

Please feel free to file issues and pull requests, become a stargazer,
contact me directly at roman.scharkov@gmail.com and join the conversation on Slack Gophers (@romshark), the international and the russian Telegram groups, as well as the original golangbridge, reddit and hackernews posts! Thank you!

[dsnet]

Nice document; clearly you spent a while on it. I only briefly glanced over it.

Copies are the only way to achieve immutability in Go 1.x, but copies inevitably degrade runtime performance. This dilemma encourages Go 1.x developers to either write unsafe mutable APIs when targeting optimal runtime performance or safe but slow and copy-code bloated ones.

Not exactly the only way. An alternative approach is to have an opaque type with only exported methods that provide read-only access, which is how reflect.Type achieves immutability. The v2 protobuf reflection API also takes this approach. It has other downsides (like needing to manually create methods for each read-only operation), but pointing out that there are non-copy approaches to immutability.

[networkimprov]

I like the idea of mut more than const, but I think similar effect could be achieved with optional naming conventions (field_m, mField, field$ :-) and a go-vet switch.

Also forgetting to use const somewhere can cause havoc down the road...

EDIT: Related: #21130, #6386

[kirillDanshin]

@networkimprov please no $ in names, I don't think we need to start another PHP again

[ianlancetaylor]

As far as I can tell in my initial skim, this doesn't solve the memchr problem. In C the standard library memchr function, which returns a pointer to the first occurence of a character in a string, is defined as char *memchar(const char *, char). The problem is that in terms of types, if memchr is passed a const char * it should return a const char *, and if memchr is passed a char * it should return a char *. That is, it should preserve the const-ness of its first argument. But there is no way to write that in C. And I don't see how to write that in your proposal.

Other comments:

• Is there any difference between this proposal and the use of const as a type qualifier in C, other than the logical extension to interfaces?
• Look up "const-poisoning."
• Pedantically, I don't particularly like using the word "immutable" to describe the parameter to func F(const []int). That slice is not immutable; all that declaration says is that F will not change it. This is particularly clear if you write func F(s1 const []int, s2 []int) and then call it as F(s, s). You can't say that s1 is immutable within F, because if s2 changes then s1 will change.
• Do we really have to worry about immutable containers of mutable values? Yes, that comes up once in a while, but it is often enough to make it worth writing const [] const T?

[romshark]

@dsnet
getters/setters are doing just that: they copy stuff from the inner scope of the struct. You
don't want to copy everything every time, and you certainly don't want to do it manually. Writing setters, getters, cloners just for the sake of ensuring immutability is not only quite tedious but also very error-prone due to pointer aliasing, which is the scariest part actually. Copy-code tends to be rather complicated in Go, one wrong copy (like copying a pointer, or naively copying a reference type such as a slice) and you've introduced aliasing that could have terrific, non-obvious consequences. With immutable types though, having read-only aliasing is just fine because there's no mutable shared state.

Currently, the safest way of avoiding manual copying of large structs are interfaces. You could define 2 interfaces where one of them lacks the mutating methods and return interfaces from the getters only. This is an "okay" solution, but it doesn't solve internal mutability problems. In big open source projects many people are working on the code, intentions must be unambiguous, clear and precise, which they're currently not. Can you automatically ensure that the methods implementing the read-only interface do not mutate the object for sure, even after merging a pull request from an external developer who's not fully aware of your intentions? You can't! You'll have to write proper unit tests and carefully analyze each and every commit! With immutable types you declare your interface methods immutable and you can be 100% sure that any implementation of it trying to mutate the object will fail the compilation. Apart from that, interfaces aren't free, they do have a slight runtime cost due to indirection, so having an option to avoid them for performance reasons while still preserving safety is a good thing!

I always prefer to solve these kinds of problems declaratively. I declare what is mutable/immutable while the compiler does all the dirty work of making sure neither me, nor my coworkers, nor the open source contributors sending in their pull requests shoot themselves in the foot introducing bugs. Isn't this the way compiled languages should make our lives easier?

[romshark]

@networkimprov
As section 3. of the design document clearly states: immutability by default and explicit mutability qualification through mut is preferable, but would only be possible in a backward-incompatible Go 2.x specification, which is not to be expected any time soon (AFAIK, the "Go 2" they're advertising is rather a Go 1.13+ because the folks at Google aren't big fans of breaking compatibility as it seems).

Naming conventions would break backward-compatibility. Old Go 1.x code could either stop compiling or fail at linting, which is unacceptable. This proposal aspires to preserve backward-compatibility at all cost. There's also a somewhat related question in the FAQ by the way.

[randall77]

You may want to read Russ' evaluation of a read-only slices proposal. It contains a lot of the issues that this proposal should grapple with.

[dsnet]

getters/setters are doing just that: they copy stuff from the inner scope of the struct.

But they don't have to. If the inner field is a composite type, the getter can return an opaque type that internally holds a pointer to the composite type and only provides exported read-only getter methods.

Writing setters, getters, cloners just for the sake of ensuring immutability is not only quite tedious but also very error-prone due to pointer aliasing, which is the scariest part actually

I've written several immutable APIs in this way. I absolutely agree that it is tedious, but I personally don't think it was "very error-prone" from the perspective of the API author. Pointer aliasing is not inherently the problem; it is problematic if a pointer to a non-opaque type leaks to the public API. However, I find it relatively straight-forward to review the public API and reason that it doesn't violate immutability.

Can you automatically ensure that the methods implementing the read-only interface do not mutate the object for sure, even after merging a pull request from an external developer who's not fully aware of your intentions? You can't!

Since read-only APIs are usually just getters, they are not terribly complicated such that you would accidentally mutate the object (e.g., it is not hard to review this and reason it is read-only).

In big open source projects many people are working on the code, intentions must be unambiguous, clear and precise, which they're currently not.

An opaque read-only API does make the intention clear. The lack of any setter methods is a clear signal that the user should not (and cannot) mutate anything.

Apart from that, interfaces aren't free, they do have a slight runtime cost due to indirection, so having an option to avoid them for performance reasons

Interfaces are one such implementation, but it doesn't have to be. It can be a concrete type too:

type MutableStruct struct {
    Field int
    ...
}
type ImmutableStruct struct { p *MutableStruct }
func (p ImmutableStruct) GetField() int { return p.p.Field }

There is practically no runtime cost to this as the compiler can inline all the getters as if they were nested field accesses (or slice indexes, map lookups, etc).

---

I am bringing the technique up not as an end-all alternative to your proposal, but more so to counter the claim that "copies are the only way to achieve immutability ... [which] degrades runtime performance". It is a legitimate approach taken today to address this problem, which the proposal seems to gloss over.

I agree that there are disadvantages to opaque APIs with read-only getters (especially with regard to their tediousness and perhaps the lack of implicit casting), but I think it would help the case of a proposal trying to add immutability to acknowledge techniques done today to work around the problem and show that the benefit of adding immutability outweighs the cost (e.g., complexity in type system and the "const poisoning" mentioned earlier).

[networkimprov]

Naming conventions would break backward-compatibility.

Above I suggested a go-vet switch to support a naming convention. Such a convention would be optional, permanently. As would a mut keyword. Lots of folks don't want to code that way.

There has been plenty of discussion about const-ness over the years, yet the two priorities for Go2 are error handling & generics, and code for them presumably won't land for a couple years (there is no defined schedule as yet).

[alvaroloes]

I love how well done the proposal is. Thank you for your hard work on this.

I have just one thought that I would like to share:
It is stated in the proposal that we can't cast an immutable var to a mutable one, as we could break the immutability. The opposite, casting a mutable var to an immutable one, can be done, which makes sense. However, in this case, we can break immutability. I know that you talk about this in section 4.5. How are constants different from immutable types?, but this could lead to very subtle and unexpected situations.

For example, let's say we have this Slice type:

type Slice []int

func (s *Slice) Add(elem const int) {
    *s = append(*s, elem)
} 

func (s const Slice) ImmutableVersion() const [] const int {
    return s
}

And then we use it like this:

slice := Slice{1,2,3,4}
immutableVersion := slice.ImmutableVersion()
// Now immutableVersion = {1,2,3,4}
slice.Add(5)
// Now immutableVersion = {1,2,3,4,5} It has changed

This behavior could be unexpected and lead to confusion, as you were guaranteed by the type system that the var immutableVersion was immutable.
This can be even worse with slices as, if the capacity is exceeded, append will allocate a new underlying array, what means that the immutableVersion won't be changed. So we don't really know if/when the immutableVersion will change.

This won't happen if the method ImmutableVersion() returns a copy.

Don't get me wrong! I love the proposal. I think it is the best one I have seen for immutability and I would like it to come true as soon as possible.

I just wanted to know the general opinion about the case I have posted.

Thanks!

[nemith]

In 2.6. Immutable Interface Methods I am not sure I understand why enforcement of mutability on the interface is important. This seems more like an implementation detail and could severely limit the usefulness of interfaces if abused too much. The answer in 4.7 doesn't make much sense to me.