proposal: spec: add integer types with explicit overflow behavior, and remove unchecked operations on built-in integers

[bcmills]

This proposal is intended to accomplish the same goals as #19624, but in a way that complies with the migration strategy outlined in https://github.com/golang/proposal/blob/master/design/28221-go2-transitions.md#language-redefinitions.

See #19624 for background.

Summary
This proposal would:

1. Add three new packages: checked, wrapped, and unbounded, containing integer types with distinguished semantics to be defined in the language spec.
2. Add the , ok form (as described in proposal: spec: change all int types to panic on wraparound, overflow #19624) to explicitly check for overflow in expressions of any bounded numeric type.
3. Remove the ability to perform unchecked arithmetic operations on the predeclared integer types.

New packages

The three new packages each provide a set of integer types:

// Package checked defines integer types whose arithmetic operations and conversions
// panic on overflow.
// Bitwise operations and logical shifts will not trigger a panic.
package checked

type Uint8 <builtin>
type Uint16 <builtin>
[…]

type Int8 <builtin>
[…]

type Byte = Uint8
type Rune <builtin>

type Int <builtin>
type Uint <builtin>
type Uintptr <builtin>

// Package wrapped defines integer types whose arithmetic operations and conversions wrap
// using two's-complement arithmetic.
package wrapped

type Uint8 <builtin>
[…]

type Int8 <builtin>
[…]

type Byte = Uint8
type Rune <builtin>

type Int <builtin>
type Uint <builtin>
type Uintptr <builtin>

// Package unbounded defines an arbitrary-precision integer type with unbounded range.
// Unbounded types do not support bitwise XOR, complement, or clear operations.
package unbounded

type Int <builtin>

Defined types

Defined types that have checked, wrapped, or unbounded types as their underlying type have the same operations and behavior as the underlying type.

Defined types that have builtin integer types as their underlying type have the same behavior as the underlying type, except that they are not mutually-assignable with checked or wrapped types.

Checked assignment

A checked assignment uses the form x, ok = <expression> or x, ok := <expression>, where <expression> can comprise any number of arithmetic, bitwise, logical, and/or conversion operations yielding a checked, wrapped, unbounded, user-defined, or builtin integer type. The operations in a checked assignment do not panic even if they involve checked or builtin integer types. The ok result, which may be assigned to any boolean type, indicates whether any arithmetic operation or conversion within the expression overflowed. The x result has the type of <expression> and a value computed using two's-complement wrapping.

Bitwise operations and logical shifts do not overflow, and therefore do not set the ok result. (Recall that signed shifts are defined to be arithmetic, while unsigned shifts are defined to be logical.)

Unchecked arithmetic operations on builtin integer types are a compile-time error.

var x int32 = 1<<31 - 1
y := x + 1  // compile-time error: `x + 1` is not checked for overflow

var x int32 = 1<<31 - 1
y, ok := x + 1  // y = -2147483648; ok = false

var x checked.Int32 = 1<<31 - 1
y := x + 1  // run-time panic: `x + 1` overflows

var x checked.Int32 = 1<<31 - 1
y, ok := x + 1  // y = -2147483648; ok = false

var x wrapped.Int32 = 1<<31 - 1
y := x + 1  // y = -2147483648

var x wrapped.Int32 = 1<<31 - 1
y, ok := x + 1  // y = -2147483648; ok = false

var x int32 = 1<<30
y, ok := x<<1  // y = -2147483648; ok = false
// signed shift is arithmetic, and shifting into the sign bit overflows

var x uint32 = 1<<31
y, ok := x<<1  // y = 0; ok = true
// unsigned shift is logical, and by definition cannot overflow

Conversion

Any integer type can be explicitly converted to a checked, wrapped, or unbounded integer type.
A checked, wrapped, or unbounded integer type can be converted to a builtin type only if either the conversion is checked or the destination type can represent all possible values of the source type.

var x checked.Int32 = 1<<31 - 1
var y = int32(x)  // y = 2147483647

var x checked.Uint32 = 1<<31 - 1
var y = int32(x)  // compile-time error: conversion from checked.Uint32 may overflow int32

var x checked.Uint32 = 1<<31
y, _ = int32(x)  // y = -2147483648

An unchecked conversion to a checked type panics if the value cannot be represented in the destination type.

var x int64 = 1<<31
y := checked.Int32(x)  // run-time panic: `x` overflows checked.Int32

var x int64 = 1<<31
y, ok := checked.Int32(x)  // y = -2147483648; ok = false

A conversion to a wrapped type wraps if the value cannot be represented in the destination type, even if the operand is of a larger checked type.

var x checked.Int64 = 1<<31
y := wrapped.Int32(x)  // y = -2147483648

A conversion to unbounded.Int from any integer type always succeeds. The conversion is applied after the operand is fully evaluated.

var x int64 = 1<<31
y := unbounded.Int(x)  // y = 2147483648

var x checked.Int32 = 1<<31-1
y := unbounded.Int(x+1)  // run-time panic: `x+1` overflows checked.Int32

var x wrapped.Int32 = 1<<31-1
y := unbounded.Int(x+1)  // y = -2147483648

var x checked.Int32 = 1<<31-1
y, ok := unbounded.Int(x+1)  // y = -2147483648; ok = false

Assignability

Each sized type in the checked and wrapped package is mutually assignable with the corresponding builtin sized type, but not with the type in the opposing package, nor with defined types of any underlying integer type.

var x wrapped.Int32 = 1<<31 - 1
var y int32
[…]
y = x + 1  // y = -2147483648

var x checked.Int32 = 1<<31-1
var y wrapped.Int32
[…]
y = x + 1  // compile-time error: checked.Int32 is not assignable to wrapped.Int32

type MyInt32 wrapped.Int32
var x wrapped.Int32 = 1<<31 - 1
var y MyInt32
[…]
y = x + 1  // compile-time error: wrapped.Int32 is not assignable to MyInt32

type MyInt32 int32
var x wrapped.Int32 = 1<<31 - 1
var y MyInt32
[…]
y = x + 1  // compile-time error: wrapped.Int32 is not assignable to MyInt32

var f func() int32
var g func(int32)
var x checked.Int32
x = f()
g(x + 1)  // ok: x+1 is checked, then passed to g as an int32

This allows functions to perform operations on checked or wrapped types, but to expose and use the corresponding builtin types at API boundaries (with less syntactic overhead for all involved).

Arithmetic operators

The type of an arithmetic expression depends on its operands. If both operands are of checked, wrapped, unbounded, or defined types, then they must have exactly the same type. If one operand is a checked, wrapped, or unbounded integer or a defined type with one of those types as its underlying type, and the other is either a builtin integer or untyped constant assignable to the first, then the result of the operation is the checked, wrapped, unbounded, or defined type.

If both operands are of a builtin integer type or a defined type with a builtin integer as its underlying type, the expression must be a checked assignment, bitwise operator, or logical shift, and its result is the same type as the operands.

Constants

An untyped integer constant can be assigned to any checked, wrapped, or unbounded integer type that can represent its value.

Unfortunately, in order to comply with https://github.com/golang/proposal/blob/master/design/28221-go2-transitions.md#language-changes, the inferred type of a variable initialized from an untyped constant must remain the built-in int.

[bcmills]

Note that part (1) would be trivial if we had operator overloading, and part (3) could really just be a vet check.

The assignability rule could perhaps be eliminated from the proposal without causing too much damage. That would make the code much more verbose, but not fundamentally alter its structure.

Similarly, the conversion check might be implementable using generic constructor functions, and can be implemented (inefficiently and without type-safety) using reflection.

IMO, the interesting part of this proposal is part (2), the aggregation of overflow checks to the whole expression. That would otherwise require constructing an overflow-tracking object, using its methods in place of the existing arithmetic expressions, and remembering to check its result at the end — a lot of syntax, and much more error-prone than a regular old , ok.

[ericlagergren]

How come the unbounded package only has a signed integer? An unsigned integer would be beneficial for some things I have in mind.

[bcmills]

@ericlagergren An unbounded signed integer type can represent all of the unsigned values too, and is closed under subtraction.

In contrast, an unsigned type carries the risk of overflow: what would the behavior be for unbounded.Uint(0) - unbounded.Uint(1)? The whole point of the unbounded package is that its behavior on overflow is “not possible by definition”.

[josharian]

I’ll admit I haven’t read this too carefully, but I hope you’ll permit me some questions anyway:

• Why have wrapped? How does it differ from what we have now?
• In the same vein, what do you think about adding saturating? (I think this was discussed elsewhere as well.)
• I see the value in package checked as a namespace, but ISTM we could dispense with unbounded in favor of a new predeclared type, like integer or infinint (joking).

[bcmills]

• Why have wrapped? How does it differ from what we have now?

The wrapped types have the same semantics as the builtin types today.

The wrapped package exists so that we can remove the arithmetic operators from the built-in types.
Otherwise, it's much too easy to forget to convert a value to checked and end up with (undiagnosed) wrapping behavior where you really meant for the operations to never overflow.

[bcmills]

• In the same vein, what do you think about adding saturating? (I think this was discussed elsewhere as well.)

I think that would be fine. I don't know of many cases where it's actually useful, but it doesn't seem actively harmful either.

[bcmills]

• I see the value in package checked as a namespace, but ISTM we could dispense with unbounded in favor of a new predeclared type, like integer or infinint (joking).

Indeed. I don't feel strongly about that either way — I'm happy to go with whatever makes the proposal more palatable to the folks making the final decision. 🙂