build: adopt Go 1.17 as bootstrap toolchain for Go 1.20

[rsc]

I propose that starting with the Go 1.18 dev cycle, we require Go 1.16 as the Go bootstrap version (instead of Go 1.4).

When we switched to writing the Go compiler in itself, I wrote two proposals related to the bootstrap process.

The original proposal, https://golang.org/s/go13compiler (December 2013), was about the overall process of converting the compiler to Go.
In that proposal, I wrote:

With a Go compiler written in Go, there must be a plan for bootstrapping from scratch. The rule we plan to adopt is that the Go 1.3 compiler must compile using Go 1.2, Go 1.4 must compile using Go 1.3, and so on. Then there is a clear path to generating current binaries: build the Go 1.2 toolchain (written in C), use it to build the Go 1.3 toolchain, and so on. There will be a shell script to do this; it will take CPU time but not human time. The bootstrapping only needs to be done once per machine; the Go 1.x binaries can be kept in a known location and reused each time all.bash is run during the development of Go 1.(x+1).

Obviously, this bootstrapping path scales poorly over time. Before too many releases have gone by, it may make sense to write a back end for the compiler that generates C code. The code need not be efficient or readable, just correct. That C version would be checked in, just as today we check in the y.tab.c file generated by yacc. The bootstrap sequence would invoke gcc on that C code to build a bootstrap compiler, and the bootstrap compiler would be used to build the real compiler. Like in the other scheme, the bootstrap compiler binary can be kept in a known location and reused (not rebuilt) each time all.bash is run.

This was all fairly hypothetical, and I certainly no longer believe it makes any sense to write a C back end for the Go compiler. (In fact, for the windows/arm64 port I did, I now have a working Go toolchain but still don't know what C compiler I'm supposed to use!)

The final proposal, https://golang.org/s/go15bootstrap (January 2015), simplified the process from an iterative one to hard-coding Go 1.4 as the bootstrap toolchain:

To build Go 1.x, for x ≥ 5, it will be necessary to have Go 1.4 (or newer) installed already, in $GOROOT_BOOTSTRAP. The default value of $GOROOT_BOOTSTRAP is $HOME/go1.4. In general we'll keep using Go 1.4 as the bootstrap base version for as long as possible. The toolchain proper (compiler, assemblers, linkers) will need to be buildable with Go 1.4, whether by restricting their feature use to what is in Go 1.4 or by using build tags.

This was an important simplification, especially for people packaging Go for other systems. That decision has served us well.

But it has now been over six years since Go 1.4. Much has happened in the world of Go, and many bugs have been fixed. Many of the systems Go runs on today aren't supported by Go 1.4 (including darwin/arm64 for M1 Macs). Those are using newer toolchains to bootstrap, and the other systems could too. At a higher level, Go is far more mature and widely available now than it was in the Go 1.4 era. There are tons of available binary distributions to use for bootstrapping.

I propose that Go 1.17 be the last version of Go requiring Go 1.4 for bootstrapping, and that Go 1.18 require Go 1.16 for bootstrapping.
Go 1.16 would remain the bootstrap version for the next few years at least.

Why not Go 1.15?

• Go 1.16 added //go:build support. We don't want the compiler to have to keep using +build even once the Go world has moved on. (Edit: this is only half true: see build: adopt Go 1.17 as bootstrap toolchain for Go 1.20 #44505 (comment).)

Why not Go 1.17?

• There's no compelling feature being added in Go 1.17.
• Using Go 1.16 as the new bootstrap toolchain gives packagers the chance to test the changes they'd need to make using Go 1.17,
  which would still work with Go 1.4.
• Go 1.17 will be the first release to use the new register-based ABI, which will likely have some lingering bugs that take a while to shake out.
  (See next section.)

Why not Go 1.18?

• Go 1.18 will contain the initial implementation of generics, which is going to require many many new lines of code, and certainly some of them will contain subtle bugs. If we adopt Go 1.18 as the bootstrap toolchain, we will be stuck working around those bugs for many years to come. Better to lock in the last release before all that churn as the version that will need to hold up for the long term.

Why not a quickly rolling version?

As noted above, I think it has served us well to have a fixed version required to build Go, as opposed to an automatically sliding version as originally envisioned. Packagers benefit from not having to update their package-building scripts to provide a different environment to each new Go release.

At the same time, bumping the version forward every five years or so lets us take advantage of newer Go capabilities and ports and to let us retire old compatibility shims (standard library packages like sort are carrying various +build'ed files to keep them building with Go 1.4). Modern C compilers are not written in pre-ANSI C.

What about a slow-rolling version?

That's essentially what this proposal would establish as our practice, although without a specific timeline.

The next obvious entry in the sequence after Go 1.4 and Go 1.16 is Go 1.256, followed by Go 1.65536.
(Or perhaps that is not quite the right pattern to establish.)

Using dates instead, assuming we switch to Go 1.16 in Go 1.18 (Feb 2022), it seems reasonable to me to revisit the bootstrap version four years later, which at our current release cycle would mean using Go 1.24 (Feb 2025) for Go 1.26 (Feb 2026).

But this proposal is not about establishing the sequence, which would depend on many other factors.

It is only about picking Go 1.16 as the bootstrap version starting in Go 1.18.

[rsc]

Actually, I confused myself while writing this.
Go 1.16 does not know how to read //go:build lines,
but it does know to reject files that contain //go:build without // +build.
And all future gofmts will know to sync (≠ insert) // +build lines alongside //go:build
So that's enough for the bootstrap toolchain.

[mdempsky]

Being able to assume a recent compiler for bootstrapping would be really nice for not having to remember all the quirks of Go 1.4, or needing to worry about working around long-fixed issues.

FWIW, OpenJDK appears to require the immediately previous release for bootstrapping (see https://openjdk.java.net/groups/build/doc/building.html, "Build JDK Requirements"), and also has a 6-month release cycle (https://en.wikipedia.org/wiki/OpenJDK#OpenJDK_versions), but building OpenJDK takes way longer than running make.bash does. I don't think we need to be nearly as aggressive at uprev'ing Go bootstrap versions, but I think it at least suggests end users could tolerate another uprev before 1.256. :)

[beoran]

This is a good idea, however, perhaps it would make sense to additionaly declare that the bootstrap version of Go is also going to be the long term support or "LTS" version, which keeps receiving backports of bugfixes for 4 years (or whatever the support period for the bootstrap version is going to be). In some organizations and for some developments, having a stable LTS version that keeps receiving bug fixes is extremely valuable.

[tianon]

FWIW, the level of aggressive bootstrap version bumps OpenJDK does are a big hurdle towards having recent OpenJDK packaged for backported distribution releases (and require a lot of extra coordination / "fancy footwork" from maintainers to coordinate or "hack" around).

As a concrete example for Go, this could make adding new architectures more difficult for a distribution until/unless gccgo gets updated to 1.16 (because gccgo is often used as a bootstrap-builder in order to onboard new architectures without having to do something drastic like uploading an old version of Go with the newer version number and "stepping" upwards until it's supported, which is really disruptive for the other architecture too -- the other alternative is cross-building, which has challenges of its own).

IMO this would be easier to understand/stomach if the proposed version were one that's already supported by at least the most recent gccgo release (which I believe in GCC 10 is 1.14?)

[ianlancetaylor]

Just a note that GCC 11 will support Go 1.16.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group