Compile packages independently, link using LTO

[aykevl]

After #285, I'd like to move one step further: by compiling packages entirely separately and doing optimizations across packages using ThinLTO (or, optionally, full LTO if desired). The main benefit is that compilation should be a lot faster. Both with a cold cache (by parallelizing codegen) and with small changes to the source code (by reusing most packages). We should be able to get close to the speed of the go toolchain: TinyGo is currently a lot slower.

How we currently compile packages is as follows:

1. LLVM IR for packages is generated in parallel (and cached in ~/.cache/tinygo).
2. This IR is then merged together to create one huge LLVM module with the IR of all packages.
3. Some generic LLVM optimizations and TinyGo specific transformation passes are applied to all this combined IR.
4. The IR is then written to a temporary location, either as bitcode (for ThinLTO) or as an object file (for non-ThinLTO builds).
5. The linker (usually lld) is invoked to link everything together to generate an executable. In the ThinLTO case, lld creates an object file internally and caches it.

What I'd like to see:

1. LLVM IR for packages is generated in parallel (and cached in ~/.cache/tinygo), as before. TinyGo specific optimizations need to be done in this phase.
2. The linker (lld) is then used to link all bitcode files together, using ThinLTO.

This means there is no phase in which all IR is combined into one big module, which avoids the serial step that currently takes up most of the compile time.

This is no small task. We currently rely heavily on merging all packages together to perform some (required) optimization passes. These will need to be changed in some way to work well with LTO, by modifying them or replacing them with something else:

• We don't support ThinLTO yet for some targets (see Use ThinLTO on Windows #2867, darwin: add support for ThinLTO #2865 for example).
• Some targets need the AddGlobalsBitmap pass to be able to scan global variables in the GC mark phase. It should be possible to convert this to simply scanning the .data/.bss sections everywhere (see Use ThinLTO on Windows #2867, darwin: scan globals by reading MachO header #2869 for example).
• WebAssembly uses the MakeGCStackSlots pass. We need to make this pass run per package. In the future, the WebAssembly GC would be an alternative.
• Reflect information is currently processed for the whole program in LowerReflect. I've been working on a replacement in Refactor reflect package #2640 but it's going to cost something. In return, the compiler itself becomes easier to understand and new reflect features are easier to add.
• Interface method calls are lowered to direct calls in LowerInterfaces. We probably need to switch to vtable style interfaces. The optimizations that we currently do might be replaced by LLVM support for whole program devirtualization for C++.
• Interrupt handlers are currently combined in LowerInterrupts. This is done late so that unused interrupts can be optimized away. I'm not sure how to do this efficiently in any other way other than at this stage.

Of course, the resulting binaries should remain small. It's hard to avoid a slight increase, but hopefully the benefits of a simpler compiler and (much) faster compile times outweigh the downsides.

[niaow]

Another important issue is interp, where we may need to rethink a bit.

[aykevl]

@niaow yes. We currently run interp once per package and then again for the whole program. I imagine an initial implementation of this feature would be opt-in and only run interp per package (not for the whole program) which should work in practice with some increase in binary size. We can then look into improving this.
I didn't include it in the list as it isn't a true blocker like most of the other items are.

[aykevl]

ThinLTO is now supported on all architectures/platforms! 🎉
That's one more checkbox checked.

[aykevl]

The reflect refactor is in 🎉

[aykevl]

Managed to run some test programs with a new -lto=thin flag! See: #3489

The next hurdle is refactoring interface type asserts and interface method calls, which is something that will likely be necessary for full reflect support anyway (to implement things like .Method(n)).