proposal: cmd/link: add support for TLS variables to the toolchain

[mwhudson]

All platforms that support cgo use some kind of thread local storage to store g across cgo calls (Intel platforms always store g there).

On most systems that have support for TLS in the dynamic linker (e.g. linux/amd64), we use that support. On other systems (e.g. darwin/arm) there is a hack to determine the offset from the thread local base to the storage for a key created with pthread_key_create. There is fragile code spread over multiple places to do the right thing -- e.g. the linker does different things to a symbol called runtime·tlsg depending on the value of GOOS.

I want to make -buildmode=shared work on other architectures, and this requires changing details in how TLS works. Rather than adding further complications, I think it would be make things easier to understand if we could simply say that we wanted to use TLS on platforms that supported it.

My idea is that one could declare (in a .s file) a variable to be thread local, e.g:

GLOBL runtime·tlsg+0(SB), TLSBSS, $4

The only operations that would be supported on this variable would be MOVs to and from a register.

cmd/internal/obj would convert these operations, based on $GOOS and any flags passed, into the necessary instructions and relocations for a given platform, for example tls_arm.s might contain:

MOV runtime·tlsg(SB), R0

and this might, on Linux, with GOARM=7 but without any flags implying the code will end up in a shared library, become (ARM syntax, and using r12 as a scratch register):

mrc     15, 0, r0, cr13, cr0, {3}
ldr r12, [pc, #off]--.
ldr r0, [r0, r12]    |
...                      |
.word <------------------' <- reloc with Type=R_TLS_LE, Sym=runtime·tlsg pointing here

The linker when linking externally would turn the relocation into R_ARM_TLS_LE32 and when linking internally would allocate each references TLS symbol an offset from the base and process the relocation accordingly (and create a PT_TLS header). If the code was compiled with go tool compile -dynlink, a different sequence and relocation would be generated and the linker would turn the relocation into R_ARM_TLS_IE32 instead.

[ianlancetaylor]

SGTM

[mwhudson]

Thinking about this some more, I'm a bit uneasy about the need for the operations involving the TLS variable to use a scratch register (or two, in some cases, one of which must be R0 on ARM). I think on RISC (well, ARM & ARM64, not sure about POWER details but they're probably similar?) platforms the assembly should be something like this:

GLOBL runtime·tlsg+0(SB), TLSBSS, $8
....
        MOVQ TLS, R0
        MOVQ runtime.tlsg(SB), R1
        MOVQ R1(R0), R2

This is more or less the platform assembly for LE TLS access. Open questions: 1) do we make people spell "MOVQ TLS, R0" in the platform-specific way (e.g. MRS TPIDR_EL0, R0 on arm64) 2) For IE model code, does one translate the above assembly into IE model in cmd/internal/obj (easy, but magical) or do we have #ifdefs in the .s files (a bit easier to understand, I suspect).

I'd like to clean up the intel stuff a bit (particularly, the current IE model stuff is a bit magical, my fault), but that seems less urgent.

[mwhudson]

(I still have no real idea how to implement GD model code, needed for -buildmode=c-plugin)

[mwhudson]

Hm, after thinking a bit more and reading a few specs, for arm64 I think that it's practical to translate the above assembly into both LE and IE code, and for a bonus, it's also the right code to use on systems where tlsg is a variable. So I think I like this plan.

[rsc]

This seems like it merits a .md file.

[mwhudson]

Yeah, definitely. I wouldn't want to do this until #11374 is done one way or another.

[gopherbot]

CL https://golang.org/cl/13990 mentions this issue.

[adg]

Has there been any progress here since 2015?

[mwhudson]

I think it's basically been done by stealth. I actually thought I commented to this effect last week but clearly I didn't hit the button or something! I don't think there's any real value in keeping this open, so closing.