Skip to content

cmd/cgo: document how to cross-compile from darwin/amd64 to darwin/arm64 with cgo #44112

Not planned
@briandealwis

Description

@briandealwis

Attempting to use cgo for cross-compiling a Go app for darwin/arm64 from darwin/amd64 is not entirely straightforward for those not familiar with Apple's toolchain when compiling from macOS 10.15 or earlier.

What version of Go are you using (go version)?

go 1.16rc1

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go1.16rc1 env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/bdealwis/Library/Caches/go-build"
GOENV="/Users/bdealwis/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/bdealwis/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/bdealwis/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/bdealwis/sdk/go1.16rc1"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/bdealwis/sdk/go1.16rc1/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16rc1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/bdealwis/Projects/Skaffold/repo-skaffold/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/jp/fyjblw1x6k71gyjnz3smzbjr00kl65/T/go-build1743423011=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I was trying to build Skaffold for darwin/arm64. We normally build Skaffold with cgo to take advantage of github.com/rjeczalik/notify's bindings to native file-watching APIs like macOS's FSEvents.

I understood that XCode 12.2 and later support cross-compilation between amd64 and arm64, I thought this would be straightforward. But my attempts failed with odd errors:

$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 \
    go1.16rc1 build ./cmd/skaffold 
# runtime/cgo
In file included from _cgo_export.c:3:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdlib.h:62:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
In file included from _cgo_export.c:3:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdlib.h:64:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
...

It turns out that cross-compiling for arm64 from amd64 requires using a macOS SDK 11.x. As I was building from a machine running macOS 10.15, the default SDK used for compilation was 10.15. It was not clear how to specify an 11.x SDK. From what I found online, it seemed like I should be able to build by specifying CC='clang --target arm64-apple-macos11, or CC='clang -isysroot to point to a macOS 11.x SDK, but these failed in the similar fashion:

$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 \
    CC='clang -isystem /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk` \
    go1.16rc1 build -o out/skaffold-darwin-arm64 ./cmd/skaffold
# runtime/cgo
In file included from _cgo_export.c:3:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdlib.h:62:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
...

After much more digging, it turns out the best way is to set the SDKROOT environment variable to point to the SDK. And to use xcrun to find the newest SDK installed:

$ CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 \
    SDKROOT=$(xcrun --sdk macosx --show-sdk-path) \
    go1.16rc1 build -o out/skaffold-darwin-arm64 ./cmd/skaffold

The --sdk macosx chooses the MacOSX.sdk, which is normally linked to the latest SDK.

Activity

seankhliao

seankhliao commented on Feb 5, 2021

@seankhliao
Member

feel free to add a page to the wiki

changed the title [-]Document how to cross-compile from darwin/amd64 to darwin/arm64 with cgo[/-] [+]cmd/cgo: document how to cross-compile from darwin/amd64 to darwin/arm64 with cgo[/+] on Feb 5, 2021
bcmills

bcmills commented on Feb 5, 2021

@bcmills
Contributor
cherrymui

cherrymui commented on Feb 5, 2021

@cherrymui
Member

I think the general question is how to cross compile with cgo, which boils down to how to cross compile C code. It is not specific to darwin. Basically, it needs a C cross compiler, and set CC and/or CGO_CFLAGS/CGO_LDFLAGS accordingly. Sometimes it is relatively easy, and sometimes it requires a nontrivial amount of work.

I don't think we want do document about how to cross compile C. Are there existing documentation about that? Maybe we can link to that, and add the part of setting CC/CGO_CFLAGS/etc.

briandealwis

briandealwis commented on Feb 6, 2021

@briandealwis
Author

I thought it was worth calling out darwin/arm64 as most build systems do not yet support Apple M1 and are on macOS 10.15 or earlier. One of the wiki pages seems fine? Perhaps https://github.com/golang/go/wiki/Darwin?

added
FeatureRequestIssues asking for a new feature that does not need a proposal.
on Feb 8, 2021
added this to the Backlog milestone on Feb 8, 2021
added
NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.
on Feb 8, 2021
MuweiHe

MuweiHe commented on Oct 8, 2021

@MuweiHe

I chose MacOSX.sdk but still got unsupported arch errors:

In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:62:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture

16 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    DocumentationIssues describing a change to documentation.FeatureRequestIssues asking for a new feature that does not need a proposal.FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.WaitingForInfoIssue is not actionable because of missing required information, which needs to be provided.compiler/runtimeIssues related to the Go compiler and/or runtime.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @cytown@briandealwis@dmitshur@bcmills@gopherbot

        Issue actions

          cmd/cgo: document how to cross-compile from darwin/amd64 to darwin/arm64 with cgo · Issue #44112 · golang/go