x/sys/windows: "Invalid access to memory location" when calling CreateAcceleratorTableW API

[mkch]

Go version

go version go1.21.5 windows/amd64

What operating system and processor architecture are you using (go env)?

set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\gopher\AppData\Local\go-build
set GOENV=C:\Users\gopher\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\gopher\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\gopher\go
set GOPRIVATE=
set GOPROXY=https://goproxy.cn,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLCHAIN=auto
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.21.5
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
go: stripping unprintable or unescapable characters from %"GOMOD"%
set GOMOD=C:\Users\gopher\a\b\code\testsyscallbug\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\Users\gopher\AppData\Local\Temp\go-build1499209731=/tmp/go-build -gno-record-gcc-switches

What did you do?

package main

import (
	"fmt"
	"unsafe"

	"golang.org/x/sys/windows"
)

type ACCEL struct {
	Virt byte
	Key  uint16
	Cmd  uint16
}

func CreateAcceleratorTableW(accel []ACCEL) (uintptr, error) {
	r, _, err := windows.NewLazySystemDLL("user32.dll").
		NewProc("CreateAcceleratorTableW").
		Call(uintptr(unsafe.Pointer(&accel[0])), uintptr(len(accel)))
	if r == 0 {
		return 0, err
	}
	return r, nil
}

func someAccels() []ACCEL {
	return []ACCEL{{83, 13, 100}}
}

func main() {
	var accels []ACCEL
	accels = append(accels, someAccels()...)
	h, err := CreateAcceleratorTableW(accels)
	fmt.Println(h, err)
}

What did you expect to see?

handle_value <nil>

What did you see instead?

0 Invalid access to memory location.

But the weird thing is if I change the main function to the following, everything will be OK:

func main() {
	var accels = make([]ACCEL, 0, 1)
	accels = append(accels, someAccels()...)
	h, err := CreateAcceleratorTableW(accels)
	fmt.Println(h, err)
}

[thanm]

@golang/runtime @alexbrainman per owners

[alexbrainman]

0 Invalid access to memory location.

The error message you get is ERROR_NOACCESS.

I don't see a problem with your program.

My guess you data is not aligned properly. Sometimes it works and sometimes it does not. My guess it probably wrong.

Alex

[mkch]

Thank you very much @alexbrainman
But I am not sure how to align my data. Please could you give me an advice?

Kevin

[mknyszek]

Given that it doesn't seem like there's a bug in Go or x/sys/windows here (especially since this is accessing a Windows API that doesn't have a wrapper in x/sys/windows) I don't think the issue tracker is the right place for this anymore.

In triage we think we should close this issue, but please feel free to follow up on https://groups.google.com/g/golang-nuts/ which is a good place for these kinds of questions. Thanks!

[alexbrainman]

@mkch

I just tried to build and run both versions of your program with go1.21.5 windows/amd64. And both of them output valid handle and no error.

Something like this:

1114789 <nil>
1049053 <nil>
393771 <nil>
721451 <nil>

Alex

[mkch]

I totally understand.
But I want to share something I just found: The problem exists only when the code is debugged in VS Code. May be the problem is related to the debugger(Delve?).

Kevin

[mkch]

Thank you for all your time.

After some investigation, I found that the problem is related to the build flags.

If I build the code with go build, everything will be OK; but If I build the code with go build -gcflags "-l -N", the problem will be reproduced.

Any suggestions?

@alexbrainman

[alexbrainman]

@mkch

The problem exists only when the code is debugged in VS Code. May be the problem is related to the debugger(Delve?).

Yes. It could be related to Delve.

If I build the code with go build, everything will be OK; but If I build the code with go build -gcflags "-l -N", the problem will be reproduced.

How can I reproduce the problem here? What are the steps?

Alex

[mkch]

@alexbrainman

How can I reproduce the problem here? What are the steps?

Here are the steps:

C:\Users\gophor\testsyscall>type main.go
package main

import (
        "fmt"
        "unsafe"

        "golang.org/x/sys/windows"
)

type ACCEL struct {
        Virt byte
        Key  uint16
        Cmd  uint16
}

func CreateAcceleratorTableW(accel []ACCEL) (uintptr, error) {
        r, _, err := windows.NewLazySystemDLL("user32.dll").
                NewProc("CreateAcceleratorTableW").
                Call(uintptr(unsafe.Pointer(&accel[0])), uintptr(len(accel)))
        if r == 0 {
                return 0, err
        }
        return r, nil
}

func someAccels() []ACCEL {
        return []ACCEL{{83, 13, 100}}
}

func main() {
        var accels []ACCEL
        accels = append(accels, someAccels()...)
        h, err := CreateAcceleratorTableW(accels)
        fmt.Println(h, err)
}

C:\Users\gophor\testsyscall>go mod init testsyscall
go: creating new go.mod: module testsyscall
go: to add module requirements and sums:
        go mod tidy

C:\Users\gophor\testsyscall>go mod tidy
go: finding module for package golang.org/x/sys/windows
go: found golang.org/x/sys/windows in golang.org/x/sys v0.16.0

C:\Users\gophor\testsyscall>go run main.go
21563259 <nil>

C:\Users\gophor\testsyscall>go run -gcflags "-l -N" main.go
0 Invalid access to memory location.

C:\Users\gophor\testsyscall>

I'm running Windows 10 Professional edition.

Kevin

[qiulaidongfeng]

Here are the steps:

C:\Users\gophor\testsyscall>type main.go
package main

import (
        "fmt"
        "unsafe"

        "golang.org/x/sys/windows"
)

type ACCEL struct {
        Virt byte
        Key  uint16
        Cmd  uint16
}

func CreateAcceleratorTableW(accel []ACCEL) (uintptr, error) {
        r, _, err := windows.NewLazySystemDLL("user32.dll").
                NewProc("CreateAcceleratorTableW").
                Call(uintptr(unsafe.Pointer(&accel[0])), uintptr(len(accel)))
        if r == 0 {
                return 0, err
        }
        return r, nil
}

func someAccels() []ACCEL {
        return []ACCEL{{83, 13, 100}}
}

func main() {
        var accels []ACCEL
        accels = append(accels, someAccels()...)
        h, err := CreateAcceleratorTableW(accels)
        fmt.Println(h, err)
}

C:\Users\gophor\testsyscall>go mod init testsyscall
go: creating new go.mod: module testsyscall
go: to add module requirements and sums:
        go mod tidy

C:\Users\gophor\testsyscall>go mod tidy
go: finding module for package golang.org/x/sys/windows
go: found golang.org/x/sys/windows in golang.org/x/sys v0.16.0

C:\Users\gophor\testsyscall>go run main.go
21563259 <nil>

C:\Users\gophor\testsyscall>go run -gcflags "-l -N" main.go
0 Invalid access to memory location.

C:\Users\gophor\testsyscall>

I'm running Windows 10 Professional edition.

Kevin

windows11 can also reproduce.

[alexbrainman]

Here are the steps:

@mkch

Thanks for the steps. I did not try them, because I am pretty sure, your problem is because your CreateAcceleratorTableW parameter is not aligned properly and Windows is complaining about this.

See #65069 for more details. The details should give you enough pointers to solve your problem.

Alex

[mkch]

Thanks for the steps. I did not try them, because I am pretty sure, your problem is because your CreateAcceleratorTableW parameter is not aligned properly and Windows is complaining about this.

See #65069 for more details. The details should give you enough pointers to solve your problem.

@alexbrainman

Thanks for your help.

In my example the ACCEL struct in go and in C both have a size of 6 bytes and an alignment of 2 bytes. So I guess "aligned properly" means aligning the address of struct in memory.

But I found it's very difficult to align the memory address when go slice is involved.

For example

// main.go
package main

import (
	"fmt"
	"unsafe"

	"golang.org/x/sys/windows"
)

type ACCEL struct {
	Virt byte
	Key  uint16
	Cmd  uint16
}

func CreateAcceleratorTableW(accel []ACCEL) (uintptr, error) {
	fmt.Println(unsafe.Pointer(&accel[0]))
	r, _, err := windows.NewLazySystemDLL("user32.dll").
		NewProc("CreateAcceleratorTableW").
		Call(uintptr(unsafe.Pointer(&accel[0])), uintptr(len(accel)))
	if r == 0 {
		return 0, err
	}
	return r, nil
}

func someAccels() []ACCEL {
	return []ACCEL{{83, 13, 100}}
}

func main() {
	var temp [1]byte // HERE!
	fmt.Println(temp)
	var accels []ACCEL
	accels = append(accels, someAccels()...)
	h, err := CreateAcceleratorTableW(accels)
	fmt.Println(h, err)
}

go run -gcflags "-l -N" main.go
[0]
0xc00000a150
18089915 <nil>

It runs OK.

But if I change var temp [1]byte to var temp [2]byte as the following

func main() {
	var temp [2]byte // HERE!
	fmt.Println(temp)
	var accels []ACCEL
	accels = append(accels, someAccels()...)
	h, err := CreateAcceleratorTableW(accels)
	fmt.Println(h, err)
}

go run -gcflags "-l -N" main.go
[0 0]
0xc00000a156
0 Invalid access to memory location.

The problem goes back.

Further more, as mentioned, -gcflags "-l -N" is also a matter of concern.

Kevin

[alexbrainman]

In my example the ACCEL struct in go and in C both have a size of 6 bytes and an alignment of 2 bytes.

How do you know that Windows expects CreateAcceleratorTableW parameter to use alignment of 2 bytes? I believe it expects 8.

Alex

[mkch]

@alexbrainman

I tested in C code and found that CreateAcceleratorTableW needs a 4-byte aligned argument on x64 machine. I don't know why it isn't 8-byte.

Anyway, 8-byte alignment is always 4-byte alignment. I tried to write an aux function that aligns go slice data to 8-byte(size of uintptr) boundary. Hopefully it is also useful to other similar windows APIs.

// alignSlice makes &s[0] word-aligned.
// If len(s) == 0, s is unchanged.
func alignSlice[T any](s *[]T) {
	const WordSize = unsafe.Sizeof(uintptr(0))
	if len(*s) == 0 || uintptr(unsafe.Pointer(&(*s)[0]))%WordSize == 0 {
		return
	}
	// p = alloc(slice_data_size+WordSize)
	p := make([]byte, len(*s)*int(unsafe.Sizeof((*s)[0]))+int(WordSize))
	// aligned = p+WordSize-(p%WordSize)
	aligned := unsafe.Slice((*T)(unsafe.Pointer(uintptr(unsafe.Pointer(&p[0]))+WordSize-uintptr(unsafe.Pointer(&p[0]))%WordSize)), len(*s))
	copy(aligned, *s)
	*s = aligned
}

and the API call:

func CreateAcceleratorTableW(accel []ACCEL) (uintptr, error) {
	// Align the slice data
	alignSlice(&accel)
	r, _, err := windows.NewLazySystemDLL("user32.dll").
		NewProc("CreateAcceleratorTableW").
		Call(uintptr(unsafe.Pointer(&accel[0])), uintptr(len(accel)))
	if r == 0 {
		return 0, err
	}
	return r, nil
}

It passed all my test now, but I am not 100% sure it is correct especially when stack copying is involved.

Any advice?