Skip to content

Fix TestActivityWorkerStop: it times out with go 1.20 #1223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 9, 2023

Conversation

dkrotx
Copy link
Member

@dkrotx dkrotx commented Mar 8, 2023

The only test that fails with go 1.20 is this one. The reason is workerOptions.WorkerActivitiesPerSecond == 0, and it is not augmented in newActivityWorker() - this supposed to be done in callers. This breaks task-dispatching: polled tasks are never executed since it is always above the limit(0).

What changed?
TestActivityWorkerStop now works with default MaxActivity

Why?
make unit_test was hanging on TestActivityWorkerStop for 10 minutes (timeout), and failed afterwards.
I believe it because of the new go version:

$ go version
go version go1.20.1 darwin/arm64

The reason is, an activity which supposed to drive the test is never executed because of rate-limiting; workerOptions.WorkerActivitiesPerSecond is zero. newActivityWorker expects this already be augmented to default (100k) by callers.

How did you test it?
make unit_test succeeds after this patch.

Potential risks
No risks, it is only a test-file changed.

@CLAassistant
Copy link

CLAassistant commented Mar 8, 2023

CLA assistant check
All committers have signed the CLA.

@Shaddoll
Copy link
Member

Shaddoll commented Mar 8, 2023

Did it only fail with go 1.20?

@dkrotx
Copy link
Member Author

dkrotx commented Mar 8, 2023

Did it only fail with go 1.20?

Actually, not only. It failed with 1.19 as well.
But then I don't quite understand how it worked before. There is definitely an issue - to start worker with WorkerActivitiesPerSecond being zero.

The only test that fails with go 1.20 is this one.
The reason is workerOptions.WorkerActivitiesPerSecond == 0, and it is not augmented in newActivityWorker() - this supposed to be done in callers.
This breaks task-dispatching: polled tasks are never executed since it is always above the limit(0).
@dkrotx dkrotx force-pushed the bugfix/hanging-test branch from cebda2f to 9cb1719 Compare March 9, 2023 10:56
@dkrotx dkrotx merged commit 6db8982 into cadence-workflow:master Mar 9, 2023
Groxx added a commit that referenced this pull request May 5, 2025
Mockery is failing when run with go 1.24 due to vektra/mockery#914 , so let's upgrade it.

Basic steps to upgrade are:
```
# add go toolchain to go.mod to enforce a higher minimum
cd internal/tools
go get golang.org/x/tools@latest github.com/vektra/mockery/v2@latest
cd ../..
make all
```
and some careful reading to verify the results.

And then some changes to address our CI needs, e.g. since runtime and tool-time versions have diverged (currently) we now automatically pull and build with the old version.
Most of these version values can be found by doing a grep like
```
❯ rg 'go(lang)?.?1.?2\d'
go.mod
3:go 1.21                # intentionally different

docker/buildkite/Dockerfile
1:FROM golang:1.24

Makefile
36:EXPECTED_GO_VERSION := go1.24

CHANGELOG.md
28:- Bump x/tools for tools, to support go 1.22 (#1336)
143:- Fix TestActivityWorkerStop: it times out with go 1.20 by @dkrotx in #1223

internal/tools/go.mod
3:go 1.23.0
5:toolchain go1.24.2
```

I had _expected_ `go 1.21` in go.mod to take care of this kind of check, but apparently not: golang/go#73603
so it's now done by hand.

---

There are *some* changes which I think stand a very small chance of causing problems:
- Our RPC client's mocks now check for `.Get(0).(func(args) (any, error)` signatures where before they did not
  - I believe anyone using this would've hit a `.Error` further down after these changes, so hopefully this just fixes buggy behavior and does not cause any existing tests to fail
- Many funcs gained a `if len(ret) == 0 { panic(..) }` check
  - Since there's a `ret.Get(0)` immediately after which also panics, I suspect this just gives better error messages.
- Our mocks now have a `DoAndReturn` method
  - Since there are no interfaces for all this, this should be fine, ignoring reflection (which we must ignore or else we will go mad).

Otherwise seems like nothing exciting, and this does not affect any user-facing libraries or Go versions.
Since this needs a newer CI Go version to build, I've updated those references too.

---

Separately, this was the source of some rather disturbing TILs:
```bash
❯ bash -ec '
[[[ "$a" = "foo"]] && echo first
echo second
'
bash: line 2: [[[: command not found
second

❯ echo $?
0
```

Apparently syntax errors are (sometimes) not script failures, even with `set -e`.

And `sh` does not support `[[`, so it was _printing an error_ and continuing without erroring.  CI uses `sh`, locally we generally get bash or zsh, so a locally-working `[[` may be a remote-silently-failing command.

Madness.  Absolute madness.  This is going to haunt me for a while, unless I can find a way to prevent it (without shellcheck).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants