Skip to content

testing: teamcity status reporting from tests has been garbage lately #27498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
knz opened this issue Jul 13, 2018 · 2 comments · Fixed by #27597
Closed

testing: teamcity status reporting from tests has been garbage lately #27498

knz opened this issue Jul 13, 2018 · 2 comments · Fixed by #27597
Assignees
Labels
A-testing Testing tools and infrastructure S-3-productivity Severe issues that impede the productivity of CockroachDB developers.

Comments

@knz
Copy link
Contributor

knz commented Jul 13, 2018

I have processed the queue of TC issues from the last few days and there are a couple entries that are utterly unusable / garbage.

Some examples:

Three main problems:

  1. the name of the test in the issue title is mangled. Maybe a string escaping problem?
  2. the test named in the title has nothing to do with the test that's failing. That's a super-annoying regression from, say, a couple weeks ago. Sometimes the body of the issue with the test details is missing entirely.
  3. the build log is usually unreadable because of JSON garbage. Sometimes this JSON is also included verbatim in the body of the github issue. This is hindering my productivity when I am triaging issues.

By scrutinizing the build log sometimes I can determine what is going on better than TC, but even then that is hard (e.g. in the first example above).

cc @benesch @jordanlewis

@knz knz added S-3-productivity Severe issues that impede the productivity of CockroachDB developers. A-testing Testing tools and infrastructure labels Jul 13, 2018
This was referenced Jul 13, 2018
This was referenced Jul 15, 2018
benesch added a commit to benesch/cockroach that referenced this issue Jul 15, 2018
go1.10 has a bug in its test2json tool that causes test names with colons in
them to be misparsed (golang/go#23920). This was causing the nightly stress
issue poster to incorrectly post dozens of issues about tests that had not
actually failed. Upgrade the builder to go1.10.3, which includes a fix for the
test2json tool.

Fix cockroachdb#27498.

Release note: None
craig bot pushed a commit that referenced this issue Jul 15, 2018
27597: build: upgrade to go1.10.3 r=tschottdorf a=benesch

go1.10 has a bug in its test2json tool that causes test names with colons in
them to be misparsed (golang/go#23920). This was causing the nightly stress
issue poster to incorrectly post dozens of issues about tests that had not
actually failed. Upgrade the builder to go1.10.3, which includes a fix for the
test2json tool.

Fix #27498.

Release note: None

Co-authored-by: Nikhil Benesch <[email protected]>
@craig craig bot closed this as completed in #27597 Jul 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testing Testing tools and infrastructure S-3-productivity Severe issues that impede the productivity of CockroachDB developers.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants