8357550: GenShen crashes during freeze: assert(!chunk->requires_barriers()) failed

[earthling-amzn]

Shenandoah captures a preliminary top-at-mark-start (TAMS) concurrently. Once concurrent marking has begun, any object beneath TAMS requires a barrier when accessed. The implementation of ShenandoahHeap::requires_barrier does not distinguish between old and young marking. This could lead to a race between freezing the stack and capturing TAMS:

1. Collector is marking old
2. Allocate stack chunk in young
3. Barrier is not needed, begin fast freeze path
4. Collector concurrently updates TAMS (young marking will start once init-mark safepoint is reached)
5. ShenandoahHeap now thinks the barrier is needed (because TAMS changed, and we are marking old)
6. Fast freeze asserts out

This fix has the generational mode check the appropriate generation and its marking state. It also enforces barriers for stackChunks in the old generation (for card marking).

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8357550: GenShen crashes during freeze: assert(!chunk->requires_barriers()) failed (Bug - P4)

Reviewers

• Kelvin Nilsen (@kdnilsen - Committer)
• Aleksey Shipilev (@shipilev - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/25905/head:pull/25905
$ git checkout pull/25905

Update a local copy of the PR:
$ git checkout pull/25905
$ git pull https://git.openjdk.org/jdk.git pull/25905/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 25905

View PR using the GUI difftool:
$ git pr show -t 25905

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/25905.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back wkemper! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@earthling-amzn This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8357550: GenShen crashes during freeze: assert(!chunk->requires_barriers()) failed

Reviewed-by: kdnilsen, shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 9 new commits pushed to the master branch:

• 3397062: 8360069: Problem list CodeInvalidationReasonTest.java until JDK-8360049 is fixed
• c51bed7: 8335986: Test javax/swing/JCheckBox/4449413/bug4449413.java fails on Windows 11 x64 because RBMenuItem's and CBMenuItem's checkmark on the left side are not visible
• a6464b7: 8358578: Small -XX:NMethodSizeLimit triggers "not in CodeBuffer memory" assert in C1
• ... and 6 more: https://git.openjdk.org/jdk/compare/c7125aa2af43a339d401f8416a2251574f6de840...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[openjdk]

@earthling-amzn The following labels will be automatically applied to this pull request:

• hotspot-gc
• shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (c5964a71)

[kdnilsen]

Just so I understand (and maybe this deserves a clarifying comment to help others understand):
The barrier we are concerned about here is SATB. If obj is above TAMS, there's no need to enforce a SATB barrier. Any reference residing above TAMS which is overwritten will have been marked already, or is about to be marked. Either the value is already "presumed marked" because it also resides above TAMS, or its value was assigned from a root (in which case root scanning marked it), or it was copied from a pre-existing object (in which case it will be marked by scanning of that pre-existing object or by SATB).

[earthling-amzn]

Yes, concerned about SATB here. That's my reasoning as well - any overwritten value in an object above TAMS will either be marked from a root or will itself have been allocated above TAMS.

[shipilev]

This makes sense.

[earthling-amzn]

/integrate

[openjdk]

Going to push as commit 17cf497.
Since your change was applied there have been 13 commits pushed to the master branch:

• c6ab63d: 8359735: [Ubuntu 25.10] java/lang/ProcessBuilder/Basic.java, java/lang/ProcessHandle/InfoTest.java fail due to rust-coreutils
• ff54a64: 8360049: CodeInvalidationReasonTest.java fails with ZGC on AArch64
• c11f36e: 8356708: C2: loop strip mining expansion doesn't take sunk stores into account
• ... and 10 more: https://git.openjdk.org/jdk/compare/c7125aa2af43a339d401f8416a2251574f6de840...master

Your commit was automatically rebased without conflicts.

[openjdk]

@earthling-amzn Pushed as commit 17cf497.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.