Skip to content

8359870: JVM crashes in AccessInternal::PostRuntimeDispatch #25958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 18 commits into from

Conversation

kevinjwalls
Copy link
Contributor

@kevinjwalls kevinjwalls commented Jun 24, 2025

ThreadDumper/ThreadSnapshot need to handle a failure to resolve the native VM JavaThread from a java.lang.Thread. This is hard to reproduce but a thread that has since terminated can provoke a crash. Recognise this and return a null ThreadSnapshot.


Progress

  • Change must be properly reviewed (1 review required, with at least 1 Reviewer)
  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue

Issue

  • JDK-8359870: JVM crashes in AccessInternal::PostRuntimeDispatch (Bug - P2)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/25958/head:pull/25958
$ git checkout pull/25958

Update a local copy of the PR:
$ git checkout pull/25958
$ git pull https://git.openjdk.org/jdk.git pull/25958/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 25958

View PR using the GUI difftool:
$ git pr show -t 25958

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/25958.diff

Using Webrev

Link to Webrev Comment

Sorry, something went wrong.

@bridgekeeper
Copy link

bridgekeeper bot commented Jun 24, 2025

👋 Welcome back kevinw! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Jun 24, 2025

@kevinjwalls This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8359870: JVM crashes in AccessInternal::PostRuntimeDispatch

Reviewed-by: amenkov, dholmes, sspitsyn

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 20 new commits pushed to the master branch:

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk
Copy link

openjdk bot commented Jun 24, 2025

@kevinjwalls The following labels will be automatically applied to this pull request:

  • core-libs
  • hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@AlanBateman
Copy link
Contributor

/label remove core-libs
/label add serviceability

@openjdk
Copy link

openjdk bot commented Jun 24, 2025

@AlanBateman
The core-libs label was successfully removed.

@openjdk
Copy link

openjdk bot commented Jun 24, 2025

@AlanBateman
The serviceability label was successfully added.

@kevinjwalls kevinjwalls marked this pull request as ready for review June 25, 2025 10:02
@openjdk openjdk bot added the rfr Pull request is ready for review label Jun 25, 2025
@mlbridge
Copy link

mlbridge bot commented Jun 25, 2025

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jun 25, 2025
Copy link
Member

@dholmes-ora dholmes-ora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something still bugging me about this one. From JBS it looked to me like we were dealing with a virtual thread but your change is for the non-virtual thread. And Alan says something about this only being possible due to a temporary condition. So I'm still unclear exactly what the problem is, or why it appeared. Where does the initial "thread" argument come from in the Java code? Is it the one that has terminated, if so why is there not an isAlive() check somewhere?

And how does this lead to the bad oop?

@kevinjwalls
Copy link
Contributor Author

Something still bugging me about this one. From JBS it looked to me like we were dealing with a virtual thread but your change is for the non-virtual thread. And Alan says something about this only being possible due to a temporary condition. So I'm still unclear exactly what the problem is, or why it appeared. Where does the initial "thread" argument come from in the Java code? Is it the one that has terminated, if so why is there not an isAlive() check somewhere?

And how does this lead to the bad oop?

Yes, I was reproducing with a regular non-virtual thread exiting. We have the the j.l.Thread Object and could for it being TERMINATED earlier in HeapDumper/Snapshot, but leaving it to the last moment avoids a bigger window where it could terminate.

(Maybe there is somewhere this should intersect with ThreadSMR...?)

On the bad oop:
I enabled the test to run in debug vm for my own testing, but in one of the earlier release crashes at:

V [libjvm.so+0x47bb10] AccessInternal::PostRuntimeDispatch<G1BarrierSet::AccessBarrier<286822ul, G1BarrierSet>, (AccessInternal::BarrierType)3, 286822ul>::oop_access_barrier(oopDesc*, long)+0x0 (accessBackend.hpp:228)
V [libjvm.so+0x10e1c1a] vframeStream::vframeStream(oopDesc*, Handle)+0x7a (vframe.cpp:523)
V [libjvm.so+0x1068a51] GetThreadSnapshotClosure::do_thread(Thread*)+0x7d1 (threadService.cpp:1319)
V [libjvm.so+0x106691d] ThreadSnapshotFactory::get_thread_snapshot(_jobject*, JavaThread*)+0x80d (threadService.cpp:1482)
V [libjvm.so+0xae23d5] JVM_CreateThreadSnapshot+0x75 (jvm.cpp:2966)
j jdk.internal.vm.ThreadSnapshot.create(Ljava/lang/Thread;)Ljdk/internal/vm/ThreadSnapshot;+0 java.base@25-ea
...

Line number info puts it in the _java_thread == null branch of:
threadService.cpp
1317 vframeStream vfst(_java_thread != nullptr
1318 ? vframeStream(_java_thread, false, true, vthread_carrier)
1319 : vframeStream(java_lang_VirtualThread::continuation(_thread_h()))); <---

And it's looking inside the Handle _thread_h() within GetThreadSnapshotClosure which was setup by get_thread_snapshot, and it's a null pointer, as
Instructions:
=>0x00007ffadc251b10: 8b 14 37 31 c0 85 d2 74 18 89 d0 48 8d 15 1e ee
mov edx,DWORD PTR [rdi+rsi*1] and RDI=0x0000000000000000

@dholmes-ora
Copy link
Member

Line number info puts it in the _java_thread == null branch of: threadService.cpp
1317 vframeStream vfst(_java_thread != nullptr
1318 ? vframeStream(_java_thread, false, true, vthread_carrier)
1319 : vframeStream(java_lang_VirtualThread::continuation(_thread_h()))); <---

And it's looking inside the Handle _thread_h() within GetThreadSnapshotClosure which was setup by get_thread_snapshot, and it's a null pointer,

But _thread_h() has already been used a number of times before we get here and if it were null we should have crashed long ago. ???

@alexmenkov
Copy link

But null from JNIHandles::resolve(jthread) is the earliest problem I found.

I'm redoing with the cv_internal_thread_to_JavaThread usage...

A little concerned that ThreadsListHandle::cv_internal_thread_to_JavaThread takes jobject jthread, our ref to a java.lang.Thread, and uses also calls 811 oop thread_oop = JNIHandles::resolve_non_null(jthread);

JNIHandles::resolve(jthread) can return null only if jthread == nullptr, this should not be possible

@dholmes-ora
Copy link
Member

I believe null here is not result of _thread_h(), but is returned by java_lang_VirtualThread::continuation(...) because _thread_h is lava.lang.Thread object and not java.lang.VirtualThread.

That could only happen if we are dealing with a terminated regular thread - which we should never do here if the TLH is used correctly and we only ever pass live threads to do_thread, or else the null which means "unmounted virtual thread".

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jun 30, 2025
Copy link
Member

@dholmes-ora dholmes-ora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor nits/suggestions but generally looks good.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: David Holmes <[email protected]>
@openjdk openjdk bot removed the ready Pull request is ready to be integrated label Jul 1, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: David Holmes <[email protected]>
Copy link
Contributor

@sspitsyn sspitsyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

kevinjwalls and others added 3 commits July 1, 2025 10:29

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Co-authored-by: David Holmes <[email protected]>
…/jdk into 8359870_threadexited
@openjdk
Copy link

openjdk bot commented Jul 1, 2025

⚠️ @kevinjwalls This pull request contains merges that bring in commits not present in the target repository. Since this is not a "merge style" pull request, these changes will be squashed when this pull request in integrated. If this is your intention, then please ignore this message. If you want to preserve the commit structure, you must change the title of this pull request to Merge <project>:<branch> where <project> is the name of another project in the OpenJDK organization (for example Merge jdk:master).

@kevinjwalls
Copy link
Contributor Author

Thanks for all the feedback and reviews!

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jul 1, 2025
@kevinjwalls
Copy link
Contributor Author

/integrate

@openjdk
Copy link

openjdk bot commented Jul 1, 2025

Going to push as commit 13a3927.
Since your change was applied there have been 20 commits pushed to the master branch:

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Jul 1, 2025
@openjdk openjdk bot closed this Jul 1, 2025
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jul 1, 2025
@openjdk
Copy link

openjdk bot commented Jul 1, 2025

@kevinjwalls Pushed as commit 13a3927.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

Copy link
Member

@dholmes-ora dholmes-ora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Belated thumbs up from me too.

@kevinjwalls
Copy link
Contributor Author

/backport :jdk25

@openjdk
Copy link

openjdk bot commented Jul 2, 2025

@kevinjwalls the backport was successfully created on the branch backport-kevinjwalls-13a39278-jdk25 in my personal fork of openjdk/jdk. To create a pull request with this backport targeting openjdk/jdk:jdk25, just click the following link:

➡️ Create pull request

The title of the pull request is automatically filled in correctly and below you find a suggestion for the pull request body:

Hi all,

This pull request contains a backport of commit 13a39278 from the openjdk/jdk repository.

The commit being backported was authored by Kevin Walls on 1 Jul 2025 and was reviewed by Alex Menkov, David Holmes and Serguei Spitsyn.

Thanks!

If you need to update the source branch of the pull then run the following commands in a local clone of your personal fork of openjdk/jdk:

$ git fetch https://github.com/openjdk-bots/jdk.git backport-kevinjwalls-13a39278-jdk25:backport-kevinjwalls-13a39278-jdk25
$ git checkout backport-kevinjwalls-13a39278-jdk25
# make changes
$ git add paths/to/changed/files
$ git commit --message 'Describe additional changes made'
$ git push https://github.com/openjdk-bots/jdk.git backport-kevinjwalls-13a39278-jdk25

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

None yet

6 participants