Segmentation fault on AArch64 release build with opcache.jit=1112 #12596
Comments
|
I cannot reproduce this. Could you please configure PHP with |
Yes, I just checked that the issue also exists with Another guess: Have you run the test on real AArch64 hardware, or just QEMU? Perhaps there's some difference. I can also reproduce this with
Sure, please see below. Another clue from GDB is that SegFault is caused by |
Yes. I run with
QEMU. I don't have a suitable AArch64 box. I'll try the release build... |
|
I can't reproduce the crash. My assembler code look identical, so I suspect, this may be related to memory model differences in QEMU and real CPU. Can you try to debug this down? |
pfustc
changed the title
|
Thanks for your guide. I just followed your steps and got the entry of the JIT-ed code. => 0xaaaaa9a01330 <JIT$test::__construct>: sub sp, sp, #0x30
0xaaaaa9a01334 <JIT$test::__construct+4>: ldr x0, [x27, #32]
0xaaaaa9a01338 <JIT$test::__construct+8>: ldr x1, [x27, #64]
0xaaaaa9a0133c <JIT$test::__construct+12>: ldr x2, [x1, #8]
0xaaaaa9a01340 <JIT$test::__construct+16>: ldr x3, [x0, #16]
0xaaaaa9a01344 <JIT$test::__construct+20>: cmp x3, x2
0xaaaaa9a01348 <JIT$test::__construct+24>: b.ne 0xaaaaa9a01548 <JIT$test::__construct+536> // b.anyUnfortunately, GDB From GDB I see 0xaaaaaaed0a18 <execute_ex+804>: str x2, [x0, #488]where Please see below GDB output. (gdb) x/40i $pc-120
0xaaaaaaed09a0 <execute_ex+684>: bl 0xaaaaaaea3d24 <ZEND_RECV_VARIADIC_SPEC_UNUSED_HANDLER>
0xaaaaaaed09a4 <execute_ex+688>: ldr x0, [x28]
0xaaaaaaed09a8 <execute_ex+692>: br x0
0xaaaaaaed09ac <execute_ex+696>: bl 0xaaaaaaea1fd0 <ZEND_INIT_DYNAMIC_CALL_SPEC_CV_HANDLER>
0xaaaaaaed09b0 <execute_ex+700>: ldr x0, [x28]
0xaaaaaaed09b4 <execute_ex+704>: br x0
0xaaaaaaed09b8 <execute_ex+708>: bl 0xaaaaaae85300 <ZEND_ECHO_SPEC_CONST_HANDLER>
0xaaaaaaed09bc <execute_ex+712>: ldr x0, [x28]
0xaaaaaaed09c0 <execute_ex+716>: br x0
0xaaaaaaed09c4 <execute_ex+720>: ldr x1, [x27, #16]
0xaaaaaaed09c8 <execute_ex+724>: ldr w2, [x28, #8]
0xaaaaaaed09cc <execute_ex+728>: cbz x1, 0xaaaaaaed09ec <execute_ex+760>
0xaaaaaaed09d0 <execute_ex+732>: add x3, x28, w2, sxtw
0xaaaaaaed09d4 <execute_ex+736>: ldr x0, [x28, w2, sxtw]
0xaaaaaaed09d8 <execute_ex+740>: ldr w2, [x3, #8]
0xaaaaaaed09dc <execute_ex+744>: str x0, [x1]
0xaaaaaaed09e0 <execute_ex+748>: str w2, [x1, #8]
0xaaaaaaed09e4 <execute_ex+752>: tst w2, #0xff00
0xaaaaaaed09e8 <execute_ex+756>: b.ne 0xaaaaaaed81a4 <execute_ex+31408> // b.any
0xaaaaaaed09ec <execute_ex+760>: ldr w24, [x27, #40]
0xaaaaaaed09f0 <execute_ex+764>: mov w0, #0x81f0000 // #136249344
0xaaaaaaed09f4 <execute_ex+768>: str x28, [x27]
0xaaaaaaed09f8 <execute_ex+772>: tst w24, w0
0xaaaaaaed09fc <execute_ex+776>: b.ne 0xaaaaaaed3040 <execute_ex+10572> // b.any
0xaaaaaaed0a00 <execute_ex+780>: ldr x1, [x27, #24]
0xaaaaaaed0a04 <execute_ex+784>: add x21, x27, #0x50
0xaaaaaaed0a08 <execute_ex+788>: ldr x0, [x19, #1392]
0xaaaaaaed0a0c <execute_ex+792>: mov w25, #0xfffffc10 // #-1008
0xaaaaaaed0a10 <execute_ex+796>: ldr w23, [x1, #72]
0xaaaaaaed0a14 <execute_ex+800>: ldr x2, [x27, #48]
=> 0xaaaaaaed0a18 <execute_ex+804>: str x2, [x0, #488]
0xaaaaaaed0a1c <execute_ex+808>: cbnz w23, 0xaaaaaaed0a54 <execute_ex+864>
0xaaaaaaed0a20 <execute_ex+812>: b 0xaaaaaaed0a80 <execute_ex+908>
0xaaaaaaed0a24 <execute_ex+816>: ldr w1, [x0, #4]
0xaaaaaaed0a28 <execute_ex+820>: cmp w1, #0x1a
0xaaaaaaed0a2c <execute_ex+824>: b.ne 0xaaaaaaed0a3c <execute_ex+840> // b.any
0xaaaaaaed0a30 <execute_ex+828>: ldrb w1, [x0, #17]
0xaaaaaaed0a34 <execute_ex+832>: tbz w1, #1, 0xaaaaaaed0a48 <execute_ex+852>
0xaaaaaaed0a38 <execute_ex+836>: ldr x0, [x0, #8]
0xaaaaaaed0a3c <execute_ex+840>: ldr w1, [x0, #4]
(gdb) p/x $x27
$7 = 0xfffff5614090
(gdb) p/x $x0
$8 = 0x0
(gdb) x/100x $x19+1350
0xaaaaa22d8e96: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8ea6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8eb6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8ec6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8ed6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8ee6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8ef6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f06: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f16: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f26: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f36: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f46: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f56: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f66: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f76: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f86: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8f96: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8fa6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8fb6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8fc6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8fd6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8fe6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d8ff6: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d9006: 0x00000000 0x00000000 0x00000000 0x00000000
0xaaaaa22d9016: 0x00000000 0x00000000 0x00000000 0x00000000 |
|
Thank you for the debugging. My assumption about It looks like I probably won't be able to dive deeper before Noveber 7. |
|
Hi @dstogov, I tracked all change points of But I find two mistakes in AArch64 register definition. php-src/ext/opcache/jit/ir/ir_aarch64.h Line 31 in 66e95d9 php-src/ext/opcache/jit/ir/ir_aarch64.h Line 65 in 66e95d9 Unfortunately, after fixing these locally, the segmentation fault issue still exists. Could you try running make test TESTS="-d opcache.enable_cli=1 -d opcache.jit=0002 -d opcache.jit_buffer_size=16M"with AArch64 release build to reproduce the failures in your QEMU environment? |
|
Thanks for the fixes and tests. I'll try to debug this issue, but it's at the end of my TODO list. |
I haven't created any PR for the fix. Would you like to push a commit to fix them directly? |
|
I already committed your fix. |
|
It seems the difference in behaviour is caused by the C compiler. In the following fragment the difference is caused by the line marked by I have no idea why your compiler accesses global variables differently, but this difference together with Do you use GCC and Linux? |
This is actually not a DSO library and the global variable is defined in the same binary. |
|
I just re-investigated this issue a bit. Previously I was using GCC 11.4.0 on Ubuntu 22.04. After I upgraded GCC to 12.3.0, the segmentation fault no longer appears. I have reproduced some differences in the code between using GCC 11.4.0 and 12.3.0 just like what you showed above. But I didn't investigate it further because I don't think it's a GCC bug. The most important thing is why This time I set breakpoint at 0xaaaaa9a01050 <JIT$/mnt/local/www/index.php+64>: str x0, [x27]
0xaaaaa9a01054 <JIT$/mnt/local/www/index.php+68>: str xzr, [x27, #8]
0xaaaaa9a01058 <JIT$/mnt/local/www/index.php+72>: str x27, [x28, #48]
=> 0xaaaaa9a0105c <JIT$/mnt/local/www/index.php+76>: ldr x19, [x28, #24]
0xaaaaa9a01060 <JIT$/mnt/local/www/index.php+80>: ldr w0, [x19, #4]
0xaaaaa9a01064 <JIT$/mnt/local/www/index.php+84>: and w0, w0, #0x800
0xaaaaa9a01068 <JIT$/mnt/local/www/index.php+88>: cmp w0, #0x0After this, the value of (gdb) p $x19
$3 = 187649842055504It's exactly the same wrong value in Does the inline asm Yes, it does something. 0xaaaaaaecccf4 <execute_ex>: stp x29, x30, [sp, #-176]!
0xaaaaaaecccf8 <execute_ex+4>: adrp x1, 0xaaaaab908000 <[email protected]>
0xaaaaaaecccfc <execute_ex+8>: mov x29, sp
0xaaaaaaeccd00 <execute_ex+12>: ldr x1, [x1, #928]
0xaaaaaaeccd04 <execute_ex+16>: stp x19, x20, [sp, #16]
0xaaaaaaeccd08 <execute_ex+20>: stp x21, x22, [sp, #32]
0xaaaaaaeccd0c <execute_ex+24>: stp x23, x24, [sp, #48]
0xaaaaaaeccd10 <execute_ex+28>: stp x25, x26, [sp, #64]
0xaaaaaaeccd14 <execute_ex+32>: ldr x2, [x1]
... (skipped some instructions)
0xaaaaaaeccdd0 <execute_ex+220>: ldp x19, x20, [sp, #16]
0xaaaaaaeccdd4 <execute_ex+224>: ldp x21, x22, [sp, #32]
0xaaaaaaeccdd8 <execute_ex+228>: ldp x23, x24, [sp, #48]
0xaaaaaaeccddc <execute_ex+232>: ldp x25, x26, [sp, #64]
0xaaaaaaeccde0 <execute_ex+236>: ldp x29, x30, [sp], #176
0xaaaaaaeccde4 <execute_ex+240>: retBut it seems that they do not wrap the JIT-ed code correctly. I ran the JIT-ed code instruction by instruction, and found it branched out at 0xaaaaa9a01534 <JIT$test::__construct+516>: mov x28, #0x8bd0 // #35792
0xaaaaa9a01538 <JIT$test::__construct+520>: movk x28, #0xa22d, lsl #16
0xaaaaa9a0153c <JIT$test::__construct+524>: movk x28, #0xaaaa, lsl #32
0xaaaaa9a01540 <JIT$test::__construct+528>: add sp, sp, #0x30
=> 0xaaaaa9a01544 <JIT$test::__construct+532>: b 0xaaaaaaeccfc4 <execute_ex+720>The jump target is around So, Something interesting:
|
|
Thanks for the debugging.
Right. This is not a GCC bug. The different GCC version just makes the bug visible.
Yeah.
I think the same.
It's clear. Debug build uses
Do you mean opcache.jit=xxx2? |
…build with opcache.jit=1112
|
I made some more research and found that the difference in generated code may be caused by I hope you are still able to reproduce the failure with old GCC. |
I just did some tests with #12813. I don't think the issue is fixed. The segmentation fault in my previous case does not appear now. But in other test cases, I still see segmentation faults at the same point caused by Below is another case that still causes segmentation fault with the same JIT option. <?php
error_reporting(0);
$a = 10;
function Test()
{
static $a=1;
global $b;
$c = 1;
$b = 5;
echo "$a $b ";
$a++;
$c++;
echo "$a $c ";
}
Test();
echo "$a $b $c ";
Test();
echo "$a $b $c ";
Test();
?>What do you think of making /* TODO: Allow usage of preserved registers ???
* Their values have to be stored in prologue and restored in epilogue
*/
available = ZEND_REGSET_DIFFERENCE(available, ZEND_REGSET_PRESERVED); |
This is not a solution. Only x19-x26 may be used across function calls without spilling. |
Perhaps I don't quite understand your intention. What do you mean by "across function calls without spilling"? AFAIK, if we conform to the C calling convention, we should save and restore x19-x26 if they are clobbered in a compiled function. I just tried CALL VM and found it works exactly in this way. The CALL VM generates If I understand correctly, the goal of Perhaps it's not a good solution to protect the callee-saves by adding |
In case we perform a CALL in JIT code and some value is used before and after the CALL then it can't be keept in scratch registers. Only in x19-x26 or in memory. This will reduce quality of generated code.
HYBRID VM executes JIT code in context of VM execite_ex(). It doesn't create additional stack frame.
I see. It should be another solution to prevent CC from allocating registers across JIT transitions.
This will increase the cost of each JIT<->VM transition. |
|
Thank you so much for your explanation. I have learned a lot from your reply. |
…build with opcache.jit=1112
|
@pfustc I understood the difference between GCC 11.4.0 from Ubuntu 22.04 and my GCC. GCC 11.4.0 was configured with You may check if your GCC 12.0.3 produces PIC code by Anyway, I updated #12813 and now it works for me even with GCC 11.4.0 |
I have run unit tests with HYBRID VM and all JIT option combinations on AArch64. It looks good. |
Description
This appears on AArch64 build with phpdbg disabled.
Build steps
INI
The following code:
Resulted in this output:
But I expected this output instead:
GDB
PHP Version
PHP master @ 95c8ad2
Operating System
Ubuntu 22.04
The text was updated successfully, but these errors were encountered: