Description
the code:
int loop_1 = 100;
int loop_2 = 7;
int flag = 0;
int test(void) {
int i = 0;
int counter = 0;
while (counter < 1) { // note: replace counter < 1 by true won't trigger segfault
if (flag & 1) {
}
flag++;
}
return 1;
}
int main() {
if (test() != 1) abort();
return 0;
}
the above code has an infinite loop in function test, however when compile it with clang-12 -O1, the produced executable will immediately segfault.
I disassembled the a.out and find that:
0000000000401110 <main>:
401110: 50 push %rax
401111: e8 fa ff ff ff callq 401110 <main>
401116: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
40111d: 00 00 00
clang-12 optimize the above code into a strange recursive call of main function and finally cause stack overflow.
I know that unterminated loop is undefined behavior, theoretically compiler can do anything, but such optimization is still counterintuitive.
Not only this, while the loop condition counter < 1 is changed into 1, this two versions should be identical, but clang-12 reasonably optimize it into infinite loop.
A much more strange case is that, when enable -O2 for clang-12, clang-12 will directly optimize out the whole main function, this cause gdb failed to break on main function:
$ clang-12 -O2 a.c -o a.out -ggdb3
$ objdump -d a.out | grep 'main>'
$ gdb -q ./a.out
Reading symbols from ./a.out...
(gdb) b main
Breakpoint 1 at 0x401110
(gdb) r
Starting program: ~/a.out
Breakpoint 1, 0x0000000000401110 in __libc_csu_init ()
(gdb)