clang-12 strangely optimize a loop into recursive call to main function

[wierton]

the code:

int loop_1 = 100;
int loop_2 = 7;
int flag = 0;

int test(void) {
  int i = 0;
  int counter = 0;

  while (counter < 1) {  // note: replace counter < 1 by true won't trigger segfault
    if (flag & 1) {
    }
    flag++;
  }
  return 1;
}

int main() {
  if (test() != 1) abort();
  return 0;
}

the above code has an infinite loop in function test, however when compile it with clang-12 -O1, the produced executable will immediately segfault.

I disassembled the a.out and find that:

0000000000401110 <main>:
  401110:	50                   	push   %rax
  401111:	e8 fa ff ff ff       	callq  401110 <main>
  401116:	66 2e 0f 1f 84 00 00 	nopw   %cs:0x0(%rax,%rax,1)
  40111d:	00 00 00 

clang-12 optimize the above code into a strange recursive call of main function and finally cause stack overflow.

I know that unterminated loop is undefined behavior, theoretically compiler can do anything, but such optimization is still counterintuitive.

Not only this, while the loop condition counter < 1 is changed into 1, this two versions should be identical, but clang-12 reasonably optimize it into infinite loop.

A much more strange case is that, when enable -O2 for clang-12, clang-12 will directly optimize out the whole main function, this cause gdb failed to break on main function:

$ clang-12 -O2 a.c -o a.out -ggdb3
$ objdump -d a.out | grep 'main>'
$ gdb -q ./a.out
Reading symbols from ./a.out...
(gdb) b main
Breakpoint 1 at 0x401110
(gdb) r
Starting program: ~/a.out 

Breakpoint 1, 0x0000000000401110 in __libc_csu_init ()
(gdb)

[LebedevRI]

That loop is an infinite loop with no side-effects, and it's condition is not one of those that are explicitly allowed
to be infinite (the condition must be a constant true, not just happen to always evaluate to true),
so this source code is UB.

[wierton]

That loop is an infinite loop with no side-effects, and it's condition is not one of those that are explicitly allowed to be infinite (the condition must be a constant true, not just happen to always evaluate to true), so this source code is UB.

Thanks for response, UB is first part in my posted issue (I have mentioned it), the second part is that, when compile it with -O2, the gdb will fail to locate main function, this may trouble programmers when debugging it.

[efriedma-quic]

gdb is in fact finding the main function; otherwise you wouldn't be able to set a breakpoint. It's just zero bytes long.

It might be worth taking a slight codesize penalty to make what's happening here more clear, though. Feel free to open a new bug with a clean description for that.

[wierton]

gdb is in fact finding the main function; otherwise you wouldn't be able to set a breakpoint. It's just zero bytes long.

It might be worth taking a slight codesize penalty to make what's happening here more clear, though. Feel free to open a new bug with a clean description for that.

Very thanks for your explanation, it solved my confusion.

[efriedma-quic]

I guess the zero-length function issue is #32380.