When destructing a module in the LTO, ilist assertion fails.

[chandlerc]

Bugzilla Link	1073
Resolution	FIXED
Resolved on	Feb 22, 2010 12:47
Version	trunk
OS	Linux

Extended Description

When destroying the LTO, and therefore its various modules, an assertion fails
during the clearing of the global variable ilist in the module. A backtrace is
as follows:

ld: /home/chandlerc/code/compilers/llvm/include/llvm/ADT/ilist:120: typename
bidirectional_iterator<NodeTy, long int>::reference
llvm::ilist_iterator::operator*() const [with NodeTy =
llvm::GlobalVariable]: Assertion `Traits::getNext(NodePtr) != 0 &&
"Dereferencing end()!"' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 47687191382816 (LWP 4590)]

---

Error while running hook_stop:
Value can't be converted to integer.
0x00002b5f09f997b5 in raise () from /lib/libc.so.6
gdb> bt
#​0 0x00002b5f09f997b5 in raise () from /lib/libc.so.6
#​1 0x00002b5f09f9aa4e in abort () from /lib/libc.so.6
#​2 0x00002b5f09f93156 in __assert_fail () from /lib/libc.so.6
#​3 0x00000000005e33b7 in llvm::ilist_iteratorllvm::GlobalVariable::operator*
(this=0x7fffa13e3c00) at
/home/chandlerc/code/compilers/llvm/include/llvm/ADT/ilist:120
#​4 0x00000000005e33d3 in llvm::ilist_iteratorllvm::GlobalVariable::operator->
(this=0x7fffa13e3c00) at
/home/chandlerc/code/compilers/llvm/include/llvm/ADT/ilist:123
#​5 0x00000000009aba7a in llvm::SymbolTableListTraits<llvm::GlobalVariable,
llvm::Module, llvm::Module, llvm::ilist_traitsllvm::GlobalVariable

::setParent (this=0xefcfb0, STO=0x0) at
/home/chandlerc/code/compilers/llvm/lib/VMCore/SymbolTableListTraitsImpl.h:35
#​6 0x00000000009aa8fe in ~Module (this=0xefcfb0) at
/home/chandlerc/code/compilers/llvm/lib/VMCore/Module.cpp:77
#​7 0x000000000069eaee in ~LTO (this=0xedb430) at
/home/chandlerc/code/compilers/llvm/tools/lto/lto.cpp:471
#​8 0x000000000069ab88 in llvm_destroy_optimizer (lto=0xedb430) at
/home/chandlerc/code/compilers/llvm/tools/lto/lto_capi.cpp:35
#​9 0x0000000000418ab5 in remove_llvm_lto () at ../../../binutils/ld/ldmain.c:195
#​10 0x000000000069a6d9 in xatexit_cleanup () at
../../../binutils/libiberty/xatexit.c:98
#​11 0x000000000069a701 in xexit (code=0x0) at ../../../binutils/libiberty/xexit.c:50
#​12 0x000000000041926b in c_main (argc=0x18, argv=0x7fffa13e3e98) at
../../../binutils/ld/ldmain.c:601
#​13 0x00002b5f09f87394 in __libc_start_main () from /lib/libc.so.6
#​14 0x0000000000407a09 in _start ()

The only idea I have is that somewhere in the optimization passes, the list
became corrupted, as when setParent was called the first time, the same loop
succeeds to add the symbols into the symbol table. Removing them causing an
error doesn't make sense unless the list were corrupted in-between.

[lattner]

I don't think it's really possible to fix this without more information. Chandler, can you dig in and try to
understand what is going on?

[chandlerc]

So far, here is what I've been able to dig up:

When the LTO is deleted, the destructor deletes each module in it. Now, in this
case, its only one module. When that module is deleted, its destructor clears
out the GlobalList it has, and sets its parent to 0. The clearing of the list
doesn't cause an error, its setting the parent to 0 that causes an error.

within the setParent() function (SymbolTableListTraitsImpl.h) the error occurs
as follows:

It gets the GlobalList (which was just '.clear()'ed).
It checks if that list is empty, and finds it is not!
It begins the for loop, and finds that .begin() != .end() in the loop condition
When it then dereferences .begin() with '->', the dereference operator is
overloaded, and performs an assertion that fails. I think that the assertion
is saying it cannot find the next node in the list, and I'm not sure why... it
of course won't find the next node in the list, if it is at .end()-1, but maybe
this is not a typical list.

In any event, directly after the list is .clear()ed, it passes !List.empty() and
this probably has something to do with it walking off past the .end() of the list.

Does this help? Should I keep investigating the list construction? I'm having
trouble following it, but I'll keep pushing as needed.

-Chandler

[llvmbot]

What if you put assert check in ~Module() to see if GlobalList is empty() or not. Does it assert ? Thanks.

GlobalList.clear()
assert (!GlobalList.empty() && "It is not empty"); // << New line
GlobalLiset.setParent(0);

[chandlerc]

Well, I think that has shed some light on things Devang.

This is from a full build of LLVM after adding that line:
llvm[4]: Compiling stacker_rt.ll to stacker_rt.bc for Debug build (bytecode)
gccas: /home/chandlerc/code/compilers/llvm/lib/VMCore/Module.cpp:77:
llvm::Module::~Module(): Assertion `!GlobalList.empty() && "It is not empty"'
failed.
/home/chandlerc/code/compilers/build/llvm/Debug/bin/gccas((anonymous
namespace)::PrintStackTrace()+0x17)[0x9571d7]
/home/chandlerc/code/compilers/build/llvm/Debug/bin/gccas((anonymous
namespace)::SignalHandler(int)+0x10e)[0x9574a6]
/lib/libc.so.6[0x2aca9de76810]
/lib/libc.so.6(gsignal+0x35)[0x2aca9de767b5]
/lib/libc.so.6(abort+0x10e)[0x2aca9de77a4e]
/lib/libc.so.6(__assert_fail+0xf6)[0x2aca9de70156]
/home/chandlerc/code/compilers/build/llvm/Debug/bin/gccas(llvm::Module::~Module()+0x90)[0x8e303e]
/home/chandlerc/code/compilers/build/llvm/Debug/bin/gccas(std::auto_ptrllvm::Module::~auto_ptr()+0x27)[0x6ec841]
/home/chandlerc/code/compilers/build/llvm/Debug/bin/gccas(main+0x6b6)[0x6eba96]
/lib/libc.so.6(__libc_start_main+0xf4)[0x2aca9de64394]
/home/chandlerc/code/compilers/build/llvm/Debug/bin/gccas(__gxx_personality_v0+0x111)[0x6eaff9]
make[4]: ***
[/home/chandlerc/code/compilers/build/llvm/projects/Stacker/lib/runtime/Debug/stacker_rt.bc]
Aborted

And when running GNU ld, linked into the LTO lib:

Attempting to read bytecode...
Success!
Optimizing bytecode...
ld: /home/chandlerc/code/compilers/llvm/lib/VMCore/Module.cpp:77:
llvm::Module::~Module(): Assertion `!GlobalList.empty() && "It is not empty"'
failed.

So it seems that .clear() isn't actually clearing the list? I'm not sure what to
make of this, but I'm going to keep throwing in assertions in this and in the
list code to see if anything crops up as a more precise location of the bug.
Also checking to see if .clear() is failing on some of the other lists in case
its specific to the globals...

[chandlerc]

Scratch that, I noticed that the assert was !GlobalList.empty(), which we in
fact want to be true. I changed it to assrt(GlobalList.empty() && "It is not
empty"), and it doesn't cause an error. So, after the .clear(), the list is
empty(), but once setParent(0) is called, and gets the list through the
round-about means, it returns !List.empty()... which is bizarre... I'm trying
find a way investigate further. any ideas? If necessary, i guess i'll fire up
gdb, and step through this sequence of calls, and try and inspect the various
values.

[chandlerc]

Ok, stepped through this mess with gdb, and got some interesting results. First,
it goes through the ~Module() function once, with success, when it optimizes and
outputs the object file with the LTO.

Next, in shutting down the LTO, it goes back through ~Module() for the same
module (this=0xefcfb0 for both). The address of GlobalList was also the same on
both of these. This second time is the time when it throws the assert. Here is
gdb's view of GlobalList before entering setParent(0), and gdb's view of List
within setParent(0):

gdb> print GlobalList
$6 = {<llvm::ilist_traitsllvm::GlobalVariable> =
{<llvm::SymbolTableListTraits<llvm::GlobalVariable,llvm::Module,llvm::Module,llvm::ilist_traitsllvm::GlobalVariable

= {SymTabObject = 0xf06550, ItemParent = 0x10ab070}, }, Head
= 0xedadb0, Tail = 0xedadb0}
gdb> print &GlobalList
$7 = (llvm::iplist<llvm::GlobalVariable,llvm::ilist_traitsllvm::GlobalVariable
*) 0xefcfb0

...

gdb> print List
$8 = (class
llvm::iplist<llvm::GlobalVariable,llvm::ilist_traitsllvm::GlobalVariable > &)
@​0x10ab070: {<llvm::ilist_traitsllvm::GlobalVariable> =
{<llvm::SymbolTableListTraits<llvm::GlobalVariable,llvm::Module,llvm::Module,llvm::ilist_traitsllvm::GlobalVariable

= {SymTabObject = 0x0, ItemParent = 0xc1}, }, Head =
0xefcfa0, Tail = 0xf097e0}
gdb> print &List
$9 = (class
llvm::iplist<llvm::GlobalVariable,llvm::ilist_traitsllvm::GlobalVariable > *)
0x10ab070

Two different lists. Not surprisingly, this list doesn't work. The best I can
figure, the optimization routines are destroying the module, and then so are the
general LTO destructor routines.

I hope this helps, I'm going to start poking around in the optimization routines
to see if I can spot where ~Module gets called from, but not sure how much
success I will have.

[llvmbot]

OK this makes sense. Check LTO::removeModule()

[chandlerc]

Found the error, its in optimizeModules()

the Linker object takes total control of every module fed into it, but those
modules aren't removed from the modules list of the LTO object. When the modules
are linked into the composite, they are destroyed, and even in one module
situations (like mine) that module is passed into the Linker as the composite,
and gets destroyed when the Linker is.

Either they should be removed from the LTO list, as they are no longer usable,
or copies should be made of them before feeding them to the linker. Which is
preferable? I think i can work up a patch for either one, just wondering which
is the preferred method.

[llvmbot]

Aha..

Just remove it from LTO list. Thanks.

[chandlerc]

Patch to fix
This is the trivial patch to fix this bug. GNU ld works cleanly with this.

[lattner]

Devang, when you get a chance, please review and apply this patch. Thanks Devang and Chandler!

[llvmbot]

I thought this got fixed today?

[lattner]

Chandler or Devang, please close this! :)

[chandlerc]

Devang did commit the patch, just closing the bug.