Nested class resolution works after second attempt

[labath]

$ cat a.cc
struct A {
  struct B {
    struct C {
      struct D {
        struct E {
          static int f;
        };
      };
    };
  };
};

int A::B::C::D::E::f = 47;

int main() {
  return A::B::C::D::E::f;
}
$ clang++ a.cc -g
$ lldb a.out -o "b main" -o run
(lldb) target create "a.out"
Current executable set to 'a.out' (x86_64).
(lldb) b main
Breakpoint 1: where = a.out`main + 11 at a.cc:16:10, address = 0x000000000040110b
(lldb) run
Process 37237 stopped
* thread #1, name = 'a.out', stop reason = breakpoint 1.1
    frame #0: 0x000000000040110b a.out`main at a.cc:16:10
   13  	int A::B::C::D::E::f = 47;
   14  	
   15  	int main() {
-> 16  	  return A::B::C::D::E::f;
   17  	}
Process 37237 launched: '/tmp/x/a.out' (x86_64)
(lldb) p A::B::C::D::E::f
error: expression failed to parse:
error: <user expression 0>:1:4: no member named 'B' in 'A'
A::B::C::D::E::f
~~~^
(lldb) p A::B::C::D::E::f
error: expression failed to parse:
error: <user expression 1>:1:7: no member named 'C' in 'A::B'
A::B::C::D::E::f
~~~~~~^
(lldb) p A::B::C::D::E::f
error: expression failed to parse:
error: <user expression 2>:1:10: no member named 'D' in 'A::B::C'
A::B::C::D::E::f
~~~~~~~~~^
(lldb) p A::B::C::D::E::f
error: expression failed to parse:
error: <user expression 3>:1:13: no member named 'E' in 'A::B::C::D'
A::B::C::D::E::f
~~~~~~~~~~~~^
(lldb) p A::B::C::D::E::f
(int) $0 = 47

[llvmbot]

@llvm/issue-subscribers-lldb

[labath]

I have a basic understanding of why this happens, but I have no idea how to fix it.

When clang asks lldb to complete a class, lldb will populate it with all of the members (variables and functions) and then say it's done.

It will not add nested types. It cannot even really do that (without scanning all of debug info) as any given dwarf "definition" of the class may not contain all nested types.

That explains why evaluating the expression fails the first time. Now comes the fun part.

When clang fails to obtain A::B, it will try to generate an error message about it. In that process it will try to perform an unqualified lookup for B. That will translate into an lldb query. While working on it, lldb will find A::B. It will (correctly) conclude that this is not the type that clang was asking about, and return an empty set. However, before it does that, it will add a B forward decl to the class A (the same one that it previously claimed to be complete).

The next time the expression is evaluated, clang will find A::B and carry on as it should. In my ridiculous example, it will then fail to find A::B::C and the whole process will repeat.

[clayborg]

I have a basic understanding of why this happens, but I have no idea how to fix it.

When clang asks lldb to complete a class, lldb will populate it with all of the members (variables and functions) and then say it's done.

It will not add nested types. It cannot even really do that (without scanning all of debug info) as any given dwarf "definition" of the class may not contain all nested types.

That explains why evaluating the expression fails the first time. Now comes the fun part.

When clang fails to obtain A::B, it will try to generate an error message about it. In that process it will try to perform an unqualified lookup for B. That will translate into an lldb query. While working on it, lldb will find A::B. It will (correctly) conclude that this is not the type that clang was asking about, and return an empty set. However, before it does that, it will add a B forward decl to the class A (the same one that it previously claimed to be complete).

The next time the expression is evaluated, clang will find A::B and carry on as it should. In my ridiculous example, it will then fail to find A::B::C and the whole process will repeat.

I found the same thing when investigating #74786. The main issue is we need to figure out if clang has any external AST support for finding nested types, or if clang expects that when a class is completed, that is should add all nested types to the class decl context. One potential fix is to modify the class completion code to note any types that are contained within the class and after creating the class itself, parse all contained types. This will effectively populate all of the contained types inside of a class/struct/union so that the expression parser can find them.
nested-lookups.patch

[Michael137]

This reminds of what @Endilll was trying to address in: #66879

There's some discussion on clang behaviour in the comments, if that helps

[Michael137]

I had to refresh my memory on this again. So stepped through the clang lookups on top-of-tree and below call-tree how we get into this situation.

Source:

struct A {               
    struct B {           
        struct D {       
            static int s;
        };               
    };                   
};                       
                         
int A::B::D::s = 12375;  
                         
int main() {             
    A::B i;              
    A o;                 
    return A::B::D::s;   
}

expr A::B::D::s call-tree:

// Parse A::
BuildCXXNestedNameSpecifier
  LookupName(“A”)
    CppLookupName(“A”)
       CppNamespaceLookup(“A”, ctx=TU)
         LookupDirect(“A”)
           TU->DeclContext::lookup(“A”)
              Source->FindExternalVisibleDeclsByName(“A”)
                 ClangExpressionDeclMap::FindExternalVisibleDecls(“A”)

// Parse B::
BuildCXXNestedNameSpecifier
  LookupQualifiedName(“B”, ctx=A)
    LookupDirect(“B”, ctx=A) <<<< doesn’t call into LLDB but somehow lookup finds it….

// Parse D::
BuildCXXNestedNameSpecifier
   LookupQualifiedName(“D”, ctx=B) <<<< Fails
   LookupName(“D”) <<< Fallback path (performs ordinary name lookup)…but only for the purposes of diagnostics. This used to add “D” to “B”, making the 2nd expr work
      CppNamespaceLookup(“D”, ctx=TU)
         DeclContext::lookup(“D”)
            ClangExpressionDeclMap::FindExternalVisibleDecls(“D”) <<<< Fails to find “D” in TU
   Diag(err_no_mem)

So that last call to FindExternalVisibleDecls("D") used to make the second expression evaluation work. But with the new TypeQuery API, this is no longer the case. Having looked at this again it does seem like the patch @clayborg attached is the way to go. Don't see anything obviously wrong with how clang uses the external source here.

[dwblaikie]

@clayborg Are you going to move ahead with the patch from #53904 (comment) ? Be great to get the regression fixed and if it fixes this bug too, all the better.

[clayborg]

I have created a fix: #77029. Not sure how to associate this PR with this issue

[clayborg]

@clayborg Are you going to move ahead with the patch from #53904 (comment) ? Be great to get the regression fixed and if it fixes this bug too, all the better.

Yes! Just submitted #77029