JIT: fragility in PGO-driven enumerator escape analysis

[AndyAyersMS]

PGO driven enumerator escape analysis relies on getting the right inlines, which is tricky because inlining happens before escape analysis. Here are a few issues that have come up during the .NET 10 endgame that make this feature fragile:

• Some enumerators (like List<T>) have methods that are rarely called (eg once per enumeration). If instrumented code is only run on long lists then the PGO data for those methods will not be sufficient to get them inlined.
• When Tier0 invocations only see very long collections the runtime will transition mid-enumerator loop execution to a OSR method, and we don't instrument OSR methods, so we lose profile data for any code in the method invoked after the loop ends (eg Dispose) .
• At the other end of the spectrum, if instrumented code only sees enumeration of empty lists then the enumerator's get_Current method may never be called and likewise cause escape

Some possible fixes:

• Only have OSR cover the loop that triggers OSR, not what comes after (likely hard), so that we transition back to the instrumented method for post-loop computations
• Deopt if OSR exits the loop often enough (likely even harder)
• Create an instrumented OSR method and invoke it every N'th (say 50-100) transition from Tier0, to handle cases where the OSR method is called (transitioned to) frequently, with the expectation that at some point it will tier up. Along with this we'd also want to instrument the OSR inlinees. Possibly risky if not all OSR invocations are created equal, so we might instead want to multiplex within the OSR method between instrumented and non-instrumented iteration. Revisit JIT: resolve issues with OSR and PGO #47942.
• Find ways to deduce missing profile data, or to induce GDVs in cases where it seems "obvious" they should exist. Eg the Dispose case above, if we have PGO data on the enumerator class elsewhere in the method.
• Find ways to boost inlining likelihood, anticipating what escape analysis might find useful (JIT: try to ensure all enumerator methods are inlined for locally allocated enumerators #116266). Would be easier if we could do data flow during inlining to be sure which values reach particular calls.
• Make it so that escape analysis can create GDVs and invoke inlining on demand, to help resolve conditional escape questions.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.