[AliasAnalysis] Introduce getModRefInfoMask() as a generalization of pointsToConstantMemory().

The pointsToConstantMemory() method returns true only if the memory pointed to
by the memory location is globally invariant. However, the LLVM memory model
also has the semantic notion of *locally-invariant*: memory that is known to be
invariant for the life of the SSA value representing that pointer. The most
common example of this is a pointer argument that is marked readonly noalias,
which the Rust compiler frequently emits.

It'd be desirable for LLVM to treat locally-invariant memory the same way as
globally-invariant memory when it's safe to do so. This patch implements that,
by introducing the concept of a *ModRefInfo mask*. A ModRefInfo mask is a bound
on the Mod/Ref behavior of an instruction that writes to a memory location,
based on the knowledge that the memory is globally-constant memory (in which
case the mask is NoModRef) or locally-constant memory (in which case the mask
is Ref). ModRefInfo values for an instruction can be combined with the
ModRefInfo mask by simply using the & operator. Where appropriate, this patch
has modified uses of pointsToConstantMemory() to instead examine the mask.

The most notable optimization change I noticed with this patch is that now
redundant loads from readonly noalias pointers can be eliminated across calls,
even when the pointer is captured. Internally, before this patch,
AliasAnalysis was assigning Ref to reads from constant memory; now AA can
assign NoModRef, which is a tighter bound.

Differential Revision: https://reviews.llvm.org/D136659

Author: pcwalton

llvm/docs/AliasAnalysis.rst

@@ -161,14 +161,24 @@ Other useful ``AliasAnalysis`` methods
 Several other tidbits of information are often collected by various alias
 analysis implementations and can be put to good use by various clients.
 
-The ``pointsToConstantMemory`` method
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The ``pointsToConstantMemory`` method returns true if and only if the analysis
-can prove that the pointer only points to unchanging memory locations
-(functions, constant global variables, and the null pointer).  This information
-can be used to refine mod/ref information: it is impossible for an unchanging
-memory location to be modified.
+The ``getModRefInfoMask`` method
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``getModRefInfoMask`` method returns a bound on Mod/Ref information for
+the supplied pointer, based on knowledge about whether the pointer points to
+globally-constant memory (for which it returns ``NoModRef``) or
+locally-invariant memory (for which it returns ``Ref``). Globally-constant
+memory includes functions, constant global variables, and the null pointer.
+Locally-invariant memory is memory that we know is invariant for the lifetime
+of its SSA value, but not necessarily for the life of the program: for example,
+the memory pointed to by ``readonly`` ``noalias`` parameters is known-invariant
+for the duration of the corresponding function call. Given Mod/Ref information
+``MRI`` for a memory location ``Loc``, ``MRI`` can be refined with a statement
+like ``MRI &= AA.getModRefInfoMask(Loc);``. Another useful idiom is
+``isModSet(AA.getModRefInfoMask(Loc))``; this checks to see if the given
+location can be modified at all. For convenience, there is also a method
+``pointsToConstantMemory(Loc)``; this is synonymous with
+``isNoModRef(AA.getModRefInfoMask(Loc))``.
 
 .. _never access memory or only read memory:
 

llvm/include/llvm/Analysis/AliasAnalysis.h

@@ -359,7 +359,9 @@ class AAResults {
 
   /// Checks whether the given location points to constant memory, or if
   /// \p OrLocal is true whether it points to a local alloca.
-  bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false);
+  bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
+    return isNoModRef(getModRefInfoMask(Loc, OrLocal));
+  }
 
   /// A convenience wrapper around the primary \c pointsToConstantMemory
   /// interface.
@@ -372,6 +374,22 @@ class AAResults {
   /// \name Simple mod/ref information
   /// @{
 
+  /// Returns a bitmask that should be unconditionally applied to the ModRef
+  /// info of a memory location. This allows us to eliminate Mod and/or Ref
+  /// from the ModRef info based on the knowledge that the memory location
+  /// points to constant and/or locally-invariant memory.
+  ///
+  /// If IgnoreLocals is true, then this method returns NoModRef for memory
+  /// that points to a local alloca.
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+                               bool IgnoreLocals = false);
+
+  /// A convenience wrapper around the primary \c getModRefInfoMask
+  /// interface.
+  ModRefInfo getModRefInfoMask(const Value *P, bool IgnoreLocals = false) {
+    return getModRefInfoMask(MemoryLocation::getBeforeOrAfter(P), IgnoreLocals);
+  }
+
   /// Get the ModRef info associated with a pointer argument of a call. The
   /// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
   /// that these bits do not necessarily account for the overall behavior of
@@ -614,6 +632,8 @@ class AAResults {
                     AAQueryInfo &AAQI);
   bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
                               bool OrLocal = false);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals = false);
   ModRefInfo getModRefInfo(Instruction *I, const CallBase *Call2,
                            AAQueryInfo &AAQIP);
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
@@ -681,6 +701,10 @@ class BatchAAResults {
   bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
     return AA.pointsToConstantMemory(Loc, AAQI, OrLocal);
   }
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+                               bool IgnoreLocals = false) {
+    return AA.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+  }
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc) {
     return AA.getModRefInfo(Call, Loc, AAQI);
   }
@@ -743,16 +767,19 @@ class AAResults::Concept {
   virtual AliasResult alias(const MemoryLocation &LocA,
                             const MemoryLocation &LocB, AAQueryInfo &AAQI) = 0;
 
-  /// Checks whether the given location points to constant memory, or if
-  /// \p OrLocal is true whether it points to a local alloca.
-  virtual bool pointsToConstantMemory(const MemoryLocation &Loc,
-                                      AAQueryInfo &AAQI, bool OrLocal) = 0;
-
   /// @}
   //===--------------------------------------------------------------------===//
   /// \name Simple mod/ref information
   /// @{
 
+  /// Returns a bitmask that should be unconditionally applied to the ModRef
+  /// info of a memory location. This allows us to eliminate Mod and/or Ref from
+  /// the ModRef info based on the knowledge that the memory location points to
+  /// constant and/or locally-invariant memory.
+  virtual ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+                                       AAQueryInfo &AAQI,
+                                       bool IgnoreLocals) = 0;
+
   /// Get the ModRef info associated with a pointer argument of a callsite. The
   /// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
   /// that these bits do not necessarily account for the overall behavior of
@@ -801,9 +828,9 @@ template <typename AAResultT> class AAResults::Model final : public Concept {
     return Result.alias(LocA, LocB, AAQI);
   }
 
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal) override {
-    return Result.pointsToConstantMemory(Loc, AAQI, OrLocal);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals) override {
+    return Result.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
   }
 
   ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) override {
@@ -856,9 +883,9 @@ class AAResultBase {
     return AliasResult::MayAlias;
   }
 
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal) {
-    return false;
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals) {
+    return ModRefInfo::ModRef;
   }
 
   ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {

llvm/include/llvm/Analysis/BasicAliasAnalysis.h

@@ -75,9 +75,15 @@ class BasicAAResult : public AAResultBase {
   ModRefInfo getModRefInfo(const CallBase *Call1, const CallBase *Call2,
                            AAQueryInfo &AAQI);
 
-  /// Chases pointers until we find a (constant global) or not.
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal);
+  /// Returns a bitmask that should be unconditionally applied to the ModRef
+  /// info of a memory location. This allows us to eliminate Mod and/or Ref
+  /// from the ModRef info based on the knowledge that the memory location
+  /// points to constant and/or locally-invariant memory.
+  ///
+  /// If IgnoreLocals is true, then this method returns NoModRef for memory
+  /// that points to a local alloca.
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals = false);
 
   /// Get the location associated with a pointer argument of a callsite.
   ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx);

llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h

@@ -52,8 +52,8 @@ class ObjCARCAAResult : public AAResultBase {
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI);
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals);
 
   using AAResultBase::getMemoryEffects;
   MemoryEffects getMemoryEffects(const Function *F);

llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h

@@ -40,8 +40,8 @@ class TypeBasedAAResult : public AAResultBase {
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI);
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals);
   MemoryEffects getMemoryEffects(const CallBase *Call, AAQueryInfo &AAQI);
   MemoryEffects getMemoryEffects(const Function *F);
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,

llvm/lib/Analysis/AliasAnalysis.cpp

@@ -147,19 +147,25 @@ AliasResult AAResults::alias(const MemoryLocation &LocA,
   return Result;
 }
 
-bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
-                                       bool OrLocal) {
+ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
+                                        bool IgnoreLocals) {
   SimpleAAQueryInfo AAQIP(*this);
-  return pointsToConstantMemory(Loc, AAQIP, OrLocal);
+  return getModRefInfoMask(Loc, AAQIP, IgnoreLocals);
 }
 
-bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
-                                       AAQueryInfo &AAQI, bool OrLocal) {
-  for (const auto &AA : AAs)
-    if (AA->pointsToConstantMemory(Loc, AAQI, OrLocal))
-      return true;
+ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
+                                        AAQueryInfo &AAQI, bool IgnoreLocals) {
+  ModRefInfo Result = ModRefInfo::ModRef;
 
-  return false;
+  for (const auto &AA : AAs) {
+    Result &= AA->getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+
+    // Early-exit the moment we reach the bottom of the lattice.
+    if (isNoModRef(Result))
+      return ModRefInfo::NoModRef;
+  }
+
+  return Result;
 }
 
 ModRefInfo AAResults::getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {
@@ -253,10 +259,11 @@ ModRefInfo AAResults::getModRefInfo(const CallBase *Call,
 
   Result &= ArgMR | OtherMR;
 
-  // If Loc is a constant memory location, the call definitely could not
+  // Apply the ModRef mask. This ensures that if Loc is a constant memory
+  // location, we take into account the fact that the call definitely could not
   // modify the memory location.
-  if (isModSet(Result) && pointsToConstantMemory(Loc, AAQI, /*OrLocal*/ false))
-    Result &= ModRefInfo::Ref;
+  if (!isNoModRef(Result))
+    Result &= getModRefInfoMask(Loc);
 
   return Result;
 }
@@ -510,9 +517,11 @@ ModRefInfo AAResults::getModRefInfo(const StoreInst *S,
     if (AR == AliasResult::NoAlias)
       return ModRefInfo::NoModRef;
 
-    // If the pointer is a pointer to constant memory, then it could not have
-    // been modified by this store.
-    if (pointsToConstantMemory(Loc, AAQI))
+    // Examine the ModRef mask. If Mod isn't present, then return NoModRef.
+    // This ensures that if Loc is a constant memory location, we take into
+    // account the fact that the store definitely could not modify the memory
+    // location.
+    if (!isModSet(getModRefInfoMask(Loc)))
       return ModRefInfo::NoModRef;
   }
 
@@ -529,10 +538,11 @@ ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
 ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
                                     const MemoryLocation &Loc,
                                     AAQueryInfo &AAQI) {
-  // If we know that the location is a constant memory location, the fence
-  // cannot modify this location.
-  if (Loc.Ptr && pointsToConstantMemory(Loc, AAQI))
-    return ModRefInfo::Ref;
+  // All we know about a fence instruction is what we get from the ModRef
+  // mask: if Loc is a constant memory location, the fence definitely could
+  // not modify it.
+  if (Loc.Ptr)
+    return getModRefInfoMask(Loc);
   return ModRefInfo::ModRef;
 }
 
@@ -552,10 +562,9 @@ ModRefInfo AAResults::getModRefInfo(const VAArgInst *V,
     if (AR == AliasResult::NoAlias)
       return ModRefInfo::NoModRef;
 
-    // If the pointer is a pointer to constant memory, then it could not have
+    // If the pointer is a pointer to invariant memory, then it could not have
     // been modified by this va_arg.
-    if (pointsToConstantMemory(Loc, AAQI))
-      return ModRefInfo::NoModRef;
+    return getModRefInfoMask(Loc, AAQI);
   }
 
   // Otherwise, a va_arg reads and writes.
@@ -572,10 +581,9 @@ ModRefInfo AAResults::getModRefInfo(const CatchPadInst *CatchPad,
                                     const MemoryLocation &Loc,
                                     AAQueryInfo &AAQI) {
   if (Loc.Ptr) {
-    // If the pointer is a pointer to constant memory,
+    // If the pointer is a pointer to invariant memory,
     // then it could not have been modified by this catchpad.
-    if (pointsToConstantMemory(Loc, AAQI))
-      return ModRefInfo::NoModRef;
+    return getModRefInfoMask(Loc, AAQI);
   }
 
   // Otherwise, a catchpad reads and writes.
@@ -592,10 +600,9 @@ ModRefInfo AAResults::getModRefInfo(const CatchReturnInst *CatchRet,
                                     const MemoryLocation &Loc,
                                     AAQueryInfo &AAQI) {
   if (Loc.Ptr) {
-    // If the pointer is a pointer to constant memory,
+    // If the pointer is a pointer to invariant memory,
     // then it could not have been modified by this catchpad.
-    if (pointsToConstantMemory(Loc, AAQI))
-      return ModRefInfo::NoModRef;
+    return getModRefInfoMask(Loc, AAQI);
   }
 
   // Otherwise, a catchret reads and writes.

llvm/lib/Analysis/BasicAliasAnalysis.cpp

@@ -676,33 +676,46 @@ BasicAAResult::DecomposeGEPExpression(const Value *V, const DataLayout &DL,
   return Decomposed;
 }
 
-/// Returns whether the given pointer value points to memory that is local to
-/// the function, with global constants being considered local to all
-/// functions.
-bool BasicAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
-                                           AAQueryInfo &AAQI, bool OrLocal) {
+ModRefInfo BasicAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+                                            AAQueryInfo &AAQI,
+                                            bool IgnoreLocals) {
   assert(Visited.empty() && "Visited must be cleared after use!");
-  auto _ = make_scope_exit([&]{ Visited.clear(); });
+  auto _ = make_scope_exit([&] { Visited.clear(); });
 
   unsigned MaxLookup = 8;
   SmallVector<const Value *, 16> Worklist;
   Worklist.push_back(Loc.Ptr);
+  ModRefInfo Result = ModRefInfo::NoModRef;
+
   do {
     const Value *V = getUnderlyingObject(Worklist.pop_back_val());
     if (!Visited.insert(V).second)
       continue;
 
-    // An alloca instruction defines local memory.
-    if (OrLocal && isa<AllocaInst>(V))
+    // Ignore allocas if we were instructed to do so.
+    if (IgnoreLocals && isa<AllocaInst>(V))
       continue;
 
-    // A global constant counts as local memory for our purposes.
+    // If the location points to memory that is known to be invariant for
+    // the life of the underlying SSA value, then we can exclude Mod from
+    // the set of valid memory effects.
+    //
+    // An argument that is marked readonly and noalias is known to be
+    // invariant while that function is executing.
+    if (const Argument *Arg = dyn_cast<Argument>(V)) {
+      if (Arg->hasNoAliasAttr() && Arg->onlyReadsMemory()) {
+        Result |= ModRefInfo::Ref;
+        continue;
+      }
+    }
+
+    // A global constant can't be mutated.
     if (const GlobalVariable *GV = dyn_cast<GlobalVariable>(V)) {
       // Note: this doesn't require GV to be "ODR" because it isn't legal for a
       // global to be marked constant in some modules and non-constant in
       // others.  GV may even be a declaration, not a definition.
       if (!GV->isConstant())
-        return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+        return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
       continue;
     }
 
@@ -718,16 +731,20 @@ bool BasicAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
     if (const PHINode *PN = dyn_cast<PHINode>(V)) {
       // Don't bother inspecting phi nodes with many operands.
       if (PN->getNumIncomingValues() > MaxLookup)
-        return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+        return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
       append_range(Worklist, PN->incoming_values());
       continue;
     }
 
     // Otherwise be conservative.
-    return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+    return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
   } while (!Worklist.empty() && --MaxLookup);
 
-  return Worklist.empty();
+  // If we hit the maximum number of instructions to examine, be conservative.
+  if (!Worklist.empty())
+    return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+
+  return Result;
 }
 
 static bool isIntrinsicCall(const CallBase *Call, Intrinsic::ID IID) {

llvm/lib/Analysis/MemoryDependenceAnalysis.cpp

@@ -525,7 +525,7 @@ MemDepResult MemoryDependenceResults::getSimplePointerDependencyFrom(
       }
 
       // Stores don't alias loads from read-only memory.
-      if (BatchAA.pointsToConstantMemory(LoadLoc))
+      if (!isModSet(BatchAA.getModRefInfoMask(LoadLoc)))
         continue;
 
       // Stores depend on may/must aliased loads.