Introduce an experimental @_rawLayout attribute.

This attribute can be attached to a noncopyable struct to specify that its
storage is raw, meaning the type definition is (with some limitations)
able to do as it pleases with the storage. This provides a basis for
implementing types for things like atomics, locks, and data structures
that use inline storage to store conditionally-initialized values.
The example in `test/Prototypes/UnfairLock.swift` demonstrates the use
of a raw layout type to wrap Darwin's `os_unfair_lock` APIs, allowing
a lock value to be stored inside of classes or other types without
needing a separate allocation, and using the borrow model to enforce
safe access to lock-guarded storage.

Author: jckarter

docs/ReferenceGuides/UnderscoredAttributes.md

@@ -832,6 +832,106 @@ Fully bypasses access control, allowing access to private declarations
 in the imported module. The imported module needs to be compiled with
 `-Xfrontend -enable-private-imports` for this to work.
 
+## `@_rawLayout(...)`
+
+Specifies the declared type consists of raw storage. The type must be
+noncopyable, and declare no stored properties.
+Raw storage is left almost entirely unmanaged by the language, and so
+can be used as storage for data structures with nonstandard access patterns
+including atomics and many kinds of locks such as `os_unfair_lock` on Darwin
+or `futex` on Linux, to replicate the behavior of things
+like C++'s `mutable` fields or Rust's `Cell<T>` type which allow for mutation
+in typically immutable contexts, and/or to provide inline storage for data
+structures that may be conditionally initialized, such as a "small vector"
+which stores up to N elements in inline storage but spills into heap allocation
+past a threshold.
+
+Programmers can safely make the following assumptions about
+the memory of the annotated type:
+
+- A value has a **stable address** until it is either consumed or moved.
+  No value of any type in Swift can ever be moved while it is being borrowed or
+  mutated, so the address of `self` within a `borrowing` or `mutating` method,
+  or more generally the address of a `borrowing` or `inout` parameter to any
+  function, cannot change within the function body.  Values that appear in a
+  global variable or class stored property can never be moved, and can only be
+  consumed by the deallocation of the containing object instance, so
+  effectively has a stable address for their entire lifetime.
+- A value's memory **may be read and mutated at any time** independent of
+  formal accesses. In particular, pointers into the storage may be "escaped"
+  outside of scopes where the address is statically guaranteed to be stable, and
+  those pointers may be used freely for as long as the storage dynamically
+  isn't consumed or moved. It becomes the programmer's responsibility in this
+  case to ensure that reads and writes to the storage do not race across
+  threads, writes don't overlap with reads or writes coming from the same
+  thread, and that the pointer is not used after the value is moved or consumed.
+- When the value is moved, a bitwise copy of its memory is performed to the new
+  address of the value in its new owner. As currently implemented, raw storage
+  types are not suitable for storing values which are not bitwise-movable, such
+  as nontrivial C++ types, Objective-C weak references, and data structures
+  such as `pthread_mutex_t` which are implemented in C as always requiring a
+  fixed address.
+
+Using the `@_rawLayout` attribute will suppress the annotated type from
+being implicitly `Sendable`. If the type is safe to access across threads, it
+may be declared to conform to `@unchecked Sendable`, with the usual level
+of programmer-assumed responsibility that involves. This generally means that
+any mutations must be done atomically or with a lock guard, and if the storage
+is ever mutated, then any reads of potentially-mutated state within the storage
+must also be atomic or lock-guarded, because the storage may be accessed
+simultaneously by multiple threads.
+
+A non-Sendable type's memory will be confined to accesses from a single thread
+or task; however, since most mutating operations in Swift still expect
+exclusivity while executing, a programmer must ensure that overlapping
+mutations cannot occur from aliasing, recursion, reentrancy, signal handlers, or
+other potential sources of overlapping access within the same thread.
+
+The parameters to the attribute specify the layout of the type. The following
+forms are currently accepted:
+
+- `@_rawLayout(size: N, alignment: M)` specifies the type's size and alignment
+  in bytes.
+- `@_rawLayout(like: T)` specifies the type's size and alignment should be
+  equal to the type `T`'s.
+- `@_rawLayout(likeArrayOf: T, count: N)` specifies the type's size should be
+  `MemoryLayout<T>.stride * N` and alignment should match `T`'s, like an
+  array of N contiguous elements of `T` in memory.
+
+A notable difference between `@_rawLayout(like: T)` and
+`@_rawLayout(likeArrayOf: T, count: 1)` is that the latter will pad out the
+size of the raw storage to include the full stride of the single element.
+This ensures that the buffer can be safely used with bulk array operations
+despite containing only a single element. `@_rawLayout(like: T)` by contrast
+will exactly match the size and stride of the original type `T`, allowing for
+other values to be stored in the tail padding when the raw layout type appears
+in a larger aggregate.
+
+```
+// struct Weird has size 5, stride 8, alignment 4
+struct Weird {
+    var x: Int32
+    var y: Int8
+}
+
+// struct LikeWeird has size 5, stride 8, alignment 4
+@_rawLayout(like: Weird)
+struct LikeWeird { }
+
+// struct LikeWeirdSingleArray has **size 8**, stride 8, alignment 4
+@_rawLayout(likeArrayOf: Weird, count: 1)
+struct LikeWeirdSingleArray { }
+```
+
+Although the `like:` and `likeArrayOf:count:` forms will produce raw storage
+with the size and alignment of another type, the memory is **not** implicitly
+*bound* to that type, as *bound* is defined by `UnsafePointer` and
+`UnsafeMutablePointer`. The memory can be accessed as raw memory
+if it is never explicitly bound using a typed pointer method like
+`withMemoryRebound(to:)` or `bindMemory(to:)`. However, if the raw memory is
+bound, it must only be used with compatible typed memory accesses for as long
+as the binding is active.
+
 ## `@_section("section_name")`
 
 Places a global variable or a top-level function into a section of the object

include/swift/AST/Attr.h

@@ -2485,6 +2485,107 @@ class MacroRoleAttr final
   }
 };
 
+/// Specifies the raw storage used by a type.
+class RawLayoutAttr final : public DeclAttribute {
+  /// The element type to share size and alignment with, if any.
+  TypeRepr *LikeType;
+  /// The number of elements in an array to share stride and alignment with,
+  /// or zero if no such size was specified. If `LikeType` is null, this is
+  /// the size in bytes of the raw storage.
+  unsigned SizeOrCount;
+  /// If `LikeType` is null, the alignment in bytes to use for the raw storage.
+  unsigned Alignment;
+  /// The resolved like type.
+  Type ResolvedLikeType = Type();
+  
+public:
+  /// Construct a `@_rawLayout(like: T)` attribute.
+  RawLayoutAttr(TypeRepr *LikeType,
+                SourceLoc AtLoc,
+                SourceRange Range)
+    : DeclAttribute(DAK_RawLayout, AtLoc, Range, /*implicit*/false),
+      LikeType(LikeType), SizeOrCount(0), Alignment(~0u)
+  {}
+  
+  /// Construct a `@_rawLayout(likeArrayOf: T, count: N)` attribute.
+  RawLayoutAttr(TypeRepr *LikeType, unsigned Count,
+                SourceLoc AtLoc,
+                SourceRange Range)
+    : DeclAttribute(DAK_RawLayout, AtLoc, Range, /*implicit*/false),
+      LikeType(LikeType), SizeOrCount(Count), Alignment(0)
+  {}
+  
+  /// Construct a `@_rawLayout(size: N, alignment: M)` attribute.
+  RawLayoutAttr(unsigned Size, unsigned Alignment,
+                SourceLoc AtLoc,
+                SourceRange Range)
+    : DeclAttribute(DAK_RawLayout, AtLoc, Range, /*implicit*/false),
+      LikeType(nullptr), SizeOrCount(Size), Alignment(Alignment)
+  {}
+  
+  /// Return the type whose single-element layout the attribute type should get
+  /// its layout from. Returns null if the attribute specifies an array or manual
+  /// layout.
+  TypeRepr *getScalarLikeType() const {
+    if (!LikeType)
+      return nullptr;
+    if (Alignment != ~0u)
+      return nullptr;
+    return LikeType;
+  }
+  
+  /// Return the type whose array layout the attribute type should get its
+  /// layout from, along with the size of that array. Returns None if the
+  /// attribute specifies scalar or manual layout.
+  llvm::Optional<std::pair<TypeRepr *, unsigned>> getArrayLikeTypeAndCount() const {
+    if (!LikeType)
+      return llvm::None;
+    if (Alignment == ~0u)
+      return llvm::None;
+    return std::make_pair(LikeType, SizeOrCount);
+  }
+  
+  /// Return the size and alignment of the attributed type. Returns
+  /// None if the attribute specifies layout like some other type.
+  llvm::Optional<std::pair<unsigned, unsigned>> getSizeAndAlignment() const {
+    if (LikeType)
+      return llvm::None;
+    return std::make_pair(SizeOrCount, Alignment);
+  }
+  
+  /// Set the resolved type.
+  void setResolvedLikeType(Type ty) {
+    assert(LikeType && "doesn't have a like type");
+    ResolvedLikeType = ty;
+  }
+  
+  /// Return the type whose single-element layout the attribute type should get
+  /// its layout from. Returns None if the attribute specifies an array or manual
+  /// layout.
+  llvm::Optional<Type> getResolvedScalarLikeType() const {
+    if (!LikeType)
+      return llvm::None;
+    if (Alignment != ~0u)
+      return llvm::None;
+    return ResolvedLikeType;
+  }
+  
+  /// Return the type whose array layout the attribute type should get its
+  /// layout from, along with the size of that array. Returns None if the
+  /// attribute specifies scalar or manual layout.
+  llvm::Optional<std::pair<Type, unsigned>> getResolvedArrayLikeTypeAndCount() const {
+    if (!LikeType)
+      return llvm::None;
+    if (Alignment == ~0u)
+      return llvm::None;
+    return std::make_pair(ResolvedLikeType, SizeOrCount);
+  }
+  
+  static bool classof(const DeclAttribute *DA) {
+    return DA->getKind() == DAK_RawLayout;
+  }
+};
+
 /// Predicate used to filter MatchingAttributeRange.
 template <typename ATTR, bool AllowInvalid> struct ToAttributeKind {
   ToAttributeKind() {}

include/swift/AST/DiagnosticsIRGen.def

@@ -49,6 +49,9 @@ ERROR(alignment_less_than_natural,none,
 ERROR(alignment_more_than_maximum,none,
       "@_alignment cannot increase alignment above maximum alignment of %0",
       (unsigned))
+      
+ERROR(raw_layout_dynamic_type_layout_unsupported,none,
+      "@_rawLayout is not yet supported for layouts dependent on generic types", ())
 
 ERROR(temporary_allocation_size_negative,none,
       "allocation capacity must be greater than or equal to zero", ())

include/swift/AST/DiagnosticsParse.def

@@ -1846,6 +1846,17 @@ ERROR(documentation_attr_metadata_expected_text,none,
 ERROR(documentation_attr_duplicate_metadata,none,
       "cannot give more than one metadata argument to the same item", ())
 
+ERROR(attr_rawlayout_expected_label,none,
+      "expected %0 argument to @_rawLayout attribute", (StringRef))
+ERROR(attr_rawlayout_expected_integer_size,none,
+      "expected integer literal size in @_rawLayout attribute", ())
+ERROR(attr_rawlayout_expected_integer_alignment,none,
+      "expected integer literal alignment in @_rawLayout attribute", ())
+ERROR(attr_rawlayout_expected_integer_count,none,
+      "expected integer literal count in @_rawLayout attribute", ())
+ERROR(attr_rawlayout_expected_params,none,
+      "expected %1 argument after %0 argument in @_rawLayout attribute", (StringRef, StringRef))
+
 //------------------------------------------------------------------------------
 // MARK: Generics parsing diagnostics
 //------------------------------------------------------------------------------

include/swift/AST/DiagnosticsSema.def

@@ -3811,6 +3811,15 @@ ERROR(attr_MainType_without_main,none,
 #undef SELECT_APPLICATION_MAIN
 #undef SELECT_APPLICATION_DELEGATE
 
+ERROR(attr_rawlayout_experimental,none,
+      "the @_rawLayout attribute is experimental", ())
+ERROR(attr_rawlayout_cannot_be_copyable,none,
+      "type with @_rawLayout cannot be copied and must be declared ~Copyable", ())
+ERROR(attr_rawlayout_cannot_have_stored_properties,none,
+      "type with @_rawLayout cannot have stored properties", ())
+ERROR(attr_rawlayout_cannot_have_alignment_attr,none,
+      "type with @_rawLayout cannot also have an @_alignment attribute", ())
+      
 // lazy
 ERROR(lazy_not_on_let,none,
       "'lazy' cannot be used on a let", ())
@@ -5342,6 +5351,8 @@ ERROR(concurrent_value_inherit,none,
       (bool, DeclName))
 ERROR(non_sendable_type,none,
       "type %0 does not conform to the 'Sendable' protocol", (Type))
+ERROR(sendable_raw_storage,none,
+      "struct %0 with @_rawLayout does not conform to the 'Sendable' protocol", (DeclName))
 
 WARNING(unchecked_conformance_not_special,none,
       "@unchecked conformance to %0 has no meaning", (Type))

include/swift/Basic/Features.def

@@ -224,6 +224,9 @@ EXPERIMENTAL_FEATURE(DeferredSendableChecking, false)
 /// "playground transform" is enabled.
 EXPERIMENTAL_FEATURE(PlaygroundExtendedCallbacks, true)
 
+/// Enable the `@_rawLayout` attribute.
+EXPERIMENTAL_FEATURE(RawLayout, true)
+
 #undef EXPERIMENTAL_FEATURE_EXCLUDED_FROM_MODULE_INTERFACE
 #undef EXPERIMENTAL_FEATURE
 #undef UPCOMING_FEATURE

lib/AST/ASTPrinter.cpp

@@ -3439,6 +3439,10 @@ static bool usesFeatureBuiltinModule(Decl *decl) {
   return false;
 }
 
+static bool usesFeatureRawLayout(Decl *decl) {
+  return decl->getAttrs().hasAttribute<RawLayoutAttr>();
+}
+
 static bool hasParameterPacks(Decl *decl) {
   if (auto genericContext = decl->getAsGenericContext()) {
     auto sig = genericContext->getGenericSignature();

lib/AST/Attr.cpp

@@ -1462,6 +1462,28 @@ bool DeclAttribute::printImpl(ASTPrinter &Printer, const PrintOptions &Options,
     Printer << ")";
     break;
   }
+  
+  case DAK_RawLayout: {
+    auto *attr = cast<RawLayoutAttr>(this);
+    Printer.printAttrName("@_rawLayout");
+    Printer << "(";
+    
+    if (auto sizeAndAlign = attr->getSizeAndAlignment()) {
+      Printer << "size: " << sizeAndAlign->first
+              << ", alignment: " << sizeAndAlign->second;
+    } else if (auto type = attr->getScalarLikeType()) {
+      Printer << "like: ";
+      type->print(Printer, Options);
+    } else if (auto array = attr->getArrayLikeTypeAndCount()) {
+      Printer << "likeArrayOf: ";
+      array->first->print(Printer, Options);
+      Printer << ", count: " << array->second;
+    } else {
+      llvm_unreachable("unhandled @_rawLayout form");
+    }
+    Printer << ")";
+    break;
+  }
 
   case DAK_Count:
     llvm_unreachable("exceed declaration attribute kinds");
@@ -1653,6 +1675,8 @@ StringRef DeclAttribute::getAttrName() const {
     case MacroSyntax::Attached:
       return "attached";
     }
+  case DAK_RawLayout:
+    return "_rawLayout";
   }
   llvm_unreachable("bad DeclAttrKind");
 }

lib/IRGen/DebugTypeInfo.cpp

@@ -37,11 +37,12 @@ DebugTypeInfo::DebugTypeInfo(swift::Type Ty, llvm::Type *FragmentStorageTy,
   assert(Align.getValue() != 0);
 }
 
-/// Determine whether this type has a custom @_alignment attribute.
+/// Determine whether this type has an attribute specifying a custom alignment.
 static bool hasDefaultAlignment(swift::Type Ty) {
   if (auto CanTy = Ty->getCanonicalType())
     if (auto *TyDecl = CanTy.getNominalOrBoundGenericNominal())
-      if (TyDecl->getAttrs().getAttribute<AlignmentAttr>())
+      if (TyDecl->getAttrs().getAttribute<AlignmentAttr>()
+          || TyDecl->getAttrs().getAttribute<RawLayoutAttr>())
         return false;
   return true;
 }

lib/IRGen/GenReflection.cpp

@@ -1674,13 +1674,15 @@ void IRGenModule::emitFieldDescriptor(const NominalTypeDecl *D) {
       needsFieldDescriptor = false;
   }
 
-  // If the type has custom @_alignment, emit a fixed record with the
-  // alignment since remote mirrors will need to treat the type as opaque.
+  // If the type has custom @_alignment, @_rawLayout, or other manual layout
+  // attributes, emit a fixed record with the size and alignment since the
+  // remote mirrors will need to treat the type as opaque.
   //
   // Note that we go on to also emit a field descriptor in this case,
   // since in-process reflection only cares about the types of the fields
   // and does not independently re-derive the layout.
-  if (D->getAttrs().hasAttribute<AlignmentAttr>()) {
+  if (D->getAttrs().hasAttribute<AlignmentAttr>()
+      || D->getAttrs().hasAttribute<RawLayoutAttr>()) {
     auto &TI = getTypeInfoForUnlowered(T);
     if (isa<FixedTypeInfo>(TI)) {
       needsOpaqueDescriptor = true;

lib/IRGen/GenType.cpp

@@ -2831,9 +2831,10 @@ SILType irgen::getSingletonAggregateFieldType(IRGenModule &IGM, SILType t,
         || structDecl->hasClangNode())
       return SILType();
 
-    // A single-field struct with custom alignment has different layout from its
+    // A single-field struct with custom layout has different layout from its
     // field.
-    if (structDecl->getAttrs().hasAttribute<AlignmentAttr>())
+    if (structDecl->getAttrs().hasAttribute<AlignmentAttr>()
+        || structDecl->getAttrs().hasAttribute<RawLayoutAttr>())
       return SILType();
 
     // If there's only one stored property, we have the layout of its field.