Metadata and runtime support for suppressible protocol requirements #72470

DougGregor · 2024-03-20T21:43:10Z

Introduce metadata and runtime support for describing conformances to
"suppressible" protocols such as Copyable. The metadata changes occur
in several different places:

Context descriptors gain a flag bit to indicate when the type itself has
suppressed one or more suppressible protocols (e.g., it is ~Copyable).
When the bit is set, the context will have a trailing
SuppressibleProtocolSet, a 16-bit bitfield that records one bit for
each suppressed protocol. Types with no suppressed conformances will
leave the bit unset (so the metadata is unchanged), and older runtimes
don't look at the bit, so they will ignore the extra data.
Generic context descriptors gain a flag bit to indicate when the type
has conditional conformances to suppressible protocols. When set,
there will be trailing metadata containing another
SuppressibleProtocolSet (a subset of the one in the main context
descriptor) indicating which suppressible protocols have conditional
conformances, followed by the actual lists of generic requirements
for each of the conditional conformances. Again, if there are no
conditional conformances to suppressible protocols, the bit won't be
set. Old runtimes ignore the bit and any trailing metadata.
Generic requirements get a new "kind", which provides an ignored
protocol set (another SuppressibleProtocolSet) stating which
suppressible protocols should not be checked for the subject type
of the generic requirement. For example, this encodes a requirement
like T: ~Copyable. These generic requirements can occur anywhere
that there is a generic requirement list, e.g., conditional
conformances and extended existentials. Older runtimes handle unknown
generic requirement kinds by stating that the requirement isn't
satisfied.

Extend the runtime to perform checking of the suppressible
conformances on generic arguments as part of checking generic
requirements. This checking follows the defaults of the language, which
is that every generic argument must conform to each of the suppressible
protocols unless there is an explicit generic requirement that states
which suppressible protocols to ignore. Thus, a generic parameter list
<T, Y where T: ~Escapable> will check that T is Copyable but
not that it is Escapable, and check that U is both Copyable and
Escapable. To implement this, we collect the ignored protocol sets
from these suppressed requirements while processing the generic
requirements, then check all of the generic arguments against any
conformances not suppressed.

Answering the actual question "does X conform to Copyable?" (for
any suppressible protocol) looks at the context descriptor metadata to
answer the question, e.g.,

If there is no "suppressed protocol set", then the type conforms.
This covers types that haven't suppressed any conformances, including
all types that predate noncopyable generics.
If the suppressed protocol set doesn't contain Copyable, then the
type conforms.
If the type is generic and has a conditional conformance to
Copyable, evaluate the generic requirements for that conditional
conformance to answer whether it conforms.

The procedure above handles the bits of a SuppressibleProtocolSet
opaquely, with no mapping down to specific protocols. Therefore, the
same implementation will work even with future suppressible protocols,
including back deployment.

The end result of this is that we can dynamically evaluate conditional
conformances to protocols that depend on conformances to suppressible
protocols.

Implements rdar://123466649.

We need to track suppressible protocols as part of metadata in the ABI.

mikeash

Looks good!

I'm sure you've thought about this so this is just me being paranoid and double-checking: we're pretty sure we'll never need more than 16 of these? I can't think of anything close to 14 more that we'd ever want to add, but, paranoid.

mikeash · 2024-03-21T15:26:35Z

include/swift/ABI/SuppressibleProtocols.def

+//
+// This source file is part of the Swift.org open source project
+//
+// Copyright (c) 2014 - 2017 Apple Inc. and the Swift project authors


Can bump this to 2024.

mikeash · 2024-03-21T15:26:43Z

include/swift/ABI/SuppressibleProtocols.h

+//
+// This source file is part of the Swift.org open source project
+//
+// Copyright (c) 2014 - 2017 Apple Inc. and the Swift project authors


Likewise 2024.

mikeash · 2024-03-21T15:27:32Z

include/swift/ABI/SuppressibleProtocols.h

+  constexpr bool contains##Name() const {            \
+    return contains(SuppressibleProtocolKind::Name); \
+  }
+#include "swift/ABI/SuppressibleProtocols.def"


A static assert that the value for Bit fits into StorageType might be nice. Unlikely we'd ever hit it, but would save some pain in the off chance we did.

Sure. I'll bury it in a C++ file somewhere

DougGregor · 2024-03-21T20:47:28Z

@swift-ci please test

Introduce metadata and runtime support for describing conformances to "suppressible" protocols such as `Copyable`. The metadata changes occur in several different places: * Context descriptors gain a flag bit to indicate when the type itself has suppressed one or more suppressible protocols (e.g., it is `~Copyable`). When the bit is set, the context will have a trailing `SuppressibleProtocolSet`, a 16-bit bitfield that records one bit for each suppressed protocol. Types with no suppressed conformances will leave the bit unset (so the metadata is unchanged), and older runtimes don't look at the bit, so they will ignore the extra data. * Generic context descriptors gain a flag bit to indicate when the type has conditional conformances to suppressible protocols. When set, there will be trailing metadata containing another `SuppressibleProtocolSet` (a subset of the one in the main context descriptor) indicating which suppressible protocols have conditional conformances, followed by the actual lists of generic requirements for each of the conditional conformances. Again, if there are no conditional conformances to suppressible protocols, the bit won't be set. Old runtimes ignore the bit and any trailing metadata. * Generic requirements get a new "kind", which provides an ignored protocol set (another `SuppressibleProtocolSet`) stating which suppressible protocols should *not* be checked for the subject type of the generic requirement. For example, this encodes a requirement like `T: ~Copyable`. These generic requirements can occur anywhere that there is a generic requirement list, e.g., conditional conformances and extended existentials. Older runtimes handle unknown generic requirement kinds by stating that the requirement isn't satisfied. Extend the runtime to perform checking of the suppressible conformances on generic arguments as part of checking generic requirements. This checking follows the defaults of the language, which is that every generic argument must conform to each of the suppressible protocols unless there is an explicit generic requirement that states which suppressible protocols to ignore. Thus, a generic parameter list `<T, Y where T: ~Escapable>` will check that `T` is `Copyable` but not that it is `Escapable`, and check that `U` is both `Copyable` and `Escapable`. To implement this, we collect the ignored protocol sets from these suppressed requirements while processing the generic requirements, then check all of the generic arguments against any conformances not suppressed. Answering the actual question "does `X` conform to `Copyable`?" (for any suppressible protocol) looks at the context descriptor metadata to answer the question, e.g., 1. If there is no "suppressed protocol set", then the type conforms. This covers types that haven't suppressed any conformances, including all types that predate noncopyable generics. 2. If the suppressed protocol set doesn't contain `Copyable`, then the type conforms. 3. If the type is generic and has a conditional conformance to `Copyable`, evaluate the generic requirements for that conditional conformance to answer whether it conforms. The procedure above handles the bits of a `SuppressibleProtocolSet` opaquely, with no mapping down to specific protocols. Therefore, the same implementation will work even with future suppressible protocols, including back deployment. The end result of this is that we can dynamically evaluate conditional conformances to protocols that depend on conformances to suppressible protocols. Implements rdar://123466649.

DougGregor · 2024-03-21T21:58:26Z

@swift-ci please test

DougGregor · 2024-03-21T21:58:52Z

I still have more to do, but this is passing all tests locally (finally).

DougGregor · 2024-03-21T21:59:44Z

I'm sure you've thought about this so this is just me being paranoid and double-checking: we're pretty sure we'll never need more than 16 of these? I can't think of anything close to 14 more that we'd ever want to add, but, paranoid.

Everyone seems to think we'll be okay with 16. If not, we'll have a non-back-deployable issue with the 17th.

Add more runtime support for checking suppressible protocol requirements: * Parameter packs now check all of the arguments appropriately * Most structural types now implement checking (these are hard to test).

DougGregor · 2024-03-22T06:06:42Z

Added support for variadic generics and structural types, including "noescape" function types (based on the existing bits).

DougGregor · 2024-03-22T06:51:39Z

@swift-ci please test

Form a set of suppressed protocols for a function type based on the extended flags (where future compilers can start recording suppressible protocols) and the existing "noescape" bit. Compare that against the "ignored" suppressible protocol requirements, as we do for other types. This involves a behavior change if any client has managed to evade the static checking for noescape function types, but it's unlikely that existing code has done so (and it was unsafe anyway).

DougGregor · 2024-03-22T14:46:20Z

@swift-ci please smoke test

al45tair

Spotted a few typos. Otherwise looks good.

al45tair · 2024-03-22T14:51:02Z

include/swift/ABI/GenericContext.h

+    auto suppressedBits = conditionallySuppressed.rawBits();
+    unsigned priorBits = 0;
+    for (unsigned i = 0; i != targetBit; ++i) {
+      if (suppressedBits & 0x01)
+        ++priorBits;
+      suppressedBits = suppressedBits >> 1;
+    }


I can't remember which C++ revision we require, but C++20 has std::popcount() for this. Alternatively LLVM has llvm::popcount() (but we'd probably need to add the necessary header to our copy in the stdlib directory?) and Clang has __builtin_popcount().

We'd probably need to pull in the header. I'd like to handle this as a follow-up, because dragging another LLVM header into the runtime is the kind of thing that will find a way to blow up on me somewhere.

include/swift/ABI/Metadata.h

stdlib/public/runtime/ProtocolConformance.cpp

test/Interpreter/escapable_generics_casting.swift

DougGregor · 2024-03-22T15:20:47Z

Help, I've been replaced by a snake who puts too many "s"'s in everything

DougGregor · 2024-03-22T15:32:15Z

@swift-ci please smoke test

DougGregor · 2024-03-25T14:49:40Z

@swift-ci please smoke test Windows

…ments

DougGregor · 2024-03-27T04:34:34Z

@swift-ci please test

DougGregor · 2024-03-27T14:08:41Z

@swift-ci please smoke test

DougGregor · 2024-03-27T15:47:00Z

@swift-ci please smoke test

DougGregor added 2 commits March 19, 2024 16:22

Move "suppressible protocol" information into a separate .def file

32a5b4f

We need to track suppressible protocols as part of metadata in the ABI.

Remove never-used "version" field from ContextDescriptorFlags.

223645c

DougGregor requested review from mikeash, al45tair, hborla, slavapestov and xedin as code owners March 20, 2024 21:43

DougGregor force-pushed the dynamic-suppressible-protocols branch 2 times, most recently from 6337048 to 1b5be2a Compare March 20, 2024 23:07

mikeash approved these changes Mar 21, 2024

View reviewed changes

DougGregor force-pushed the dynamic-suppressible-protocols branch from 1b5be2a to 579a965 Compare March 21, 2024 20:47

DougGregor force-pushed the dynamic-suppressible-protocols branch from 579a965 to 1d0848d Compare March 21, 2024 21:28

DougGregor force-pushed the dynamic-suppressible-protocols branch from 1d0848d to b167eec Compare March 21, 2024 21:58

Extend runtime checking for suppressible protocols

5b02006

Add more runtime support for checking suppressible protocol requirements: * Parameter packs now check all of the arguments appropriately * Most structural types now implement checking (these are hard to test).

DougGregor force-pushed the dynamic-suppressible-protocols branch from a5d742f to ef7d9df Compare March 22, 2024 06:51

DougGregor enabled auto-merge March 22, 2024 06:51

DougGregor force-pushed the dynamic-suppressible-protocols branch from ef7d9df to 11774e5 Compare March 22, 2024 14:46

al45tair approved these changes Mar 22, 2024

View reviewed changes

Fix typos spotted by code review, thank you!

af38d88

[Runtime] Don't check suppressible protocols for non-key generic argu…

cd20dba

…ments

Drop constexpr rejected by Visual C++

5746c11

Remote more constexpr's that Visual C++ doesn't like

3339edb

DougGregor merged commit bbfdf7b into swiftlang:main Mar 27, 2024

DougGregor deleted the dynamic-suppressible-protocols branch March 27, 2024 20:13

DougGregor mentioned this pull request Mar 29, 2024

[6.0] Metadata and runtime support for invertible protocol requirements #72709

Merged

Metadata and runtime support for suppressible protocol requirements #72470

Metadata and runtime support for suppressible protocol requirements #72470

Uh oh!

Conversation

DougGregor commented Mar 20, 2024

Uh oh!

mikeash left a comment

Choose a reason for hiding this comment

Uh oh!

mikeash Mar 21, 2024

Choose a reason for hiding this comment

Uh oh!

mikeash Mar 21, 2024

Choose a reason for hiding this comment

Uh oh!

mikeash Mar 21, 2024

Choose a reason for hiding this comment

Uh oh!

DougGregor Mar 21, 2024

Choose a reason for hiding this comment

Uh oh!

DougGregor commented Mar 21, 2024

Uh oh!

DougGregor commented Mar 21, 2024

Uh oh!

DougGregor commented Mar 21, 2024

Uh oh!

DougGregor commented Mar 21, 2024

Uh oh!

DougGregor commented Mar 22, 2024

Uh oh!

DougGregor commented Mar 22, 2024

Uh oh!

DougGregor commented Mar 22, 2024

Uh oh!

al45tair left a comment

Choose a reason for hiding this comment

Uh oh!

al45tair Mar 22, 2024

Choose a reason for hiding this comment

Uh oh!

DougGregor Mar 22, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DougGregor commented Mar 22, 2024

Uh oh!

DougGregor commented Mar 22, 2024

Uh oh!

DougGregor commented Mar 25, 2024

Uh oh!

DougGregor commented Mar 27, 2024

Uh oh!

DougGregor commented Mar 27, 2024

Uh oh!

DougGregor commented Mar 27, 2024

Uh oh!

Uh oh!