allow construction of non-exhaustive structs when using functional update syntax

[metajack]

Proposal

Summary and problem statement

Support instantiation of non-exhaustive structs outside their defining crate
if they use functional update syntax with a base (e.g. ..Default::default()).

Motivation, use-cases, and solution sketches

Modifying core structs requires code changes everywhere in a code base where
those structs are constructed. Often in libraries, construction is
abstracted by constructor methods and this isn't a huge concern. However, structs
used as part of serialization or communication, for example, are often
plain-old-data style types and are constructed directly. It would be nice if
these kinds of structs could undergo certain modifications without requiring
downstream code changes everywhere they are created. #[non_exhaustive]
attributes on structs allow such modification but then prohibit construction
outside of the defining crate.

This proposal would allow construction of #[non_exhaustive] structs if the
construction uses the functional update syntax from a suitable base such as
..Default::default(). This would allow construction outside the defining
crate and require no code changes when common modifications are made such as
adding a new field with a default value.

Links and related work

None

Initial people involved

• Owner, if known: metajack?
• Liaison

What happens now?

This issue is part of the lang-team initiative process. Once this issue is filed, a Zulip topic will be opened for discussion, and the lang-team will review open proposals in its weekly triage meetings. You should receive feedback within a week or two.

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[rustbot]

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[scottmcm]

Concern (more details in the zulip thread, but I think it's important enough to reproduce here):

#[non_exhaustive] was originally designed so that people could stop putting private _unused: () fields in their structs to keep people from constructing them. Now that people have largely taken those out, in favour of using #[non_exhaustive], I think it would be a real shame to have to tell them to put them back again, which is what this change (as originally described) would force even on simple things like #[non_exhaustive] struct MyError {}.

[nikomatsakis]

@scottmcm First, thanks for highlighting key summary points here, I think that's really helpful.

That said, I'd appreciate a bit more detail: can you clarify how this feature would people to use _private fields e.g. with struct MyError { }. What would they be trying to prevent?

If I'm understanding correctly, this feature would allow you to make a new MyError by consuming an old one, but that is basically equivalent to moving it from place to place.

I could see that the feature would allow you to change the types of public fields (at least when combined with some other related RFCs that I don't care to look up right now.)

Anyway, could you elaborate on this note a bit for the benefit of those us (✋) who haven't had time to look into the Zulip thread?

[scottmcm]

The most important piece of context here is that Foo { a: b, ..base} is not, despite what I thought originally,

{
    let mut temp = base;
    temp.a = b;
    temp
}

instead, to the initial surprise of everyone I've talked with, it's

Foo {
    a: b,
    every: base.every,
    other: base.other,
    field: base.field,
    in_the_type: base.in_the_type,
}

And thus requires that all the fields be visible from the FRU location, per https://rust-lang.github.io/rfcs/0736-privacy-respecting-fru.html. (A quick example of why: picture types with Drop impls but Copy fields, such as Vec { ..otherVec }.)

Because of that, allowing non_exhaustive types in FRU would, in isolation, mean that non_exhaustive changes from "you can add any fields" to "you can add public fields, or if you already have private fields then you can add more private fields".

So if this happened, people wanting to get what they currently think they're getting from #[non_exhaustive] would need to put an unused private field back into their types -- and not needing to do that any more is a big part of what non_exhaustive was for in the first place.

See, for example, rust-lang/rust@3f14f4b#diff-945c190996af8e7a1bac96b889fd86cceaccd5a8629bb079d029e34d8e307b2cR123 which made this change:

 #[derive(Debug, Clone, PartialEq, Eq)]
+#[non_exhaustive]
 #[stable(feature = "rust1", since = "1.0.0")]
-pub struct ParseBoolError {
-    pub(super) _priv: (),
-}
+pub struct ParseBoolError;

So thus I think it's unacceptable to just allow FRU through, as that'd be adding a surprise semver requirement to every crate currently expecting to be able to do what https://rust-lang.github.io/rfcs/2008-non-exhaustive.html?highlight=private#structs said they could.

---

There are certainly options for a more nuanced change here, though.

• The most direct one would be to add a variant of non_exhaustive, strawman #[non_exhaustive(pub)], which requires that all the fields are pub, and thus can allow FRU. (This is similar to the idea that after Esteban's field-defaults RFC we might want a #[non_exhaustive(defaults)] to say that any added fields will have defaults, and thus it would be allowed to construct such a type with Foo { fields: here, .. }.)

• As a more broad thing, we could change the FRU desugaring to the thing that most people expect, at which point it would naturally work with non_exhaustive types without issue. However, that's a breaking change per petrochenkov in 2017, so it would need careful sequencing of an edition update. It's also been made harder recently because of (the accepted but not-yet-stable) https://rust-lang.github.io/rfcs/2528-type-changing-struct-update-syntax.html, which only really makes sense in the current desugaring. Amusingly, I mentioned the following on that RFC in 2018:

  I would like to soundly support FRU on non_exhaustive structs and those with private fields at some point

  (Note that just using the different desugar for non_exhaustive types would make removing the attribute a breaking change, which I don't think it currently is, so I think I'm negative on that.)

• Or there could be a different syntax for the alternative desugar. It looks like Foo { field1: xxx, ..move Default::default() } has been mentioned as far back as 2016: https://internals.rust-lang.org/t/proto-rfc-expanded-functional-record-update/3077?u=scottmcm

• A no-lang-change version would just be to add a macro to do it, since the "change some public fields and return it" version doesn't need any special smarts about the types or anything. A basic example of such (hard-coded to Default::default(), but could easily take a base too) can be seen in https://internals.rust-lang.org/t/short-enum-variant-syntax-in-some-cases/13113/9?u=scottmcm

But further details and debating of those options or others should be on zulip.

[nikomatsakis]

Unfortunately I believe @metajack won't have time to drive this forward -- we in the @rust-lang/lang team are still interested in seeing it -- but it would need a new owner!

[cchiw]

@rustbot claim

[rustbot]

Error: The feature assign is not enabled in this repository.
To enable it add its section in the triagebot.toml in the root of the repository.

Please let @rust-lang/release know if you're having trouble with this bot.

[joshtriplett]

@rustbot second

[joshtriplett]

Note: the second here is based on an updated problem-space statement @cchiw is putting together, which we're going to have a design meeting about later this month.