Hierarchy of Sized traits

[davidtwco]

All of Rust's types are either sized, which implement the Sized trait and have a statically known size during compilation, or unsized, which do not implement the Sized trait and are assumed to have a size which can be computed at runtime. However, this dichotomy misses two categories of type - types whose size is unknown during compilation but is a runtime constant, and types whose size can never be known. Supporting the former is a prerequisite to stable scalable vector types and supporting the latter is a prerequisite to unblocking extern types. This RFC proposes a hierarchy of Sized traits in order to be able to support these use cases.

This RFC relies on experimental, yet-to-be-RFC'd const traits, so this is blocked on that. I haven't squashed any of the previous revisions but can do so if/when this is approved. Already discussed in the 2024-11-13 t-lang design meeting with feedback incorporated.

See this comment for the most recent summary of changes to this RFC since it was opened.

Rendered

[Ericson2314]

Wow, a funny (but good!) feeling to think that this might finally get resolved after a decade. :)

Instead of introducing a new marker trait, std::ptr::Pointee could be re-used if there were some mechanism to indicate that associated types or methods could only be referred to with fully-qualified syntax. Alternatively, it would be possible to introduce forward-compatibility lints in current edition, the new traits were introduced in the next edition and the edition migration previously described in the next next edition.

I think std::ptr::Pointee should definitely be reused. I am not sure which alternative is, but it seems to me like the right thing to do is:

• Use std::ptr::Pointee
• Do the hack to avoid new ambiguity in the current addition
• Remove the hack in the next addition

---

Also, I agree with @RalfJung than the ? avoidance is quite confusing, and we are mixing up syntax and semantics. The bottom line is that semantically, the trait hierarchy is the free lattice over the trait partial order. Decoding the complex math speak: that means the minimum element (or maxiumum, whatever way you want the arrows to point) is not a trait but the empty set of traits. So we need to specify what that means.

After we do that, we we can debate syntax.

In particular, the "Why have Pointee?" is a bit misleading in that semantically there currently is indeed no reason, it is just there for syntactic purposes.

However, if we reuse std::ptr::Pointee then there is a semantic reason, namely the associated type. ({} / no traits has no associated types or any other trait members, because it has no traits!) So here is an extra incentive to reuse std::ptr::Pointee --- one can side-step what @RalfJung and I are saying :).

At the point I still think it is good education to talk about the empty trait set as:

• You can't use it behind a pointer
• You can't use it as a value
• You could use it in PhantomData, but probably this is left as future work
• Maybe you can't quantify over it all (as opposed to you can, but then you can't use the variable anywhere, and then its an unused type parameter error)

Oh and a final thing on semantics, I might also also present the back compat / editions story this way:

• Old semantic hierarchy
• New semantic hierarchy
• mapping from old to new semantics, (old {} (no traits) is mapped not to new {} but to const MetaSized) --- in particular, do not speak of ?Size when describing this semantic mapping.
• mapping from new syntax in next edition maps to new semantics.

In particular, the existing old syntax maps to the new semantics via the old semantics; we can cleanly factor that into those two steps, and I think this brings conceptual clarity.

[programmerjake]

• Use std::ptr::Pointee
• Do the hack to avoid new ambiguity in the current addition
• Remove the hack in the next addition

imo having an associated type on practically every type will be annoying, so I think it might be useful to require writing <T as Pointee>::Metadata instead of T::Metadata on all editions.

[workingjubilee]

the trait's unstable, if it's just a matter of bikeshedding we can rename the type PtrMetadata or sth

[davidtwco]

I think std::ptr::Pointee should definitely be reused. I am not sure which alternative is, but it seems to me like the right thing to do is:

• Use std::ptr::Pointee
• Do the hack to avoid new ambiguity in the current addition
• Remove the hack in the next addition

I've added this as one of the unresolved questions.

In particular, the "Why have Pointee?" is a bit misleading in that semantically there currently is indeed no reason, it is just there for syntactic purposes.

I've rewritten this section, it should have been updated earlier with the addition of the alternatives which keep the ?Sized syntax and clarified that it's just a syntactic difference.

At the point I still think it is good education to talk about the empty trait set as:

• You can't use it behind a pointer
• You can't use it as a value
• You could use it in PhantomData, but probably this is left as future work
• Maybe you can't quantify over it all (as opposed to you can, but then you can't use the variable anywhere, and then its an unused type parameter error)

Oh and a final thing on semantics, I might also also present the back compat / editions story this way:

• Old semantic hierarchy
• New semantic hierarchy
• mapping from old to new semantics, (old {} (no traits) is mapped not to new {} but to const MetaSized) --- in particular, do not speak of ?Size when describing this semantic mapping.
• mapping from new syntax in next edition maps to new semantics.

In particular, the existing old syntax maps to the new semantics via the old semantics; we can cleanly factor that into those two steps, and I think this brings conceptual clarity.

I haven't made these changes at the moment - I've had good feedback about the explanations in the RFC and would prefer not to make major changes to that until the proposal itself needs changing.

[davidtwco]

I've updated the previous summary comment so it is still accurate.

[davidtwco]

I've pushed one small change with a reference to the now-upstream const traits RFC in #3762.

[safinaskar]

Linux has at its syscall boundary struct statmount. See here for details: https://brauner.io/2024/12/16/list-all-mounts.html . Its first field is size of whole struct itself. And it is u32 on all platforms! So, yes, I think we should support such structs. Note: this is not size of DST in its end, it is size of whole struct.

There is even more exotic case: struct dirent ( https://www.man7.org/linux/man-pages/man3/readdir.3.html ). (Yes, I already talked about it in previous comment, but I want to add more.) It embeds its own size, too. But not as a first field, but as a third. :) And that field (d_reclen) is not usize, it is u16! And again, this is not size of last DST field, this is size of struct itself.

Note: manpage says that size of last field is 256. This is a lie. In fact, this is DST, and it can be both bigger and smaller than 256.

Also, the string in the last field is always proper null-terminated C string, so we have two different ways of determining struct size: by calling strlen on d_name (with necessary calculations) and by reading d_reclen. So, technically d_reclen field is redundant. And to properly support this struct in Rust, Rust should have at least one of the following two features:

• ThinCStr we all dream about. It should be allowed to be located in the end of a struct (this will support this d_name field)
• Embedding size of a struct (of a whole struct!) inside struct itself. Not necessary as a first field. And not necessary as a usize. u16 should be supported

Not supporting both features will lead to horrible hacks, such as this: https://github.com/rust-lang/rust/blob/c1132470a6986b12503e8000e322e9164c1f03ac/library/std/src/sys/pal/unix/fs.rs#L730-L768 (this is from standard library!)

And again: to support statmount we need possibility to embed size of struct inside struct itself. As a first field of size u32 (even on 64-bit platforms). In this case calling strlen on last field will not help, because it is allowed to contain many embedded null bytes

[RalfJung]

Not supporting both features will lead to horrible hacks, such as this:

In the latest version, this hack is mostly gone. What remains is not so bad: https://github.com/rust-lang/rust/blob/58d6301cad7ac20407e708cd9d1ec499d1804015/library/std/src/sys/pal/unix/fs.rs#L726-L753
(rust-lang/rust#136479 cleans this up further by removing the macro.)

IMO it is not worth contorting the language to natively express esoteric types such as those, as long as we have some way of interfacing with C code using such types (and we do).

[davidtwco]

Another summary comment!

Here are the previous update summaries copied below so you don't need to uncollapse GitHub comments to find it:

2024-11-25

Summary comment: #3729 (comment)

For those following along or catching up, these are the notable the changes to the RFC since this was posted:

• Clarify proposed behaviour for ?Trait syntax for non-Sized, which is currently accepted
  • 23cb8a2
• Stop re-using std::ptr::Pointee and make Pointee its own new marker trait to avoid backwards incompatibility
  • 0c2f6e6, 2831cca
• Clarify backwards compatibility implications of ?Sized syntax and add alternatives to removing the default bound using positive bounds which continue to use ?Sized
  • These are particularly valuable additions to the RFC as some of these alternatives are quite viable and have significant advantages
  • 5f4b8ff, ab8e7f6, b050c2b, 432a77a, 6e124e5, 4faeb83
• Add that relaxing existing bounds in trait methods would be backwards incompatible
  • b4258c7
• Elaborate on necessity of implicit const ValueSized bound on Self type of traits
  • 9158ebb, b71401c, 5573be7
• Add MetaSized alternative to ValueSized which would resolve interactions with mutexes
  • c9e71bd, ef2f5cd
• Clarified that bounds on return types can never be relaxed.
  • 145446d

And these are all the other smaller changes that don't materially impact what is being proposed:

• Fixed some minor wording errors where supertrait/subtrait were used backwards
  • a840d8b, 260c980
• Removed HackMD's rust= syntax from codeblocks
  • 95a1b7d
• Fixed referring to the introduction of a const Sized trait, but rather adding a const modifier to the existing Sized trait
  • b670517
• Added some background/context on dynamic stack allocation
  • b8ea881
• Use current experimental const trait syntax
  • a703feb
• Corrected incorrect syntax for traits
  • fad04b0
• Listed all alternate bounds (adding ~const ValueSized to a list of bounds that it was missing from)
  • 2fb3aaa
• Fixed bound in description of size_of_val changes
  • 397e474
• Corrected description of current size_of_val and align_of_val behaviour
  • 6c1482b
• Corrected description of extern type usage in structs
  • 1f85898
• Mention Rust for Linux's interest in extern type
  • 98af888
• Weaken language in the externref future possibility to make it clear this proposal would not be sufficient on its own to support these
  • 60eab1a
• Re-write Aligned future possibility so that it is clear Aligned couldn't be added to the proposed hierarchy
  • 61f8d6d

At the moment, I prefer the following alternatives to the primary proposal of the RFC, and may re-write to incorporate these as the primary proposal:

• Adding MetaSized (either replacing ValueSized or in addition to it), as per the "What about MetaSized instead of or in addition to ValueSized?" section.
• Using ?Trait syntax instead of positive bounds as per "Adding ?ValueSized"

2024-12-04

Summary comment: #3729 (comment)

These are the major changes since the last summary:

• Change from ValueSized to MetaSized
  • 72743b6, 57b9a85
• Add unresolved question about whether re-using std::ptr::Pointee is important.
  • d6d73b2

These are the minor changes since the last summary:

• Clarify that bounds on parameters used as return types can never be relaxed (return types still need to implement Sized)
  • 145446d
• Clarify that non-const Sized is not nameable in the current edition due to the backwards compatibility migration, only in the next edition
  • b620534
• Clarify which Pointee (marker trait Pointee or std::ptr::Pointee) is referenced in the Custom DSTs future possibility
  • 02d1759
• Improve clarity of the table describing alternative syntax proposals for relaxing bounds
  • 8cbaef9
• Strengthen wording describing use of runtime-sized types in a const context as unsound
  • 9fd3dff
• Expand on motivation for MetaSized
  • 462d016
• Specify that methods of the new traits would be backwards-incompatible too, not just associated types
  • aa21450
• Use the same indentation in code blocks throughout
  • c0ee97d
• Fix typo in list of when each trait gets implemented
  • 7a11fe9
• Add a concrete example of code that would break if a supertrait of Clone were relaxed
  • 762c6d8
• Added note about potential reasons why delaying relaxation of size_of's bound might be desirable
  • 52b776f
• Fixed typo, replacing "alignment" with "size"
  • 5d602fb
• Rewrote "Why have a Pointee trait?" section to clarify that the trait is only necessary due to the proposed removal of ?Sized syntax
  • f26784b
• Updated reference to const traits now that it has an open RFC
  • edec592
• Fixed a broken link
  • 19469dc
• Fixed broken indentation on a bullet list
  • 0f9ec84

These are the alternatives described in the RFC that I think are worth consideration as the primary proposal:

• Using ?Trait syntax instead of positive bounds as per "Adding ?ValueSized"

---

We had a design meeting with the language team yesterday (Feb 5th) which led to the changes described below, a rough summary:

• No significant concerns were raised with the proposal itself (i.e. the specific traits proposed, the edition migration proposed, etc)
• There was a clear consensus on some of the unresolved questions and no strong opinion on others (see 5806017)
• t-lang would be okay with a experiment proceeding with this, but the preference of the const traits implementors is that no experiment take place while const traits' syntax is undecided, as this would increase the work they have to do when making any potential syntax changes requested by t-lang
• It is still unclear how to proceed with regards to the const traits dependency
  • Further experimentation following the meeting (see ) has reinforced the necessity of the const traits dependency
  • There is disagreement within the language team whether the not-yet-accepted const traits dependency blocks this from being accepted (without a possibility stabilisation w/out const traits stabilisation)

I've started an implementation of this - I've more-or-less finished implementing the non-const parts of the proposal, and will proceed with the const parts. It won't land upstream until const traits implementors are okay with it.

These are links to the compare with previous versions of the document (switch to the "Files changed" tab and then the rich diff):

• Changes since the pre-RFC version of this discussed at the first language team design meeting
• Changes since the version of this discussed at the second language team design meeting
• Changes since the 2024-11-25 summary comment
• Changes since the 2024-12-04 summary comment

These are the major changes since the last summary:

• Add note describing a niche but unavoidable backwards incompatibility
  • 6533e10
• Note that const traits isn't a strict dependency
  • 7b99ac6 (superseded by below)
  • 4ba5b07
• Describe the perspective of the language team following our meeting on the unresolved questions (leaving them unresolved but expressing a bias in a direction)
  • 5806017

These are the minor changes since the last summary:

• Describe another alternative which would enable re-using std::ptr::Pointer
  • bef7459
• Note that these traits can be stabilised independently of each other
  • 96a70f9

[Skepfyr]

Thank you for the summary comments they're incredibly useful. I've re-read this and looked through the updates and this proposal feels closer, however I still have a few comments...

Positive trait bounds are growing on me though, I like that they stop us having to worry about what "the empty set of traits" means, however I'm not convinced that's possible, because of supertrait bounds (as described in a comment).

[Skepfyr]

I feel like this RFC is implicitly dependent on unsized locals here, the only way I can see the Clone or Copy traits supporting Sized types is with it. Consider <svint8_t as Clone>::clone that returns an svint8_t but that doesn't have a stack size known at compile time, so you need a bunch of the unsized locals machinery to support it right?

[Skepfyr]

This feels like surprisingly weak motivation for all the const stuff that's making this hierarchy complicated. I wonder if we can replace Clone with:

trait Clone {
    fn clone(&self) -> Self;
    fn clone_from(&mut self, other: &Self) where Self: Sized { ... }
}

and replace Clone bounds with Clone + Sized, admittedly this is more edition shenanigans that feels a bit dirty (and I haven't completely thought it through).

I do agree that having const trait stuff would give you the ability to do some stuff: size_of on non-const Sized types, and have arrays&slices of non-const Sized types (although this would require some complier work). I'm just not clear how useful those are.

[Skepfyr]

I'm still very nervous about this, I believe this is done solely for backwards compatibility reasons but I want to make sure it's clear this is sacrificing forwards compatibility.

If traits gain (debatably keep) an implicit const MetaSized bound then:

• Everything keeps working in a nice way
• People writing libraries will just write trait Foo {} rather than trait Foo: Pointee {} unless they're specifically thinking about unsized types for some reason making them a second class citizen in the ecosystem (and relaxing the bund when asked is a breaking change)
• If we ever add another layer below Pointee (eg externref) then no existing traits will work with them and it would be breaking to relax them. I think this means no std traits could ever support them.

If traits have no implicit bound (also debatably keep, depending on how you imagine this) then:

• You'd have to do this over an edition, and add in : const MetaSized everywhere so that default method implementations keep working. (removing that bound is safe for std, and dubious for crates).
• You'd need to write : Pointee etc if you need to write default method impls, but I think that's a good thing.
• This does force us to work out what you can do with "the empty set of traits", because that what these types have in default trait impls (with no supertraits) (this makes me less favourable towards positive bounds)

[Skepfyr]

I think this was straw-manned slightly in the design meeting, I disagree with the phrasing:

'I don't want to think of the "next most precise thing"'

and

"one level below" in the hierarchy

Neither proposal makes you think like that (ignoring the ?MetaSized below), the ?Sized variant requires you to write the positive bound always (and I'd argue that the T: Pointee case should be written T: ?(const Sized) + Pointee) so the only negative thinking you have to do is "do I need to disable the default const Sized bound?".

However, I agree this discussion can be punted until stabilisation given it has essentially no impact on the semantics of what's going on.

P.S The idea of negative trait aliases (trait Foo = ?Sized;) terrifies me, it feels like it'd be incredibly confusing at the use-site.