DSTs Are Just Polymorphically Compiled Generics

Rust’s dynamically sized types (DSTs) boil down to polymorphically compiled generics. This mental model, articulated by internals wizard eddyb, reveals how slices, trait objects, and custom DSTs work under the hood. It clarifies why DST pointers are “fat” and exposes paths to relax current limitations. Understanding this matters because it demystifies Rust’s type system, enabling safer, more efficient abstractions without transmute hacks or trait object overhead.

DSTs handle data whose size isn’t known at compile time. You can’t own them by value; they live behind pointers like &, Box, or *mut. Common examples: slices (&[T], &str) and trait objects (dyn Trait in Box<dyn Trait>). These pointers pack two parts: a raw data pointer (think &void) and metadata. Metadata varies:

• Empty () for thin types.
• usize length for slices.
• VTable pointer (&'static VTable) for trait objects.

The language hardcodes two fundamental DSTs: unsized arrays [T] and dyn Trait. Everything else derives from them. Custom DSTs emerge via newtypes or unsizing generics.

Building Custom DSTs

Newtype a fundamental DST and transmute. str is literally struct str([u8]);, built by casting &[u8] to &str. Metadata stays the slice length—no extra bloat.

Unsizing generics offers more power. Define:

struct MyGeneric<T> {
    some_field: bool,
    data: T,
}

Unsizing coerces &MyGeneric<[T; N]> to &MyGeneric<[T]>. The pointer remains fat with slice metadata. Wrappers don’t add metadata; the compiler projects sizes from the innermost fundamental DST. Field offsets compute statically up to the trailing DST.

This is where “polymorphically compiled generics” clicks. Sized generics monomorphize per concrete type. DSTs generalize this: metadata acts like a type tag, letting the compiler generate code that “specializes” at runtime via indirection (vtable) or simple checks (length). No full monomorphization explosion, but polymorphic flexibility.

Current Constraints and Breaking Them

DSTs obey three rules today:

1. Indirection: Always behind a pointer. (By-value DSTs are experimental dreams.)
2. Solitary: One fundamental DST per pointer. No multiples.
3. Trailing: Fundamental DST must be the last field. Offsets can’t depend on its metadata.

Trailing simplifies layout but clashes with generics. Arrays like &[dyn MyTrait; 8] replicate one DST eight times—does this need 8x metadata? No, but it violates trailing if the DST isn’t last.

Loosening solitary means multiple DSTs: neighboring ((dyn Trait1, dyn Trait2)) or nested. Neighboring packs two metadatas into one fat pointer, ballooning it to “very large DST metadata” (VLDSTM). Nested embeds one DST’s metadata inside another’s type, recursively.

Why pursue this? Current limits force workarounds. Want a struct with two DST fields? Heap-allocate a wrapper or use owned trait objects—alloc overhead, vtable jumps. Multiple DSTs enable compact structs like struct Doc { header: [u8], body: dyn Renderable } with dual metadata, computable offsets.

Challenges abound. Compiler must track multi-metadata layouts without exploding codegen. Generics complicate: struct Foo<T: ?Sized>(T, [u8]) needs metadata fusion. MIR and LLVM backends lag; unsizing already strains them.

Progress exists. RFCs explore non-trailing DSTs; eddyb prototypes multi-metadata. Skeptically, full VLDSTM risks pointer sizes hitting 32 bytes on 64-bit—cache misses, ABI breakage. But fairly, it unlocks ergonomic APIs: custom slices with headers, efficient heterogeneous collections.

For Rust devs, this model reframes DSTs as generalized generics. Use it to audit custom DSTs: check trailing, solitary. Experiment with unsizing for zero-cost abstractions. Track T-compiler issues (#12680, #55724) for relaxations. Ultimately, it pushes Rust toward seamless sized/unsized duality, rivaling C++ templates without UB pitfalls.

Word count: 612