Place traits

text/3921-place-traits.md

@@ -0,0 +1,306 @@
+- Feature Name: `deref_move_trait`
+- Start Date: 2026-01-23
+- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/3921)
+- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)
+
+## Summary
+[summary]: #summary
+
+This RFC introduces the `DerefMove` trait. This trait allows arbitrary types to implement the
+special dereference behavior of the `Box` type. In particular, it allows an arbitrary type
+to act as an owned place allowing values to be (partially) moved out and moved back in
+again.
+
+## Motivation
+[motivation]: #motivation
+
+Currently the Box type is uniquely special in the rust ecosystem. It is unique in acting
+like an owned variable, and allows a number of special optimizations for direct
+instantiation of other types in the storage allocated by it.
+
+This special status comes with two challenges. First of all, Box gets its special status
+by being deeply interwoven with the compiler. This is somewhat problematic as it requires
+exactly matched definitions of how the type looks between various parts of the compiler
+and the standard library. Moving box over to a place trait would provide a more pleasant
+and straightforward interface between the compiler and the box type, at least in regards
+to the move behavior.
+
+Second, it is currently impossible to provide a safe interface for user-defined smart
+pointer types which provide the option of moving data in and out of it. There have been
+identified a number of places where such functionality could be interesting, such as when
+removing values from containers, or when building custom smart pointer types for example
+in the context of an implementation of garbage collection.
+
+## Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+This proposal introduces a new unsafe trait `DerefMove`:
+```rust
+unsafe trait DerefMove: DerefMut {
+    fn place(&self) -> *const Self::Target;
+    fn place_mut(&mut self) -> *mut Self::Target;
+}
+```
+
+The `DerefMove` trait allows values of the type to be treated as a `Box` with a content. That
+is, they behave like a variable of type `Deref::Target`, just (potentially) stored in a
+different location than the stack. The contents can be (partially) moved in and out of
+dereferences of the type, with the borrow checker ensuring soundness of the resulting
+code. As an example, if `Foo` implements `DerefMove` for type `Bar`, the following would be
+valid rust code:
+```rust
+fn baz(mut x: Foo) -> Foo {
+    let y = *x;
+    *x = y.destructive_update()
+    x
+}
+```
+
+In implementing the `DerefMove` trait, the type transfers responsibility for managing its
+content to the compiler. In particular, the type should not assume the contents are
+initialized when `DerefMove::place` and `DerefMove::place_mut` are called.
+
+It also transfers responsibility for dropping the contents to the compiler. This will
+be done in drop glue, and the `Drop::drop` function will therefore see the contents as
+uninitialized.
+
+The transfer of responsibility also puts requirements on the implementer of `DerefMove`. In
+particular, instances of the type where the contents are uninitialized through a means
+other than moving out of a dereference should not be created. Breaking this rule and
+dereferencing the resulting instance is immediate undefined behavior.
+
+There is one oddity in the behavior of types implementing `DerefMove` to be aware of.
+Automatically elaborated dereferences of values of such types will always trigger an abort
+on panic, instead of unwinding when that is enabled. However, generic types constrained to
+only implement `Deref` or `DerefMut` but not `DerefMove` will always unwind on panics during
+dereferencing, even if the underlying type also implements `DerefMove`.
+
+## Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+This proposal introduces one new main language item, the traits `DerefMove`. We also introduce
+a number of secondary language items which are used to make implementation easier and more
+robust, which we shall define as they come up below.
+
+Instances of a type implementing the trait `DerefMove` shall provide a "contents" of type
+`Deref::Target`, which shall act as a place for borrow checking. The implementer of the
+`DerefMove` trait shall ensure that:
+- The `DerefMove::place` and `DerefMove::place_mut` return pointers to the storage of the content
+  valid for the lifetime of the self reference.
+- The `DerefMove::place` and `DerefMove::place_mut` functions shall not assume the contents to
+  be initialized
+- The `Drop::drop` function shall not interact with the content, but only with its storage.
+- With the exception of uninitialization through moving out of a dereference of the instance,
+  the implementer shall ensure that the contents are always initialized when dereferencing
+  or dropping instances of the type.
+
+In return, the compiler shall guarantee that:
+- When the contents are uninitialized through moving out of a dereference of an instance,
+  references to that instance will only be available through invocations of `DerefMove::place`,
+  `DerefMove::place_mut` and `Drop::drop` by the compiler itself.
+- The contents shall be dropped or moved out of by the compiler before invoking `Drop::drop`
+  of the instance.
+
+Furthermore, as there seems to be no good reasons for `DerefMove::place` and `DerefMove::place_mut`
+to panic, the compiler shall handle panics in non-explicit calls to these functions by
+aborting. This allows the compiler to compile code more efficiently and provides more
+avenues for optimizations.
+
+Note that the above requirements explicitly allow changes in the storage location of the
+contents. This is only restricted by the contract imposed by `DerefMove` during the lifetime
+of self pointers passed to `DerefMove::place` and `DerefMove::place_mut`.
+
+As a consequence, `Pin<Foo>` does not automatically satisfy all the requirements of Pin
+when Foo implements place. This will need to be verified on an implementation by
+implementation basis.
+
+### Implementation details
+
+Given the above contract, dereferences of a type implementing `DerefMove` can be lowered
+directly to MIR, only being elaborated after borrow checking. This allows the borrow
+checker and drop elaboration logic to provide the guarantees above, and ensure that
+dereferences are sound in the presence of moves out of the contents.
+
+The dereferences and drops of the contained value can be elaborated in the passes after
+borrow checking. This process will be somewhat similar to what is already done for Box,
+with the difference that dereferences of types implementing `DerefMove` may panic. These
+panics will be handled by aborting, avoiding significant changes in the control flow graph.
+
+In order to generate the function calls to the `DerefMove::place` and `DerefMove::place_mut`
+during the dereference elaboration we propose making these functions additional language
+items.
+
+## Drawbacks
+[drawbacks]: #drawbacks
+
+There are three main drawbacks to the design as outlined above. First, the traits are
+unsafe and come with quite an extensive list of requirements on the implementing type.
+This makes them relatively tricky and risky to implement, as breaking the requirements
+could result in undefined behavior that is difficult to find.
+
+Second, with the current design the underlying type is no longer aware of whether or not
+the space it has allocated for the value is populated or not. This inhibits functionality
+which would use this information on drop to automate removal from a container. Note
+however that such use cases can use a workaround with the user explicitly requesting
+removal before being able to move out of a smart-pointer like type.
+
+Finally, the type does not have runtime-awareness of when the value is exactly added.
+This means that the proposed traits are not suitable for providing transparent locking of
+shared variables to end user code.
+
+In past proposals for similar traits it has also been noted that AutoDeref is complicated
+and poorly understood by most users. It could therefore be considered problematic that
+AutoDeref behavior is extended. However the behavior here is identical to what Box already
+has, which is considered acceptable in its current state.
+
+## Rationale and alternatives
+[rationale-and-alternatives]: #rationale-and-alternatives
+
+Ideas for something like the `DerefMove` trait design here can be found in past discussions of
+`DerefMove` traits and move references. The desire for some way of doing move dereferences
+goes back to at least https://github.com/rust-lang/rfcs/issues/997.
+
+The rationale behind the current design is that it explicitly sticks very closely to what
+is already implemented for Boxes, which in turn closely mirror what can be done with stack
+variables directly. This provides a relatively straightforward mental model for the user,
+and significantly reduces the risk that the proposed design runs into issues in the
+implementation phase.
+
+### DerefMove trait
+
+Designs based on a simpler `DerefMove` trait have been previously proposed in the unmerged
+[RFC2439](https://github.com/rust-lang/rfcs/pull/2439) and an [internals forum thread](https://internals.rust-lang.org/t/derefmove-without-move-why-dont-we-have-it/19701).
+These come down to a trait of the form
+```
+trait DerefMove : DerefMut {
+    fn deref_move(self) -> Self::Target
+}
+```
+
+The disadvantage of an approach like this is that it is somewhat unclear how to deal with
+partial moves. This has in the past stopped such proposals in their tracks.
+
+Furthermore, such a trait does not by itself cover the entirety of the functionality
+offered by Box, and given its consuming nature it is unclear how to extend it. This also
+leads to the potential for backwards incompatible changes to the current behavior of Box,
+as has previously been identified.
+
+### &move based solutions
+
+A separate class of solutions has been proposed based on the idea of adding &move
+references to the type system, where the reference owns the value, but not the allocation
+behind the value. These were discussed in an [unsubmitted RFC by arielb1](https://github.com/arielb1/rfcs/blob/missing-derefs/text/0000-missing-derefs.md),
+and an [internals forum thread](https://internals.rust-lang.org/t/pre-rfc-move-references/14511)
+
+Drawbacks of this approach have been indicated as the significant extra complexity which
+is added to the type system with the extra type of reference. There further seems to be a
+need for subtypes of move references to ensure values are moved in or out before dropping
+to properly keep the allocation initialized or deinitialized after use as needed.
+
+This additional complexity leads to a lot more moving parts in this approach, which
+although the result has the potential to allow a bit more flexibility makes them less
+attractive on the whole.
+
+### More complicated place traits
+
+Several more complicated `DerefMove` traits have been proposed by tema2 in two threads on the
+internals forum:
+- [DerefMove without `&move` refs](https://internals.rust-lang.org/t/derefmove-without-move-refs/17575)
+- [`DerefMove` as two separate traits](https://internals.rust-lang.org/t/derefmove-as-two-separate-traits/16031)
+
+These traits aimed at providing more feedback with regards to the length of use of the
+pointer returned by the `DerefMove::place` method, and the status of the value in that
+location after use. Such a design would open up more possible use cases, but at the cost
+of significantly more complicated desugarings.
+
+Furthermore, allowing actions based on whether a value is present or not in the place
+would add additional complexity in understanding the control flow of the resulting binary.
+This could make understanding uses of these traits significantly more difficult for end
+users of types implement these traits.
+
+### Nadrieril custom_refs proposal
+
+Similar functionality is also being discussed as part of the [custom reference proposal
+originally created by Nadrieril](https://hackmd.io/N0sjLdl1S6C58UddR7Po5g). This is also
+fits in the category of significantly more complicated traits. However, the very large
+amount of additional affordances it could offer may make it more worth it in this
+specific case.
+
+This RFC still prefers the simpler approach, as pretty much all of the Nadrieril
+proposal would need to be implemented to get `DerefMove`-like behavior. This would bring
+significant implementation effort and risk.
+
+If desired, the functionality of this proposal can at a latter time be reimplemented
+through the trait in that proposal, meaning that this can be seen as an intermediate
+step.
+
+### Limited macro based trait
+
+Going the other way in terms of complexity, a `DerefMove` trait with constraints on how the
+projection to the actual location to be dereferenced was proposed in [another internals forum thread](https://internals.rust-lang.org/t/derefmove-without-move-references-aka-box-magic-for-user-types/19910).
+
+This proposal effectively constrains the `DerefMove::place` method to only doing field
+projections and other dereferences. The advantage of this is that such a trait has 
+less severe safety implications, and by its nature cannot panic making its use more
+predictable.
+
+However, the restrictions require additional custom syntax for specifying the precise
+process, which adds complexity to the language and makes the trait a bit of an outlier
+compared to the `Deref` and `DerefMut` traits.
+
+### Existing library based solutions
+
+The [moveit library](https://docs.rs/moveit/latest/moveit/index.html) provides similar
+functionality in its `DerefMove` trait. However, this requires unsafe code on the part of
+the end user of the trait, which makes them unattractive for developers wanting the
+memory safety guarantees rust provides.
+
+## Prior art
+[prior-art]: #prior-art
+
+The behavior enabled by the trait proposed here is already implemented for the Box type,
+which can be considered prime prior art. Experience with the Box type has shown that its
+special behaviors have applications.
+
+Beyond rust, there aren't really comparable features that map directly onto the proposal
+here. C++ has the option for smart pointers through overloading dereferencing operators,
+and implements move semantics through Move constructors and Move assignment operators.
+However, moves in C++ require the moved-out of place to always remain containing a valid
+value as there is no intrinsic language-level way of dealing with moved-out of places in
+a special way.
+
+In terms of implementability, a small experiment has been done implementing the deref
+elaboration for an earlier version of this trait at
+https://github.com/davidv1992/rust/tree/place-experiment. That implementation is
+sufficiently far along to support running code using the `DerefMove` trait, but does not yet
+properly drop the content, instead leaking it.
+
+## Unresolved questions
+[unresolved-questions]: #unresolved-questions
+
+Right now, the design states that panic in calls to `DerefMove::place` or `DerefMove::place_mut`
+can cause an abort when the call was generated in the MIR. This is done to make compilation
+and the resulting code more performant. It is however possible to implement these with
+proper unwinding, at the cost of generating a more complicated control flow graph before
+borrow checking for code using the dereference behavior of `DerefMove`. This would likely
+result in longer compile times and less optimized results, however that could be judged
+to be a worthwhile tradeoff.
+
+## Future possibilities
+[future-possibilities]: #future-possibilities
+
+Should the trait become stabilized, it may become interesting to implement non-copying
+variants of the various pop functions on containers within the standard library. Such
+functions could allow significant optimizations when used in combination with large
+elements in the container.
+
+It may also be interesting at a future point to reconsider whether the unsized_fn_params
+trait should remain internal, in particular once Unsized coercions become usable with user
+defined types. However, this decision can be delayed to a later date as sufficiently many
+interesting use cases are already available without it.
+
+Finally, there is potential for the trait as presented here to become useful in the in
+place initialization project. It could be a building block for generalizing things like
+partial initialization to smart pointers. This would require future design around an api
+for telling the borrow checker about new empty values implementing `DerefMove`, but that seems
+orthogonal to the design here.