Representation of Rust references (`&T`, `&mut T`) and raw pointers (`*const T, `*mut T`)

[nikomatsakis]

Representation of Rust references:

• Are &T and &mut T guaranteed to be a pointer?
• Must always be aligned, non-null
• Guaranteed to be ABI compatible with C pointer types ("in every way?")
  • presuming that the referent is compatible, of course

Representation of raw pointers:

• Guaranteed to be ABI compatible with C pointer types
  • presuming that the referent is compatible, of course

Other factors:

• Considerations for storing things in the low bits of pointers
  • safe with raw pointers, not safe with references (from this comment)

[RalfJung]

This is related to Stacked Borrows: References might, at least abstractly, have extra metadata attached to them, making transmutes to/from raw pointers potentially non-trivial to handle.

[joshtriplett]

• Considerations for storing things in the low bits of pointers (likely answer: safe with raw pointers, not safe with references).

[RalfJung]

likely answer: safe with raw pointers, not safe with references

Given that we are passing aligned attributes to LLVM, I think this is the only possible answer (unless we want to change which attributes we emit).

[hanna-kruppe]

Even if we got rid of aligned on function attributes and return values, tagged references also can't be marked as dereferencable which would probably be a bigger loss.

[alercah]

I agree that the low bits of a pointer, but not a reference, can be used for storage.

That said, I would like it to be possible to write a type which does store values in the low bits that has an interface to safe Rust basically equivalent in functionality to a (&[mut] T, E) where E is some fieldless enum with few enough values to fit in the low bits. I think that if we can store values in the low bits of raw pointers, it works out, but we should double-check.

[joshtriplett]

I do agree that we could teach the compiler to internally treat the pointer values of references as having holes.

[nikomatsakis]

I do agree that we could teach the compiler to internally treat the pointer values of references as having holes.

This is trickier than it sounds. The problem is that you can create a &mut &T value — in that case, we would not be able to simply store the value, we'd have to work hard to preserve any low bits (as well as masking them off on a load, potentially). That would potentially affect all loads and stores since you never know if the &mut points into an enum or not.

In general, to do more advanced optimizations of this kind, it would be useful to be able to mark an enum as not permitting "ref bindings", which sidesteps the problem.

[gnzlbg]

Does the "&T is just a pointer" also applies to unsized Ts ?

I ran into the following example in the wild today:

pub struct Wrapper([i32]);

impl Wrapper {
    pub fn new(x: &[i32]) -> &Self { unsafe { mem::transmute(x) } }
}

Somebody suggested that making Wrapper #[repr(transparent)] could help, but on discord nobody appears to know whether this should even work.

---

EDIT: I get the feeling that this is only for references to sized types. Is there a different issue for the layout of trait objects, DSTs, etc. or is it fine to discuss that here?