Just allocating into such a memory region (essentially custom page swapping) will eventually be possible with data structures that can be constructed with custom allocators, but it still wouldn't be ok to reload those.
I found an old discussion on this topic here: https://news.ycombinator.com/item?id=13890011
Note that the Microsoft C++ compiler implements based pointers, using the __based keyword.
So to do this safely all the types must be whitelisted through a custom trait. Copy is close, but not quite, since references are still Copy.