
Non-Lexical Lifetimes in Rust - kibwen
http://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/
======
kibwen
Short background: in today's Rust, the lifetime of a reference is bound to a
lexical scope. Sometimes this forces one to create temporary variables solely
to change the lexical scope of a reference, which is an annoyance. Take the
following example:

    
    
        let mut nums = Vec::new();
        nums.push(nums.len());
    

The above code would compile if we had non-lexical lifetimes, but today it
doesn't. This is because method calls are effectively desugared to something
like this:

    
    
        let mut nums = Vec::new();
        Vec::push(&mut nums, nums.len());
    

The arguments are evaluated left-to-right, and the mutable borrow prevents the
immutable borrow later. Once non-lexical lifetimes arrives this will be
possible thanks to control-flow analysis, but today we have to make a
temporary:

    
    
        let mut nums = Vec::new();
        let length = nums.len();
        nums.push(length);
    

It doesn't involve any new user-facing features to the language and is fully
backward-compatible, so it's basically just a straight quality-of-life
improvement for Rust programmers.

~~~
eridius
I'm not quite sure how non-lexical lifetimes is supposed to fix this issue.
Control-flow analysis doesn't change the fact that the arguments are evaluated
in left-to-right order, and it doesn't change the fact that the &mut reference
is created prior to the method arguments being evaluated and lives until the
method itself is invoked. Control flow analysis will simply confirm the fact
that the &mut borrow exists while the arguments are evaluated.

What might fix this is a special-case rule where an implicit¹ & or &mut
reference as a method receiver is temporarily ignored while the arguments are
being evaluated (with the borrow checker double-checking after the arguments
are evaluated that the & or &mut reference is still legal, i.e. that the
referenced value wasn't moved and, in the case of a &mut reference, that no
other borrow exists as a result of evaluating the arguments). This rule, being
a special case, doesn't require non-lexical lifetimes.

¹I say implicit because it would be potentially confusing for an explicit
reference to be ignored while the arguments are evaluated.

~~~
cwzwarich
The assumed fix here is to distinguish between a mutable borrow and the
reservation of a location to later do a mutable borrow. Locations based on
local variables and their fields can be reserved, although a move out of the
variable would invalidate a reservation. Other locations (e.g. the result of a
function call) can't be reserved and would give an error.

~~~
eridius
Seems like it would be confusing to allow

    
    
      let mut nums = Vec::new();
      nums.push(nums.len());
    

but disallow

    
    
      let mut nums = Box::new(Vec::new());
      nums.push(nums.len());
    

The second example would have to be disallowed because the mutable reference
is created by calling DerefMut::deref_mut().

~~~
cwzwarich
The particular case of Box is safe, and is along the lines of special
knowledge about Box that is assumed by the type checker, but the special-cased
nature of Box has been gradually decreasing over time. I don't think there
would be an easy safe way to allow user-defined types to establish same
guarantee.

In retrospect, changing the evaluation order of a function with respect to its
arguments might have been a better choice, but something like the solution I
mentioned is probably the only backwards-compatible option.

------
mtanski
Can't happen fast enough. For the last two years in a row I try Rust every 6
months to see if it's at a place that I would consider using it. I hit the 3
problems described here every time I try it. You can work around all of them
by restructuring your code it's just so tedious. I rather be doing something
else versus working around this.

The community hasn't really been helpful, the answer is always restructure how
write things. I'm writing a column store query engine (with table blocks
accessing them by columns) that leads to a lot of borrowing so the advice is
not all that helpful. I want to like Rust, but I can't.

~~~
TillE
I've also found that basic problems like how to implement typical data
structures (without unsafe code) are controversial, difficult, or actually
impossible. That's really not good.

Still optimistic about Rust. But it has some fundamental issues to solve
before it's generally usable.

~~~
rectang
I'm just learning Rust and this is the primary question I have: If I can't use
the data structures I'm accustomed to (because of mutability issues), _what
are the alternatives?_

Where are the articles describing those alternatives? I've seen a lot of
interesting discourse about the low-level stuff (e.g. error handling), but not
higher-level "start here" approaches.

~~~
bsder
> If I can't use the data structures I'm accustomed to (because of mutability
> issues), what are the alternatives?

You can use the data structures you are used to just fine, thanks. Just
because everything is immutable by default doesn't mean everything is
immutable all the time.

In addition, any "unsafety" is generally buried deep inside a library and
contained properly. Yes, if you are trying to implement something like a
ConcurrentSkipListMap, you're probably going to have "unsafe" in your code
(and probably some assembly language). But, if you are using the library,
probably not.

------
gsg
This reminds me of an irritating problem in CPS based compilers where you can
have a transformation that is obvious and legal except that after the
transformation one of the relevant variables would not be in scope. The
solution(s) are to decline to do that transformation, fart around with sinking
and hoisting in order to get the scope right, or ditch scope and move to a
graph model in which dependence is represented with explicit edges rather than
being approximated by the lexical structure of the program.

I'm guessing that the lexical borrow problems that Rust are facing are an
instance of the same problem. It will be interesting to read the follow up
post and see what relation, if any, the suggested solution has to the graph
approaches that I am familiar with.

~~~
cwzwarich
The solution is basically the same as the CPS case: switch to dominator-based
scoping rather lexical scoping. I wrote an RFC on what would need to be done
to the region system for nonlexical borrows a while back:
[https://github.com/rust-lang/rfcs/pull/396](https://github.com/rust-
lang/rfcs/pull/396).

------
glass-animal
I want to like Rust, and in the abstract I do, but every time I dive in the
concepts have too much complexity for me to get going, and this post
reinforces that.

It's not that I couldn't eventually grok Rust, it's that I don't have time to
given that it's not part of my day job. I really hope that Rust leads to some
of the same features being expressed in a more intuitive way by a subsequent
language.

~~~
jeffdavis
I wonder how much of this is required to use rust effectively. A lot of
languages have fairly tricky semantics and type systems, and most developers
don't fully understand them. C++ is obviously a big language, but even C has
things like volatile pointers and pointer aliasing and numerical type
promotion.

And if you don't understand non-lexical lifetimes, I think that's OK. You can
just be more explicit about the lifetimes and it will still work. And reading
someone else's code should still make sense.

------
jeffdavis
"The lifetime of a value"

Surely he means _variable_ , not _value_ , right?

