Oh to be clear I don't have an answer for this off-hand, I'm not that smart. However, if we'd given the 30 years of people developing faster for-loops the task of figuring out how to manage memory statically, I'd wager we'd not be having this conversation right now. It is by no means an easy problem, however.
I think there are essentially two ways of making the compiler statically aware of lifetimes: you can annotate the program with lifetimes, as in Rust, or you can have the compiler infer all of the lifetime information somehow.
Adding annotations requires more programmer effort and reduces the level of abstraction of your programming language. Rust takes this approach, but tries to keep the level of annotation somewhat reasonable, and provides the "unsafe" escape hatch for when you cannot convince the compiler that what you're doing makes sense. (Although I do believe that it would be possible for the Rust compiler to infer most/all lifetimes).
For instance, in a functional programming language, you are typically creating closures all over the place, like one of the other posters mentioned. It seems quite unlikely that there is any kind of reasonable lifetime annotation system that would capture all of the complex lifetime patterns that you would see in such a language.
One can say "Oh, but surely someone smarter than me must be able to come up with something that works for those situations", but I don't think that is a very constructive line of thought, since you could say that about pretty much everything. Rust is an example of a language where lifetimes work reasonably well, but Rust already makes some tradeoffs that reduce its level of abstraction to some degree. It doesn't provide the same level of abstraction that, say, Haskell does.
Program analysis would allow you to keep your level of abstraction and ideally not require only little annotations. However, since any non-trivial program property is undecidable, program analysis is fundamentally incomplete and it is again unlikely that it would be able to deal with all lifetime patterns that you see in practice.
Therefore, it seems to me that there is no free lunch here. Either you sacrifice abstraction by requiring annotations or you don't require any annotations but then you have an incomplete program analysis.
(Also, you seem to think 30 years of good research has been wasted on the "obviously bad idea" of garbage collection, but somehow none of these researchers were good enough to realize this and come up with the silver bullet that makes GC unnecessary? After all, people were already thinking about GC-less languages long before Rust was a thing)
possible for the Rust compiler
to infer most/all lifetimes)
I'd be interested to learn why you think this is possible. As you point out yourself, by Rice's theorem, precise lifetimes are not statically decidable. Rust's lifetimes are essentially based on nesting (think well-balanced brackets) and tracked by enriching the standard Damas-Hindley-Milner approach with a notion of affin-ness which is a weakening of Girard's linearity. Circular data structures immediately violate affine-ness. What mechanism do you propose to get around this conundrum?
"All" might not be possible, but every Rust release tends to see improvements in lifetime elision. I just went through a few Rust apps I wrote 6 months ago and was able to get rid of nearly every explicit lifetime annotation thanks to improvements.
So there's definitely room for improvement, but also good progress being made.
I think that for function definitions in Rust you would in principle be able to leave out lifetime annotations on arguments/return values in more cases than is currently allowed by lifetime elision. I believe they don't allow this since it can become very confusing if the lifetimes are not written out when there are multiple references arguments. I'm not entirely sure about this, though.
I know this is possible because Rust already does this. Rust's types in closures can be fully inferred. Try replacing all Rust functions with closures. It goes surprisingly far.
It's a little bit contradictory that you simultaneously believe that the people who worked on GC are much smarter than you, but at the same time they were wrong to choose to work on GC rather than your research program ;-) Not all problems can be solved by throwing smart people at it. There has been research along the lines you suggest. Look at the MLkit region inference research, or escape analysis research, for example. It doesn't work very well in practice. It can correctly infer like 95% of memory allocation/deallocation, but that remaining 5% leaks memory which means you need a GC anyway. It's still valuable research though. Go, for example, made the bet that they can use a low-pause low-throughput GC by reducing the pressure on the GC by doing escape analysis. Maybe that would work even better with the far more powerful region inference of the MLkit.
