What I never understood was why backwards compatability was not overcome by simply changing the file suffix. A second fixed up version (something like Ceylon) could be mixed in with old Java code. Just change the suffix to .java2 to signify a class using the new language spec.
In my opinion basic type inference, name parameters, better support for immutability* and multiline strings represent the majority of the deficit in terms of making java more expressive and convenient. Other things such as higher order functions (would require an extension of the type system) raise issues that make the benefits less clear cut.
* my view on immutability is that it should be possible to 'bake' a class or an interface. So it can be manipulated before being made immutable.
(Apparently something like this is possible in Java 6 using annotations and compiler plugin extensions, but apparently it is not straight forwards to get working with IDEs (and as such, given the importance of IDEs one may as well change language).
Backwards compatibility was never a technical problem, the JVM can support heterogeneous programs written in multiple languages out of the box. As long as it all compiles down to class files everything can interoperate seamlessly.
Strong-typed alternatives to Java have existed for ages, such as Nice, KSL, and Scala, so if you want you can take your pick in mix it in with Java code right now.
As far as I understand it Sun just didn't want to fork the Java language.
I understand this, but my point is this. They add features to java e.g. generics and all the tools (IDEs) have to make adjustments to catch up. It seems to me that similarly sized adjustments could be made to support a newer cleaner version of Java (i.e. strictly better, just the low hanging improvements - really just syntactic sugar). The only additional change would be the introduction of a new file suffix, or even a version before the package declaration. The impact on the tool chain/developer adoption would be the same. This is not true for Scala and so on since here the departure is so great that the IDE support needs to be rewritten.