
Kotlin: The Problem with null - bmc7505
http://arturdryomov.online/posts/kotlin-the-problem-with-null/
======
xenadu02
>Unfortunately, it is not really possible to change Kotlin behave the same
way. Apple uses a bridging mechanism to connect Objective-C and Swift
binaries, when Kotlin uses the same bytecode as Java. To make a simpler mental
picture, imagine Objective-C and Swift being connected side-by-side and Kotlin
and Java as a stack, where Kotlin is on top. I presume it would be pretty
challenging to provide a proper compatibility with Java, transforming all
nullable Kotlin values to Optional and vice-versa, especially in such tight
areas like Java reflection.

This is not correct. The Kotlin compiler could treat every parameter and
return value that doesn't have a nullability annotation as an implicitly
unwrapped optional (Type!). It could even support all of the popular
nullability annotations. There is no requirement to choose just one.

The beauty of IUO is that you can ignore the nullability and you have the same
amount of safety you have today: if you touch a null value you get an
exception (or abort in Swift's case). The benefit is you can insert a null
check in one place which then propagates through the rest of your Kotlin (or
Swift) code.

~~~
dcow
Yeah (agree) that's not correct. Kotlin has to compile to Java bytecode, true,
but it does not overlay Java like e.g. Groovy does. Kotlin and Java very much
exist side-by-side in exactly the same way Swift and objective-c do: they
(can) target the same machine but have different language scemantics.

~~~
pvg
The Chris Lattner quote at the end explains the fundamental difference. Kotlin
is an interloper in a Java world - that includes the JVM, a VM designed to run
Java.

Swift integrates and interoperates with but is not based on the Objective C
runtime. Apple also controls all the relevant bits and pieces and they can
bridge, compile-to, shim, wrap, etc however they want. They can can pick the
design approach and fiddle with all the parts to make them fit. This isn't a
luxury Kotlin has.

~~~
peterashford
That would have been true to say if this was a decade ago. After the advent of
.NET and CLI's ability to support multiple languages, Sun started moving the
JVM away from being Java specific and including features to support other
languages. Your criticism hasn't been correct for years.

~~~
extempore
It is not a criticism as much as it is a fact, no less true than ever. The JVM
is designed to run Java. Other languages also compile to Java bytecode, and
Sun and Oracle have made some helpful changes.

But the VM is OVERWHELMINGLY designed around the requirements of Java and to
be performant for Java. Try compiling a non-OO language to fast bytecode
without spending a lot of time considering what the analogous Java code would
compile to.

Java bytecode is so close to Java, you can decompile it almost directly to
readable Java source. Try that with JRuby or Clojure or Scala and see how
close you land.

~~~
cel1ne
Kotlin can compile to Javascript too. Just saying.

------
kodablah
They could have gone hardcore and said any non-primitive, unanotated platform
type is nullable but that would have made interop really ugly. And all of the
null checks would have muddied up code and added (admittedly minimal) runtime
costs.

I want something in between Kotlin and Scala. I want option as a real type
that is treated as a max-single-element collection that can be flattened with
the same APIs as other collections. But I want no runtime penalty. People in
Rust are so lucky to have zero cost abstractions for these things. I suppose
I'll need better compile time support (even more than Scala macros) and whole
program optimizations (i.e. cross JAR) to get zero cost optional on the JVM.
Things like Scala Native use LLVM and surely Option things are inlined or
otherwise optimized out.

~~~
sjrd
I wrote a zero-allocation `Option`-style (monadic) data structure for Scala a
while ago [1]. Unlike all the previous attempts, it supports the distinction
between `None`, `Some(None)`, `Some(null)`, `Some(Some(None))`, etc., which is
what allows it to remain monadic. The surprise is that it does _not_ use
`null` as the `None` value. The downside is that `toString()` is altered:
`Some("hello").toString()` returns `hello`, not `Some(hello)`.

There was an experiment to use it as a replacement for the implementation of
`scala.Option` in the dotty compiler code base [2], but it is inconclusive so
far; it should be tried directly in the collections library.

[1] [https://github.com/sjrd/scala-unboxed-
option](https://github.com/sjrd/scala-unboxed-option)

[2]
[https://github.com/lampepfl/dotty/pull/3181](https://github.com/lampepfl/dotty/pull/3181)

~~~
benjaminjackman
Just curious, if Some(hello).toString produces `hello`, what does
None.toString() and Some(None).toString() and Some(Some(None)).toString()
produce? (My guess is `None` for all)

I think it's impressive† that you got nesting to work, I'm really curious how
you pulled that off for longs and doubles without incurring extra overhead.

† Though, given the magic you worked in getting Union types working in the
ScalaJS facades I shouldn't be surprised.

~~~
sjrd
Oh primitive types do get _one_ level of boxing (on the JVM): a `Double`
becomes a `java.lang.Double`. But it doesn't become a `Some` with a
`java.lang.Double` inside, so we gained one allocation anyway. It is not
possible to remove that box without compiler support, and even then not in all
cases (return values for examples) because `Double` contains a finite amount
of values 2^64, and `Option[None]` has 2^64 + 1 values.

And in that implementation, `None.toString == "None"`, `Some(None).toString ==
"Some(None)", etc. Although that could be changed.

~~~
e12e
Seems like one could use the wasted digit of signed numbers to stor options
rather than have asymmetric range (two's complement) or positive/negative
zero... IE have a [ed:bit string] that indicates none/some?

------
_Codemonkeyism
Shameless little self promotion

"Comparing Optionals and Null in Swift, Scala, Ceylon and Kotlin"

[http://codemonkeyism.com/comparing-optionals-and-null-in-
swi...](http://codemonkeyism.com/comparing-optionals-and-null-in-swift-scala-
ceylon-and-kotlin/)

Personally I like Monads better than the Kotlin solution. Best is probably the
way Swift does it.

~~~
greenhouse_gas
My issue with Monads (and why I like kotlin and swift nullable types) is that
it allows the compiler to infer nullability like the following code:

fun f(a1: Int?, a2: Int?): Int {

    
    
      if (a1 == null || a2 == null) {
    
         return 0
    
       } 
    
       //now you can do something like
    
       return a1+a2
    

}

In contrast, if it didn't, the code would look like:

fn func1(q: Option<i32>, z: Option<i32>) -> i32 {

    
    
        if let Some(q_in) = q {
    
            if let Some(z_in) = z {
    
                return q_in + z_in
    
            }
    
        }
    
        return 0
    

}

Note the deeply nested parenthesis.

~~~
yiransheng
In languages like Haskell and Scala that supports `do` / `for` comprehension
sugar, this could be done without the nesting:

    
    
      fn :: Maybe Int -> Maybe Int -> Int
      fn q z = fromMaybe 0 $
        do 
          q_in <- q
          z_in <- z
          return (q_in + z_in)

~~~
zenhack
Or just:

    
    
        fn (Just q) (Just z) = q + z
        fn _ _ = 0

~~~
masklinn
Surely since you have monads you can just use the tools they give you?

    
    
        fn = liftM2 (+)

~~~
zenhack
That doesn't do quite the same thing; the original example gets rid of the
Maybe, defaulting to 0.

~~~
masklinn
> That doesn't do quite the same thing

You're missing the point entirely.

> the original example gets rid of the Maybe, defaulting to 0.

[http://hackage.haskell.org/package/base-4.10.1.0/docs/Data-M...](http://hackage.haskell.org/package/base-4.10.1.0/docs/Data-
Maybe.html#v:fromMaybe)

------
jontro
At least for android kotlin programming, with the release of api level 27 +
support libraries, it’s a lot better. They have placed the annotations all
over the place. Using nullable annotations have good use cases in java too.

I would think optional types and everything else being non nullable is the way
forward for java too

------
phreack
Well, the null issues in Java libraries have always been there, I don't really
think it's a fault of Kotlin and its tooling that those bugs are not caught at
all by the compiler, as all object types in Java are Kotlin optionals if not
annotated.

Maybe in time most of the necessary open source projects will be pure Kotlin
ones, but for now I believe annotating Java code with null constraints is
certainly a boon for everyone and should be encouraged.

~~~
mkobit
I agree that the tooling is getting better. Some of the popular libraries are
still figuring out the right patterns to apply to make them more "Kotlin
friendly". Tools like the Checker Framework [1], JSR 308 [2], and compiler
plugins like Traute [3] can also help with the safety issues on the Java side.
Kotlin is beginning to understand these more, and some of the basic
annotations are supported, but the tooling is only going to improve with all
of the backing that Kotlin currently has.

[1]: [https://checkerframework.org/](https://checkerframework.org/)

[2]:
[https://jcp.org/en/jsr/detail?id=308](https://jcp.org/en/jsr/detail?id=308)

[3]: [https://github.com/denis-zhdanov/traute](https://github.com/denis-
zhdanov/traute)

------
smitherfield
_> Turns out Swift does not have null pointers. At all!_

 _Mostly_ true under the hood; optional value types are tagged unions (I
didn't know this before reading this article and checking for myself), but
optional reference types are still, as you'd expect, nullable pointers.[1]

[1] [https://godbolt.org/g/QZk6wL](https://godbolt.org/g/QZk6wL)

------
taeric
The java compiler might not know if a variable is safe to pass as null, but it
is not unknowable. I think on every thread similar to this that I am on, I
have plugged Coverity as a tool that can and will flag spots like this. That
is, it won't just say "it is possible this will be a null pointer exception,"
instead it will say "setting this value to null will lead to this dereference
of a null." It can look magical sometimes, because they will trace pretty
deeply into your code.

Which is just my way of saying over and over again that tooling can improve
that does not require a complete rewrite of your software.

~~~
drfuchs
It sounds like you’re claiming that Coverity has a solution to the Halting
Problem? That would indeed be magical.

~~~
taeric
Uh, no? Null pointers are far from the halting problem, though.

And to be clear, there are still limitations. It can't say if any value you
pass into a function that is not part of the current compilation is safe. For
somewhat obvious reasons.

But for a large large class of bugs, using advanced tooling can go a long way.
As evidenced by the power of the advanced tooling new languages bring into the
compilers. :)

~~~
drfuchs
The claim seems to be that the tool will always tell you absolutely whether a
null-dereference will happen, and never say “maybe.” The Halting Problem is
trivially reducible to this (just put an intentional null-deref in front of
each halt).

~~~
taeric
Not necessarily the same point being made. At least, they are subtly
different, to my understanding.

It can tell you that, "if this line is reached, and it was called from this
path (with evidence on how this could happen), it will dereference a null." It
is not saying that "running this program will guarantee give you a null
dereference."

The halting problem is more in line with "this program will terminate on any
possible inputs", which is more expansive. It is trivial to say that you can
prove some programs won't terminate on a particular input. Question is if you
can do it for all inputs, no?

~~~
zenhack
Rice's theorem is a bit more direct here: C is Turing complete, some C
programs derefernce null pointers, and some don't. Therefore, the question of
whether a an arbitrary C program derefernces a null pointer is in general
undecidable.

But I don't think the Op was claiming it was possible to do this perfectly,
just possible to write very useful tools. You will always have either flase
positives or false negatives (or I suppose inputs where you just hang).

Turing completeness is the bane of static analysis, but that doesn't make it a
fruitless endeavor.

~~~
UncleMeat
I wouldn't say that Rice's thm is the bane of static analysis. It is what
makes the field _interesting_. If the problems were not undecidable then we
wouldn't have do so many interesting and challenging things to make working
tools.

------
yxhuvud
After coding enough Crystal, which instead of optionals have raw and anonymous
sum types, I can say this: I don't see the point of a type system where nil is
a special case. I don't want optionals, I want to have nil as a totally
separate type.

~~~
anaphylactic
Optionals are sum types, too! No special-casing there. It's just that Crystal
just has anonymous sum types, I guess, which can be a lot nicer to look at.

~~~
brabel
Optional is a special case of a sum type where one of the types is Null. If
you have Null as a type whose only value is null, then an Optional is just a
T|Null. No need for Optional to exist.

------
stewbrew
I don't quite get the problem as stated in the article. I assume it's a
contrived example and the real (kotlin) code rather looks like:

    
    
        val foo: T? = null
        Observable.just(foo).subscribe()
    

Here foo is defined as nullable, hence the problem. If you make it

    
    
        val foo: T = null
        Observable.just(foo).subscribe()
    

the kotlin compiler will bark on foo being null and you'll never have a
runtime error.

I think the motivating example should be described better but maybe I
misunderstand it.

~~~
masklinn
> I don't quite get the problem as stated in the article.

> Here foo is defined as nullable, hence the problem.

How do you know it's "the problem"? The answer is that you don't until it
blows up in your face, because this is a Java interop call.

And of course the local could come from an other call into Java which returns
a nullable reference, or it could make complete sense for it to be nullable.

~~~
smichel17
Remove part of your sentence, and I think you've answered your own question.

> How do you know it's "the problem"? [...] because this is a Java interop
> call.

If you're making platform calls, you know you have to deal with nulls (or at
least, you should, in my opinion). So you declare returns nullable and avoid
passing nulls unless that's documented to be okay.

Kotlin's Elvis operator makes this really easy:

val a: String? = null val b: String = a ?: "alternative"

------
paulddraper
I wrote a post a few years ago comparing nullable types to maybe/optional.
[https://www.lucidchart.com/techblog/2015/08/31/the-worst-
mis...](https://www.lucidchart.com/techblog/2015/08/31/the-worst-mistake-of-
computer-science/)

A lot of it is a more specific cas of sum types vs union types.

------
flareback
It seems like a good article but the grammar makes it difficult to read at
times.

------
user2994cb
Aren't null pointers just a variation on the Maybe Monad?

~~~
anaphylactic
Maybe is just another Optional type, but with a much weirder name, so that's
pretty obvious. But this makes me wonder - does Haskell optimize `Maybe`-like
types down to a null-pointer, like Rust does?

~~~
user2994cb
Dunno, but I'd have thought that you can make None look like a null pointer,
but Some(p) needs to be tagged in some way.

~~~
user2994cb
Actually, having Some(x) be just a pointer to x seems to work fine without an
explicit tag, as dragonwriter says below.

------
thought_alarm
.online is a terrible TLD.

