To summarize, we were looking for a language that could cut down our code base (IntelliJ platform and server side tools are written in Java), be concise but still expressive, toolable, as fast as Java, easy ramp-up time, and very important, interoperable with all the existing code base we had.
Given the candidates at the time, it was decided to start Kotlin.
Based on my admittedly limited experience in the JVM world, your needs sounds like Scala's sales pitch. What factored in to your decision to develop Kotlin, rather than going with Scala?
The two languages have completely different philosophies. Kotlin is a strict adherent of the Java philosophy[1], a "blue collar language" that adopts only tried-and-true features, with PL research being a non-goal. Scala OTOH, adopts almost every new PL concept, and PL research is very much one of its goals. A quick look at Scala's list implementation will show you how different those languages are. Kotlin is a modern Java; Scala is a Haskell/OOP/macros combo.
Yet, I think many of Scala's users will find that Kotlin gives them all they need, in a much simpler, more readable and maintainable language. If, however, you're aching to write type-safe monad transformers, you're bound to be disappointed by Kotlin.
As the Scala tech lead, I'm glad to learn we've finally adopted "almost every new PL concept!" I guess by now everyone will recognize your username and adjust for bias accordingly, but still, how about sticking to the facts?
As someone who doesn't follow such things (eg; I don't recognize the "pron" or "adriaanm" usernames, and haven't used Scala or any other JVM language), I found your comment to be a turn-off towards using Scala. That's not the kind of mature tone I'd look for from the tech lead of an open-source language.
That said, it is good to know that "pron" was incorrect/exaggerating when they said Scala adopts a lot of PL concepts; thanks for setting the record straight (though I'm still just not sure where Scala stands on that after reading this).
FYI, pron's the author of a would be competing project to Scala's Akka (i.e. competing for share of distributed computing market on the JVM), and takes every opportunity to danger-danger-dragons-be-there on Scala related threads/comments.
Saying that, there is always some truth in what pron brings to the table, it's sort of a higher kinded trolling ;-)
As for Scala and PL concepts, sure, plenty on offer if that's of interest, but not at all a requirement in using the language. A great example of a highly successful project based on the non-FP-centric aspects of Scala is Spark. See their high performance computing style guide [1], no monad transformers there...
FYI, I had taken "every opportunity to danger-danger-dragons-be-there" Scala — on record — years before I even dreamed of the database company that would one day have an open-source project which would end up competing with Akka (I'm talking 2009/10, while Quasar was first conceived in mid 2013).
It was after leading an attempt to adopt Scala at a large organization, which failed because of what I saw as Scala's bait-and-switch marketing: it presented itself as "a better Java" while it really was (and is) a strict-Haskell-with-OOP-and-macros-and-more. We needed a car and got a helicopter. While some would consider a helicopter superior to a car, I think everyone would agree it imposes a completely different set of tradeoffs. So yeah, for a good while I was telling people, listen, it's a helicopter, so make sure a helicopter is what you want (the Scala community’s response was, what is this FUD? Can’t you see it's got wheels?! You can drive it on the ground if you want; you don’t have to take off! To which I responded, but you still can’t take it to a car garage for maintenance, to which their response was, you’re a stupid biased troll!). I don't remember when was the last time I participated in a Scala thread, but that community doesn’t forget. So if I'm damned if I don't, I might as well do:
Spark is a good example for a project that would have been better off using Kotlin. They wouldn't have missed any Scala features, would have had better build times, better IDE support and better Java interop (of course, Kotlin wasn’t mature enough to even be considered when that project started). In fact, using Kotlin would have saved them writing 95% of that style-guide[1], which reads like instructions on how to drive a helicopter on the ground without accidentally taking off (Perhaps ironically, that style guide drew derision from the Scala community, who said, you're in a helicopter for fuck's sake! Why won't you take off?!)
[1]: No implicits, no multiple argument lists, no Try, no infix notation, no non-math operators, no recursion, no being-fancy-with-monads, no Scala concurrent collections, no Scala collections, no for loops. They don't mention dynamics and structural types, but I think it goes without saying that they're off-limits, too.
Interesting analogy, incredible how Spark drives the Scala helicopter like a car; they seem to have nailed parallel parking ;-)
Seriously though, I doubt Kotlin would ever be the first choice for a project like Spark since Scala excels in the DSL department (really without rival on the JVM, and off, only Haskell comes close). Look through Spark source code [1] and you'll see: operator overloading, (sealed) ADTs, exhaustive pattern matching, etc. functionality that you can't even think the thought in in a language like Kotlin. This has nothing to do with monad transformers and other FPisms, just straight DSL power.
Finally, to put things in perspective in terms of the mindhsare that Scala and its ecosystem are drawing, IBM is investing over 100 million dollars per year in Spark research and development [2]. Sorry, but that's absolutely massive, maybe there are no dragons? Or if there are, the big players don't seem overly bothered.
Do you actually know Kotlin? It has many features for DSL support, including operator overloading (though not the ability to invent random new operators fortunately), infix notation, a form of exhaustive pattern matching, extension functions, annotation processing, inlineable anonymous extension functions which sounds bizarre but actually is key to defining many DSLs in the language, and so on. It's a different approach to Scala but to say there's no way to do DSLs isn't right.
As an example, check out funKtionale. It's a library that adds various functional features like function currying and partial application to Kotlin, entirely by using its DSL features.
1. None of those features (except sealed ADTs, which really aren't necessary) is missing from Kotlin. The set of Scala features Spark is using is pretty much identical to Kotlin.
2. I don't see how IBM's investment has anything to do with Scala, though. Spark is a great product, and deserves an investment even if it were written in BASIC.
3. Even if Spark did rely on Scala features that aren't in Kotlin (it doesn't), Spark is a library/framework/middleware completely unlike enterprise projects. It is orders of magnitude smaller, for one. That is not to say it isn't awesome, but I wouldn't extrapolate from its choice of technologies (including the language) to the enterprise.
4. Clojure makes writing DSLs easier and with far less magic. Groovy DSLs are also as powerful as Scala's and easier to write. Kotlin DSLs are not as powerful, but are more than enough for Spark.
5. I've never, ever, said Scala is unusable. In fact, I've always said (and believed), it is a very powerful language. I have said that that kind of power is not what most companies want or need, and that that kind of unopinionated kitchen-sink, cutting-edge language is probably not suitable for long-time maintenance in large companies. I completely stand by this. My problem with Scala, again, is not that it's a bad helicopter, but that it insists that it's a good choice if what you want is a car (unless you actually try to drive it like a car -- as Spark does -- in which case you invite the ire of the entire community), and, IMO, it isn't.
6. It's OK to not like your language. It's OK to say it. It's OK to explain why. And it's OK for you to disagree. That doesn't make me biased, and certainly not some kind of bogeyman. It also doesn't justify ad hominem attacks, which -- unlike my criticism of Scala -- are completely unfounded.
> None of those features (except sealed ADTs, which really aren't necessary) is missing from Kotlin. The set of Scala features Spark is using is pretty much identical to Kotlin.
I didn't mention implicit conversions, but that's one area where Scala has taken the high risk high reward path. If you look a bit further down on the linked Spark page you'll see the conversions defined [1]. I'd say Spark's style guide is more geared toward Scala consumers and not necessarily library authors (i.e. if you know what you're doing have at it; otherwise keep to a strict subset).
Basically, in aggregate Scala's feature set is incredibly powerful; that power (despite the complexity, tooling challenges, and slow builds) is what draws dsl authors to Scala.
When IBM throws down an absolute boatload of cash on a project written in Scala, by proxy Scala benefits. Right off the bat there are de facto 3,500 new Scala developers (since the initial IBM Spark team will of course be learning Scala). Furthermore, it's not like competitors aren't aware of this, Spark automically becomes more of a "big deal" as a result, which cannot help but be a boon for Scala.
As for other JVM languages, agreed, Clojure has its own supply of awesome, though the lack of (built-in) static types are a no-op here. Groovy, meh, seems to be on a downtrend, I suspect non-immutable-by-default dynamic languages (except of course, javascript) will continue to fall out of favor.
Now, Kotlin, it's a nice language. If they can deliver on near Java build times, might just be the Java killer that Scala will never be, or at least not in its current design (where, as you rightly say, there is often more than one way to do the same thing).
Let's see how things play out over the next 3-5 years, JVM alternative language landscape is diverse and evolving quickly, good times now and great times ahead...
> I didn't mention implicit conversions, but that's one area where Scala has taken the high risk high reward path.
We C++ veterans know that implicit conversions are one of the worst ideas to put in a language. They start simple enough and then they get clever, which is where trouble starts.
> i.e. if you know what you're doing have at it; otherwise keep to a strict subset
There are few things about programming language design we know, because getting empirical data is so hard. One of the few things we do know (from C++ in the nineties, of course) is that this kind of design doesn't work. The average codebase lifespan is about a decade, during which time people move around and team leads change. Everyone has their own discipline, and by year 5 (usually earlier), the codebase is unmaintainable.
Of course, now that C++ has been re-branded and re-purposed as a specialized language for close-to-hardware software, and is used by specialists, it is doing quite well. I certainly wouldn't mind Scala marketing itself -- like C++ and Haskell do -- as a language for specialists (although even specialists would be better off opting for the excellent OCaml implementation on the JVM, or even Frege if Haskell is your cup of tea).
There is absolutely no reason to be as multi-paradigm as Scala is on the most polyglot platform around, which brings me to:
> that power... is what draws dsl authors to Scala.
One of the great things about the JVM is its relatively easy inter-language interop. I would use a meta-language made for DSLs (like Clojure, MPS or Xtext) for my DSL, while I'll write the infrastructure in Java/Kotlin. Learning Java and Clojure, or Kotlin and MPS, is easier than learning Scala alone (well enough for DSLs), and the results are cleaner, produce more maintainable code and better error messages (well, I wouldn't say Clojure excels at error messages, but at least the DSL writer has full control over them, and can make them quite good with an extra bit of effort).
Also, I'd like to see what those DSL codebases look like after five years of heavy usage (not to mention the error messages Scala spits out, which are completely unparsable to anyone who does not understand how the DSL is constructed).
> We C++ veterans know that implicit conversions are one of the worst ideas to put in a language
Too late to yank them now, can be disastrous in the wrong (read: beginner) hands, but they're fantistically useful in the dsl department so...
> The average codebase lifespan is about a decade
maintainability is an issue for the long haul, sure, Scala won't win that battle until it stabilizes, which is at minimum 3 years away when Dotty (Scala 3) comes on the scene.
> I certainly wouldn't mind Scala marketing itself -- like C++ and Haskell do -- as a language for specialists
Typesafe does the marketing, and as a business they'll never sell their technology short, even if what you say is closer to the truth wrt to specialization.
> better off opting for the excellent OCaml implementation on the JVM, or even Frege if Haskell is your cup of tea
no users = no dice; doubt either will grab much mindshare in the near-term. Personally I would love to see SML cherry pick the non-crufty good parts of OCaml (including coming implicit parameters) and have that find a following outside of current tiny academic circle it languishes in.
> Also, I'd like to see what those DSL codebases look like after five years of heavy usage
They probably look a lot different than how they started out as...paraphrasing someone, "you're going to rewrite it [your application] at least 3X" ;-)
Basically the entire Scala ecosystem is a moving target, including the language itself. Enterprise outfits looking for a static dependency graph that will work today and 10 years from now should look at Java as the host language. I doubt Kotlin will be able to deliver the same guarantees, but if it can, watch out Oracle.
> Typesafe does the marketing, and as a business they'll never sell their technology short, even if what you say is closer to the truth wrt to specialization.
I don't think that's selling it short. Accurately describing your product only prevents people who aren't really your target -- like me -- from being disappointed and telling the world about it, while it helps attract precisely those people who would be most excited about it.
Also, I think Typesafe realize now that there is not much money to be made in developing and supporting a programming language (ironically, there is some to be made -- though not too much -- supporting specialist languages; see Ada and Matlab). That's why they want to change their name, right?
> no users = no dice; doubt either will grab much mindshare in the near-term.
That's splitting hairs a bit; Scala doesn't exactly have mass appeal either. We're talking about the difference between tiny and small.
> I doubt Kotlin will be able to deliver the same guarantees, but if it can, watch out Oracle.
I know they're trying to. Also, I know Oracle actively encourages alternative JVM languages. Their product is the Java platform, which currently -- to be honest -- is not too far from being the same as Java the language (all alternative JVM languages combined don't amount to 10%, or possibly even 5% of the JVM ecosystem). But they wouldn't mind if that changes. Their research team working on Graal, HotSpot's next-gen JIT, coming to OpenJDK in Java 9, spends as much time on making sure it runs Ruby, R, C, Python, and JavaScript well as it does on making sure it runs Java well. I know Oracle's Java team isn't all too happy about it, but I also know that that's what Oracle wants.
To add to my previous comment (now below), Scala DSLs are without rival only in the complexity of their composition. Clojure DSLs are based on macros. Groovy's are on builders and method-missing. Haskell's are on symbols, monad syntax (do) and lazy evaluation. Kotlin's are built with typesafe builders and inline functions. Scala's are built with symbols, right-associative operators, macros, implicits, monad syntax (for) and possibly even dynamics.
Even this most powerful yet most dangerously-clever capability offered by some programming language is supported by like five different Scala features with ten different ways of achieving the same goal. It's absolutely perfect if you want to explore different ways of programming. It's not so great if you need to maintain your DSL for the next 10-15 years in a large organization.
Besides, given that the JVM has two of most advanced meta-programming tools anywhere, Xtext[1] and JetBrains' own MPS[2], I wouldn't call any JVM language's embedded DSLs "without rival". All this is about to get even better with Truffle/Graal in Java 9. Saw an interesting talk about easily constructing very efficient languages with Truffle and object algebras at ECOOP[3]. So, especially on the JVM, going for an embedded DSL, let alone one built with very complicated techniques (rather than just Lisp macros) is probably not the best way to use DSLs. It has no advantages and lots of disadvantages.
I very quickly skimmed through that scala style guide and starred it on Github, but having used both Kotlin and Scala, I am of the opinion that Kotlin (once it reaches v1.0) would remove the need for ~50% of the content of this style guide.
Scala was our primary candidate, but there were issues back then, namely in terms of compiler performance and also tooling. Tooling isn't easy for Scala, and I believe compiler speed, Kotlin is already faster than Scala in terms. Also, we want to continue to provide support for JDK 6 thru 8.
I can't speak for JetBrains, but from my own experience with Java, Scala and Kotlin, Java interop in Kotlin is a lot better. Scala also offers a lot more features than Kotlin, which is great for highly experienced programmers, but usually means having to subset Scala for mixed experience teams. Kotlin seems to strike a better balance in terms of approachability and expressiveness, and hence being better suited for teams of different experience levels.
Clojure's Java interop is smoother than Scala's, but not as seamless as Kotlin's. Kotlin is modern syntax for the same Java APIs and programming philosophy. Scala and Clojure are radically different languages (from Java as well as each other), for better or worse.
Kotlin is a modern Java-replacement, while Clojure and Scala have different strengths and weaknesses. That's not to say that you can't write many applications in any of the four languages, but the "feel" of the codebase would be completely different in Clojure or in Scala, while Kotlin would feel the same (but would be leaner).
By "performance" I think you specifically mean "startup time."
I think there is a big difference between startup time and runtime performance. It is well-known and fully acknowledged by the Clojure community that starting up a Clojure environment and REPL takes a while. But are you also suggesting there is a problem with Clojure runtime performance?
> be concise but still expressive, toolable, as fast as Java, easy ramp-up time, and very important, interoperable with all the existing code base we had
... and when you went to the Scala developers, they said "no, we aren't interested in such improvements at all, even if you implemented them yourself"?
Was there a need to develop a Java IDE in Java to begin with? I can understand dog fooding by building IDE-primarily-for-lang in lang itself. But were there technical reasons?
Or maybe Java just was the best alternative [at the time].
It's worked out pretty well for them I'd say. Consider that any developer can write a plugin for it easily, that cannot segfault the entire app, and which runs on all the platforms the IDE supports. That's just one advantage. Higher productivity of feature development is probably another.
To see how things could have been different, look at Visual Studio which is still a 32 bit program even in 2015. Users of it complain that it simply can't load big projects without running out of RAM even if their computer has enough. The VS team says they aren't doing a 64 bit port because it's such a big code base and besides, their plugin model is fully native. And of course it's Windows only.
At the time (~2002?), I think Java was a very reasonable choice for an cross-platform IDE. As more programming language research and different programming paradigms continue to gain mainstream adoption, Java's emphasis on backwards compatibility and stability makes it a bit difficult to change some of the initial design decisions that Java started out. However, since Java is so mature with an even more mature eco-system and an incredible runtime specification (JVM), it made sense to build a modern day "blue collar" language that could be classified as "Java, if it started out today." With that in mind, and some of the same principles that Java was founded on, Kotlin delivers on this promise. IMHO, it is the only JVM language that really does bridge this gap.
They didn't have to - JetBrains has been around for a long time and Kotlin is new. The productivity benefits mostly come from things like fast compile times, garbage collection etc.
As a user of it, I appreciate that I've been able to use it on three different OS's with minimal adjustment, which Java facilitates more so than other languages, especially other languages which would be reasonable candidates for a thick-client application like this.
It seems that Kotlin was adequate, which is why they chose it. With no idea of the project or the team working on it, I'll assume good faith that they were right.
They didn't choose Kotlin; they created it. (Though, having looked into alt JVM langs recently, I agree that they probably made the right choice in creating something new).
I remember reading comp.compilers for a long time and people would always post about creating the next great language. They first thing people (usually John Levine) would ask it, "Why does the world need yet another programming language?" Not necessarily to discourage, but to invoke thoughtfulness.
There's nothing wrong with new languages. But it seems people are popping them out left and right these days. I think Javascript frameworks are the only thing that come faster.
So, do we _really_ need yet another new programming language?
Down vote me all you want, but at least I ask the question.
Well, 99% of those are silly little "my first programming language" thingys, and these are easy (if you care about those things) to recognize and ignore. They invariably focus on syntax: removing the semicolon, or braces, or some other trivial thing along these lines.
BTW even those can be useful for PL researchers as the language may have an interesting new idea or a new way of doing things. Highly unlikely, but does happen.
On the negative side, and this is really bad, some of these may be picked up simply because they are new, and then they just create negative value for people that pick them up (e.g. Coffescript) and fragmentation. Then they get abandoned because there is another newer thing. Sad.
Then there a languages made by people with background in creating viable (as in not-just-academic) languages. Typescript and Go comes to mind. These are good and "we" need them. These may add new features to the table, eg generics or union types or goroutines.
There is no easy answer to your question, my best answer is "depends on who made it".
Given the candidates at the time, it was decided to start Kotlin.