Given the candidates at the time, it was decided to start Kotlin.
Yet, I think many of Scala's users will find that Kotlin gives them all they need, in a much simpler, more readable and maintainable language. If, however, you're aching to write type-safe monad transformers, you're bound to be disappointed by Kotlin.
That said, it is good to know that "pron" was incorrect/exaggerating when they said Scala adopts a lot of PL concepts; thanks for setting the record straight (though I'm still just not sure where Scala stands on that after reading this).
Saying that, there is always some truth in what pron brings to the table, it's sort of a higher kinded trolling ;-)
As for Scala and PL concepts, sure, plenty on offer if that's of interest, but not at all a requirement in using the language. A great example of a highly successful project based on the non-FP-centric aspects of Scala is Spark. See their high performance computing style guide , no monad transformers there...
It was after leading an attempt to adopt Scala at a large organization, which failed because of what I saw as Scala's bait-and-switch marketing: it presented itself as "a better Java" while it really was (and is) a strict-Haskell-with-OOP-and-macros-and-more. We needed a car and got a helicopter. While some would consider a helicopter superior to a car, I think everyone would agree it imposes a completely different set of tradeoffs. So yeah, for a good while I was telling people, listen, it's a helicopter, so make sure a helicopter is what you want (the Scala community’s response was, what is this FUD? Can’t you see it's got wheels?! You can drive it on the ground if you want; you don’t have to take off! To which I responded, but you still can’t take it to a car garage for maintenance, to which their response was, you’re a stupid biased troll!). I don't remember when was the last time I participated in a Scala thread, but that community doesn’t forget. So if I'm damned if I don't, I might as well do:
Spark is a good example for a project that would have been better off using Kotlin. They wouldn't have missed any Scala features, would have had better build times, better IDE support and better Java interop (of course, Kotlin wasn’t mature enough to even be considered when that project started). In fact, using Kotlin would have saved them writing 95% of that style-guide, which reads like instructions on how to drive a helicopter on the ground without accidentally taking off (Perhaps ironically, that style guide drew derision from the Scala community, who said, you're in a helicopter for fuck's sake! Why won't you take off?!)
: No implicits, no multiple argument lists, no Try, no infix notation, no non-math operators, no recursion, no being-fancy-with-monads, no Scala concurrent collections, no Scala collections, no for loops. They don't mention dynamics and structural types, but I think it goes without saying that they're off-limits, too.
Seriously though, I doubt Kotlin would ever be the first choice for a project like Spark since Scala excels in the DSL department (really without rival on the JVM, and off, only Haskell comes close). Look through Spark source code  and you'll see: operator overloading, (sealed) ADTs, exhaustive pattern matching, etc. functionality that you can't even think the thought in in a language like Kotlin. This has nothing to do with monad transformers and other FPisms, just straight DSL power.
Finally, to put things in perspective in terms of the mindhsare that Scala and its ecosystem are drawing, IBM is investing over 100 million dollars per year in Spark research and development . Sorry, but that's absolutely massive, maybe there are no dragons? Or if there are, the big players don't seem overly bothered.
As an example, check out funKtionale. It's a library that adds various functional features like function currying and partial application to Kotlin, entirely by using its DSL features.
2. I don't see how IBM's investment has anything to do with Scala, though. Spark is a great product, and deserves an investment even if it were written in BASIC.
3. Even if Spark did rely on Scala features that aren't in Kotlin (it doesn't), Spark is a library/framework/middleware completely unlike enterprise projects. It is orders of magnitude smaller, for one. That is not to say it isn't awesome, but I wouldn't extrapolate from its choice of technologies (including the language) to the enterprise.
4. Clojure makes writing DSLs easier and with far less magic. Groovy DSLs are also as powerful as Scala's and easier to write. Kotlin DSLs are not as powerful, but are more than enough for Spark.
5. I've never, ever, said Scala is unusable. In fact, I've always said (and believed), it is a very powerful language. I have said that that kind of power is not what most companies want or need, and that that kind of unopinionated kitchen-sink, cutting-edge language is probably not suitable for long-time maintenance in large companies. I completely stand by this. My problem with Scala, again, is not that it's a bad helicopter, but that it insists that it's a good choice if what you want is a car (unless you actually try to drive it like a car -- as Spark does -- in which case you invite the ire of the entire community), and, IMO, it isn't.
6. It's OK to not like your language. It's OK to say it. It's OK to explain why. And it's OK for you to disagree. That doesn't make me biased, and certainly not some kind of bogeyman. It also doesn't justify ad hominem attacks, which -- unlike my criticism of Scala -- are completely unfounded.
> None of those features (except sealed ADTs, which really aren't necessary) is missing from Kotlin. The set of Scala features Spark is using is pretty much identical to Kotlin.
I didn't mention implicit conversions, but that's one area where Scala has taken the high risk high reward path. If you look a bit further down on the linked Spark page you'll see the conversions defined . I'd say Spark's style guide is more geared toward Scala consumers and not necessarily library authors (i.e. if you know what you're doing have at it; otherwise keep to a strict subset).
Basically, in aggregate Scala's feature set is incredibly powerful; that power (despite the complexity, tooling challenges, and slow builds) is what draws dsl authors to Scala.
When IBM throws down an absolute boatload of cash on a project written in Scala, by proxy Scala benefits. Right off the bat there are de facto 3,500 new Scala developers (since the initial IBM Spark team will of course be learning Scala). Furthermore, it's not like competitors aren't aware of this, Spark automically becomes more of a "big deal" as a result, which cannot help but be a boon for Scala.
Now, Kotlin, it's a nice language. If they can deliver on near Java build times, might just be the Java killer that Scala will never be, or at least not in its current design (where, as you rightly say, there is often more than one way to do the same thing).
Let's see how things play out over the next 3-5 years, JVM alternative language landscape is diverse and evolving quickly, good times now and great times ahead...
We C++ veterans know that implicit conversions are one of the worst ideas to put in a language. They start simple enough and then they get clever, which is where trouble starts.
> i.e. if you know what you're doing have at it; otherwise keep to a strict subset
There are few things about programming language design we know, because getting empirical data is so hard. One of the few things we do know (from C++ in the nineties, of course) is that this kind of design doesn't work. The average codebase lifespan is about a decade, during which time people move around and team leads change. Everyone has their own discipline, and by year 5 (usually earlier), the codebase is unmaintainable.
Of course, now that C++ has been re-branded and re-purposed as a specialized language for close-to-hardware software, and is used by specialists, it is doing quite well. I certainly wouldn't mind Scala marketing itself -- like C++ and Haskell do -- as a language for specialists (although even specialists would be better off opting for the excellent OCaml implementation on the JVM, or even Frege if Haskell is your cup of tea).
There is absolutely no reason to be as multi-paradigm as Scala is on the most polyglot platform around, which brings me to:
> that power... is what draws dsl authors to Scala.
One of the great things about the JVM is its relatively easy inter-language interop. I would use a meta-language made for DSLs (like Clojure, MPS or Xtext) for my DSL, while I'll write the infrastructure in Java/Kotlin. Learning Java and Clojure, or Kotlin and MPS, is easier than learning Scala alone (well enough for DSLs), and the results are cleaner, produce more maintainable code and better error messages (well, I wouldn't say Clojure excels at error messages, but at least the DSL writer has full control over them, and can make them quite good with an extra bit of effort).
Also, I'd like to see what those DSL codebases look like after five years of heavy usage (not to mention the error messages Scala spits out, which are completely unparsable to anyone who does not understand how the DSL is constructed).
Too late to yank them now, can be disastrous in the wrong (read: beginner) hands, but they're fantistically useful in the dsl department so...
> The average codebase lifespan is about a decade
maintainability is an issue for the long haul, sure, Scala won't win that battle until it stabilizes, which is at minimum 3 years away when Dotty (Scala 3) comes on the scene.
> I certainly wouldn't mind Scala marketing itself -- like C++ and Haskell do -- as a language for specialists
Typesafe does the marketing, and as a business they'll never sell their technology short, even if what you say is closer to the truth wrt to specialization.
> better off opting for the excellent OCaml implementation on the JVM, or even Frege if Haskell is your cup of tea
no users = no dice; doubt either will grab much mindshare in the near-term. Personally I would love to see SML cherry pick the non-crufty good parts of OCaml (including coming implicit parameters) and have that find a following outside of current tiny academic circle it languishes in.
> Also, I'd like to see what those DSL codebases look like after five years of heavy usage
They probably look a lot different than how they started out as...paraphrasing someone, "you're going to rewrite it [your application] at least 3X" ;-)
Basically the entire Scala ecosystem is a moving target, including the language itself. Enterprise outfits looking for a static dependency graph that will work today and 10 years from now should look at Java as the host language. I doubt Kotlin will be able to deliver the same guarantees, but if it can, watch out Oracle.
I don't think that's selling it short. Accurately describing your product only prevents people who aren't really your target -- like me -- from being disappointed and telling the world about it, while it helps attract precisely those people who would be most excited about it.
Also, I think Typesafe realize now that there is not much money to be made in developing and supporting a programming language (ironically, there is some to be made -- though not too much -- supporting specialist languages; see Ada and Matlab). That's why they want to change their name, right?
> no users = no dice; doubt either will grab much mindshare in the near-term.
That's splitting hairs a bit; Scala doesn't exactly have mass appeal either. We're talking about the difference between tiny and small.
> I doubt Kotlin will be able to deliver the same guarantees, but if it can, watch out Oracle.
Even this most powerful yet most dangerously-clever capability offered by some programming language is supported by like five different Scala features with ten different ways of achieving the same goal. It's absolutely perfect if you want to explore different ways of programming. It's not so great if you need to maintain your DSL for the next 10-15 years in a large organization.
Besides, given that the JVM has two of most advanced meta-programming tools anywhere, Xtext and JetBrains' own MPS, I wouldn't call any JVM language's embedded DSLs "without rival". All this is about to get even better with Truffle/Graal in Java 9. Saw an interesting talk about easily constructing very efficient languages with Truffle and object algebras at ECOOP. So, especially on the JVM, going for an embedded DSL, let alone one built with very complicated techniques (rather than just Lisp macros) is probably not the best way to use DSLs. It has no advantages and lots of disadvantages.
Kotlin is a modern Java-replacement, while Clojure and Scala have different strengths and weaknesses. That's not to say that you can't write many applications in any of the four languages, but the "feel" of the codebase would be completely different in Clojure or in Scala, while Kotlin would feel the same (but would be leaner).
Just check how the JVM gets the blame for what is essentially Clojure initialization time.
There are some in the community trying to improve the situation, but those are kind of forks like Skummet.
Ridiculous. Clojure devs care a lot about performance. We just care more about correctness.
I think there is a big difference between startup time and runtime performance. It is well-known and fully acknowledged by the Clojure community that starting up a Clojure environment and REPL takes a while. But are you also suggesting there is a problem with Clojure runtime performance?
... and when you went to the Scala developers, they said "no, we aren't interested in such improvements at all, even if you implemented them yourself"?
Hard to imagine...
Or maybe Java just was the best alternative [at the time].
To see how things could have been different, look at Visual Studio which is still a 32 bit program even in 2015. Users of it complain that it simply can't load big projects without running out of RAM even if their computer has enough. The VS team says they aren't doing a 64 bit port because it's such a big code base and besides, their plugin model is fully native. And of course it's Windows only.
So productive that they had to invent a new language for the JVM platform?
Color me skeptical.
I remember reading comp.compilers for a long time and people would always post about creating the next great language. They first thing people (usually John Levine) would ask it, "Why does the world need yet another programming language?" Not necessarily to discourage, but to invoke thoughtfulness.
So, do we _really_ need yet another new programming language?
Down vote me all you want, but at least I ask the question.
On the negative side, and this is really bad, some of these may be picked up simply because they are new, and then they just create negative value for people that pick them up (e.g. Coffescript) and fragmentation. Then they get abandoned because there is another newer thing. Sad.
Then there a languages made by people with background in creating viable (as in not-just-academic) languages. Typescript and Go comes to mind. These are good and "we" need them. These may add new features to the table, eg generics or union types or goroutines.
There is no easy answer to your question, my best answer is "depends on who made it".