Hacker News new | past | comments | ask | show | jobs | submit login
The Rest of the Story (Yammer & Scala) (codahale.com)
198 points by pbreit on Nov 30, 2011 | hide | past | web | favorite | 60 comments

I can appreciate that having a personal opinion about Scala's use in one particular context belched out into the world is unfortunate, but I have to say that Coda's email reads like my laundry list of concerns about Scala. I could have written this email just by theorizing about Scala's shortcomings based on my short time playing with it. Except in this case, it's correlated with real-world stories.

I like Scala, and I think it's the most powerful and flexible mainstream language available today. Unfortunately, that's the problem.

Wow. If this is Charles Nutter I'm surprised no one has commented on this. I'm sure someone could make the argument that as there is ample evidence that humans are much less rational than we typically care to admit, that we are feeling animals who also happen to think rather than thinking animals who also happen to feel, so it is that the creator of what it is today easily the most popular non-Java JVM language could conceivably be "blinded" by their own personal investment in a "competing" language. But I think the much more interesting fact about this comment is that the commenter is basically a fucking programming god. And he thinks Scala is too complex. If I was Colebourne I would be posting on my blog, a very short post, basically just saying "Hey, all you bitches telling me 'Yeah, yeah, yeah, you wrote a little date library, WTF are you again?' Does Charles Nutter carry enough weight for you?"

My own personal viewpoint is that the Nutters and Colebournes are totally on the money. I remember reading a quote from Miguel DeAcaza talking about thread based concurrency programming models in the new N-core paradigm, he said it didn't scale, because "there are not enough PhDs in the world." Despite it's name, I don't think Martin's baby will in the end have lived up to it. Didn't scale, not enough PhDs in the world.

"If someone asks me for an honest opinion of them or their work, in private, I feel morally compelled to be as honest as I can with them."

I can vouch for that. Around a year ago I asked him for feedback on Cassandra and his response was both thoughtful and helpful. I appreciated that a lot. Thanks, Coda.

Can you share that Cassandra feedback, please?

Is that response public?

He wouldn't say that in public because it would be harmful to the Scala community, but everyone who read it is very glad for having read it.

But if everyone thinks this way, those innovations will have one view on the web (where there is an image to care for) and another one of engineering departments, where people get real... and that sucks.

I think his point was that the original email was very particular to his use, and it might not have the same issues used by different people. So his public opinion of Scala wouldn't be him lying, it would be him generalizing it more. He was asked what the problems were so he answered. If it was a general post about the language, more care would have probably been given to point out both the negatives and positives. My .02 on how I read the various posts.

Obviously he took the time to write this post in such detail because he hopes that its intended recipients can take action to address the issues he raises.

It's not like there isn't already plenty of Scala bashing going around these days anyway.

I think it is very constructive and I expect that improvements caused by it will land in trunk soon.

I found it constructive too, but its not like the Scala implementors only heard about those problems now and can easily fix it.

I'm curious to know why Yammer went with Scala to begin with (not to imply that it was a bad decision at the time).

Was it Language of the Week the week they started building Yammer? Were they trying to follow in Twitter's footsteps? Were they a bunch of Java developers looking for something better than Java? Did someone on the team have previous Scala experience?

Would Yammer be where it is today had they gone with Java (or another language, JVM or not)? Before running into the limitations described in the leaked email, was everyone happy with Scala?

Their whole product is a ripoff of Twitter so that was probably it.

Not just Twitter. Twitter and Facebook combined.

It's good to see honest evaluations of technologies. It helps people better decide when something is mature enough to adopt for their organization, and helps in avoiding the landmines when they do. It also lets the the "owners" know what is impeding adoption so they have better focus. Scala is several years old now and really is gaining adoption in larger (ie conservative) firms.

Scala is far from perfect and has some warts, but that's the type of technology that always seems to win. C++ vs Eiffel, Intel vs Motorola.

Why are the needs of the "Scala community" to save face more important than the truth?

The reality of the Scala community is that most of the major players (the Typesafe side) prior to commercialising directly were hoping for some level of popular adoption with devs that would bubble up into startups/SME and for whatever reason that hasn't happened. So now they are banking on the idea they can sell directly into large enterprise space as a one size fits all approach that involves cross training developers (as there are very few out there due to the mentioned lack of adoption). This is pretty risky and needs momentum that can easily be killed by negative stories.

A lot in the original email is correct (I can't speak for the performance stuff as my skills are more webdev side of Scala), especially the way that libraries/OSS projects/sbt pan out. Something like Dispatch that is a fantastic tool and should be simple is a mess of functions-as-operators and poor to no documentation or examples to the point where for some things even reading the source code is not a lot of help. I'm not going to type out my thoughts on the shambles that was the sbt rewrite at 0.10+... just complete madness.

Evangelists sold a bills of goods to non-technical managers. Can't be letting something silly like facts or criticism get in the way of putting a pet language choice into production.

He's not even a little wrong about HN.

In HN's defense, the original email contains relevant information, and it should get attention. Of course, it has to be understand in context, as things change (specially after such thoughtful email) and not every app has the same constraints as the Yammer.

Not all of HN's discussion is keen on it, but often some of it has good signal.

He's not. And so I laughed when I opened this thread and the highest rated comment was a java vs scala post. Rather than a reply to his thesis.

A Java vs Scala post is far more interesting to me than discussing the "shut up about the tools and just work" thesis.

He's not wrong, but at the same time I don't think anybody who so enjoys pissing contests on Twitter can afford to be too condescending about slapfights on HN. The last thing either place needs is more hipper-than-thou posturing.

I am getting more and more tried about this kind of arguments, I am studding Scala and I like it, It can be right for some things and wrong for others, it's just another tool in my toolbox like Java, JavaScript, Ruby... You gotta have tools, and be able to choose between a Hammer and a screwdriver when needed, that's the real "Hacker"

I would like to see a time where JVM languages completely overshadow and remove Java from the scene, leaving only the VM itself as a requirement. It's a terribly depressing tool to use, and it's frustrating when I have to refer to it when using Scala or Clojure, particularly in I/O .

Coda's opinions, even if leaked, were highly appreciated. This guy has invested a lot of time on Scala so it's in better position to say what he thinks about the language. In the end, it should help to fix imperfections and help the community.

Yammer using or not using Scala isn't going to make or break the language. Adoption seems to be ramping up in organisations of all sizes: http://thecodegeneral.wordpress.com/2011/11/27/scala-adoptio...

Can't help but upvote half the replies in this thread. Sometimes I forget how many ridiculously smart people hang out around here. Happy to be amongst you all, even as a generally passive participant.

Here's why I, personally, prefer to put all my investment into languages like Scala and Clojure.

Scala and Clojure, for their problems (which are not severe, but not to be ignored) have the advantage of being closer to the languages and approaches of the future, in which building comprehensible systems at very high levels of complexity/intellectual intricacy will be even more important relative to performance of executables. I expect that the problems of these languages (including performance) are going to become much less major over time, just as performance gaps between Java and C and between C and hand-written assembly have nearly disappeared.

Spending another 5 years in Java means you're preparing yourself for a 1996 job in 2016.

Why Scala and Clojure? Unfortunately, the 10-year forecasts for Haskell and Ocaml aren't great. I wish they were, because those are great languages. The reason: libraries and tooling matter, and they seem to be mattering more as the years pass (while differences in performance between languages narrow and matter less over time).

Hiring and talent: It's true that you can't hire 50 Scala programmers per month. On the other hand, most startups are far away from being in a position to hire 50 programmers per month at all, much less 50 good ones. Choosing an "esoteric" language works both ways. On one hand, it reduces the total pool of ready developers. On the other hand, I know companies that use Haskell and Ocaml and are able to hire 60-75% of their in-office interviewers while remaining selective; this would be impossible for a Java shop-- there are just too many unqualified Java developers. Choosing a language like Scala or Clojure will do a lot of the filtering for you.

"Spending another 5 years in Java means you're preparing yourself for a 1996 job in 2016."

In the type of environments where I work, five years experience shipping successful software will advance your career further than five years experience in one particular language.

If you choose a bleeding edge language, what is the risk you will spend time solving edge cases with the language instead of delivering great software? Given everyone's lack of experience in a language, what is the risk you will write something unmaintainable? If your bleeding edge runtime is slower, what is the risk that it will fall apart under load?

It's possible these are acceptable risks for the gains in programmer productivity. That risk to gain ratio would have to be massive to persuade me.

Yes, picking bleeding edge technology will filter out bozos during hiring. You'll also filter out engineers who are more passionate about shipping than new technology. You're left with an high concentration of engineers who have never seen bleeding edge technology blow up in their face.

Do people think the hiring and talent argument holds water?

The more I hear this argument (e.g., from the "Python Paradox"), the less I buy it.

Is the theory that true "hackers" care more about the programming language syntax and semantics than whether it has the right libraries for the subject domain of the startup?

Is the theory that true "hackers" care more about whether the startup uses a new and un-tested programming language than traction, likelihood of success, money, or the actual problems being solved?

Do the best "hackers" come in by searching job sites for niche programming languages, rather than referrals?

Even ignoring the huge disadvantages in library support, tooling, community sizes, and so on, are the hiring and talent advantages believable? Are the sort of people who learn Haskell also the sort of people who would be 99th percentile C++ systems engineers for some reason?

Startups are outliers, most of the metrics you mentioned for startups are at best noise and at worst cargo cult business.

The important thing is that people who are using esoteric languages more likely to try new and off the wall things that could lead to a new type of business. A new italian restaurant is not a startup, a restaurant that serves only dehydrated food might be.

If your startup serves italian food then you should probably go hire a bunch of java programmers the python paradox doesn't apply, and arguably most startups by number are serving italian food. (eg. your latest mobilesociallocal coupon startup). If you're truly doing something new then you're probably best off hiring people used to doing new things where there are no established best practices. (eg. trail blazers) You're most likely to find trail blazers on the trail rather than reading best practices docs.

The python paradox is about people not programming languages. (At least that's what I took from it) Also, there's nothing wrong with a new italian restaurant, on average it's probably going to be more successful than an dehydrated food establishment, but if you're looking to create a new nation-wide fad, you'll probably have more luck with dehydrated food than italian restaurants.

So it means you're used to following through on your curiosity, and capable of fast growth? That's at least what I'm getting out of it right now.

There are all sorts of people those that get their kicks on language selection and semantics, there are those that love to discuss process (agile, scrum, TDD, whatever), others that want to focus solely on delivery. Reality is, the best people are somewhere in the middle.

An engineer worth hiring is one that is bright, curious, and learns quickly. That said, they should also have an interest in some aspect of what you are producing. I, personally, have searched for jobs on the basis of esoteric / niche items, mainly hoping for something different. The most rewarding experiences have been where I've worked with/brought in good people and built cool things. One of those involved Windows Server and WinCE work, and I personally dislike working on Windows. But the overall project was interesting.

Technologies, languages, approach, and product. A good engineer who is interested will likely have deeper interests and help contribute in one of those four areas.

I would certainly prefer a startup that uses non braindead languages.

If I were to code in Java all day I might as well get a more secure job with a real company.

Of course there is a slighting scale here -- I would still prefer interesting problems, great environments, vesting, etc.

But if I worked at Yammer now I would be pretty annoyed.

The 99th percentile C++ systems engineer uses C++ in their day job, likes it, and uses Haskell on the weekends for fun. They can also give you a detailed set of things Haskell needs to fix before it becomes more widely used (fast Mutable Maps among them).

> Is the theory that true "hackers" care more about the programming language syntax and semantics than whether it has the right libraries for the subject domain of the startup? Is the theory that true "hackers" care more about whether the startup uses a new and un-tested programming language than traction, likelihood of success, money, or the actual problems being solved?

As a basically useless anecdote... for random mucking around, I use whatever language sounds interesting. This has been Perl, Python, Forth, CL, Clojure, Ruby, and many others by now. (I used to consider myself a hacker, until I realized I wasn't working on any hard problems.)

But for real business, I'd choose something I know I can deliver with, which is pretty much Perl, C, or PHP for me. I've tried to extend this set by playing in Java and C# a few times, but the interest just isn't there. (Edit: I don't seem to have any problem delivering on boring problems in php, though.)

I think the idea is that enthusiastic hackers are more likely to try new and "weird" things than, say, somebody who learned a language in school, got a job at some BigCo that happened to be using that language, and spends their days sticking that language's libraries together with bubblegum and Popsicle sticks.

You'll probably filter out some good candidates, but people who make it through at least stand out from the crowd in some positive way.

I think it's this: the best programmers generally want to learn new languages and approaches. This means that by mid-career, they've been exposed to at least 5 languages and seriously used two or more. They've at least tried functional programming. Rarely is the general-case favorite, for such a person, C++ or Java.

Finding great programmers who know C++ and Java is not very hard. Finding great programmers who prefer those languages is quite difficult.

Agree on Java, disagree on C++.

Many outstanding engineers use C++ (or a particular variant thereof) and like it.

C++ is the only choice to write serious performance critical apps with acceptable abstractions. The STL may be hard to debug, but it's the only way to do certain things.

And now with C++11, we're going to see it starting to give many other languages a run for their money.

I think it's this: the best programmers generally want to learn new languages and approaches.

This is true, however an inherent danger to Haskell/Lisp/ML-filtering is that this group has also a relatively high percentage of people who think every problem is a nail.

So filter for people who know functional languages, but also have plenty of experience with, say, C or C++ projects.

You really believe functional programmers are more willing to treat their language choice as a golden hammer than C++ or Java programmers? You must have installed a different version of reality than I have.

I didn't say filtering on Java was any better. Just that 'filter on Haskell or Lisp' is no golden rule. You will miss out on good candidates. There are great programmers out there who never really touched Scheme or Haskell outside a CS course.

Also, I think that a fairly large subset of C++ programmers who think that C++ is an ugly and complex language. But it's often practical, e.g. because it is used in legacy software, because it's fast (and provides more abstraction than C), or because many good cross-platform UI libraries use C++.

Languages are but a tool to build things. What specifically about Scala and Clojure, do you feel, make them closer to the languages and the approaches of the future?

While one should keep abreast of current technologies (and languages), there is still plenty of work that requires knowledge of C from the system level to SAAS platform technologies and implementations. It is a 35+ year old language.

If your argument was "learn these knew languages for the advantages the provide given the likely emergence of X, Y or Z", I would buy it. Saying a newer language is a path to the future? Lacking.

Functional programming more easily fosters the parallelism that will be required to keep making performance gains as individual core speeds top out and CPUs only get faster by adding cores.

My experience is exactly the opposite: in number crunching it's often easier to parallelize processing in a low-level language such as C or C++. Current C compilers make loop parallelization via OpenMP. And since data is mutable, you do not have the overhead of allocating new chunks of memory (plus garbage collection).

Functional languages will eventually be able to do most parallel computations on vectors and matrices nearly as fast as C. E.g. in Haskell a new project has started for adding support for SIMD intrinsics. Packages such as repa have been shown to be much slower (see the repa papers), where a program using repa with 8 cores (without SIMD) has about the same performance as a single-threaded C program (with SIMD). With SIMD support, Haskell plus repa will probably get a lot closer to the performance of C.

Where functional programming with immutable data structures does help tremendously is in concurrency. Anyone who has written a fairly complex C++ program using concurrency, such as a GUI application, has probably run into complexity issues. There is a lot of mutable state, and no clearly defined model of interaction between threads.

Anyway, in my opinion the real benefit of a pure functional language is that code written in a functional language is often more succinct and easier to reason about. There is no state outside a function (either global or within an object), side-effects are controlled and typing provides neat abstractions (monads, functors, etc.). The primary source of complexity in some functional languages is laziness.

There are functional programming languages (Erlang, for instance) and the concepts functional programming embraces. While the functional languages make these concepts explicit, people have adopted the concepts into their languages of choice. Search on "functional <X>", for instance, functional javascript brings up an interesting article on IBM developer works:


For, myself, I am open to the concepts, but I don't need to be necessarily locked into a given language in this case.

Some of the functional advantages don't really work until fully adopted and supported by language runtime and syntax explicitly. Immutability in particular I consider really an all-or-nothing proposition. Either you can count on the language using value-only-based semantics, or you can't. Any attempt at middle ground will fail when you're someday dealing with a library built on a library built on a library that, whoops, turns out to use the mutable variables your language supports after all, and then the mutability seeps out from there. Or your junior devs come in and spray mutability around 'cause that's what they know and before you're right back in the morass. (One I've grown sensistive to, since it's the one I work professionally in.)

(Also just declaring the semantics of how mutable variables and immutable variables relate in one language turns out to be really tricky problem. Not impossibly tricky; it's been done, after all. But I suspect trying to mix the two in any sort of deep way means that you have unavoidably created a cognitively "large" language.)

If Haskell is going to "win", it'll be on the strength of properties like these, that either you go all in, or you don't get the benefits of them, just the costs. I think instead of 90-10, "90% of the benefits for 10% of the work", it's rather closer to 10-90.

If this is true, can you point me to an elegant implementation of, say, the Either monad in a non-functional language? Bonus points if you can show me how to implement algabraic data structures plus pattern matching in your favorite non-functional language.

I don't see how someone writing Java in 2016 would be stuck in a 1996 job.

Is the same true of someone using C or C++ today?

It's not so true for C and/or C++, because some of their niches are cutting-edge, like gaming and VMs.

Where it is true is for languages like Visual Basic, or heaven forbid COBOL. If you are working with COBOL today, or even ten years ago, you're probably maintaining a 1980's mainframe for a 1980's company (VB is probably 90's).

If I were to pull a prediction completely out of my ass, then my complete lack of experience or expertise in the matter says that Java is more likely to end up the VB of tomorrow than the C++ of tomorrow.

It depends what the person's using the language for. I'd say "no" for C.

C is still the leading language for a lot of purposes: operating systems, runtimes for higher-level languages, device drivers, and applications where tight control of memory use is important. In these domains, C hasn't been supplanted yet-- not even close.

I don't know that C++ and Java have substantial domains in which they're superior to other choices. C++ is a badly-conceived cross between C and high-level languages and Java is the worst of the JVM languages.

C++ isn't a badly-concieved cross. It is a bucket with everything thrown in.

That said I wouldn't want to write larger programs in C and would prefer C++ simply because it scales better (assuming the other programmers don't use all the features it has, but confusing code can be written in any language). Also RAII is awesome.

In AAA game development, it's C++ or go home. People can argue all day about the relative merits of Java, but C++ certainly has a domain in which it's superior to other choices.

C++ has a domain where it has a large foothold and extreme inertia. That says nothing about its fundamental merits.

Your statement is also misleading for another reason. Core engine logic might be in C++, but actual "game" logic is increasingly being shoved into scripting languages like Lua.

Concern for and access to low-level implementation details is one of the fundamental features -- and in some cases merits -- of C++. In applications where program execution speed is important and you can't scale out by adding machines, C++ is a better option than dynamic languages. See: game engines. Yeah, you might add a scripting layer on top for some of the logic, but you can't write the engine in a dynamic language and get Battlefield 3.

The other merit of C++ is that it's object-oriented, however awkward its implementation may be (insert Alan Kay quote here). The fact that it's both fast and natively object-oriented is why C++ for the most part replaced C in the game development community, which once had as much C-based inertia as it currently has C++-based.

Is it a beautiful language? Noooooo way. But it has merits.

Edit: before someone responds with "But computers will get faster and make C++ obsolete..." People's expectations for videogame graphics tend to increase as better graphics are made possible. It's a tough fight against an equilibrium, and the fact that the graphics of today will eventually be possible with dynamic languages in the future won't matter for as long as games are unable to reach perfect photorealism -- better-looking future games will come out, and Crysis-level graphics will look dated and shabby by comparison.

For an example of previously-heralded graphics become obsolete even within a single console generation, when the first screenshots of Oblivion came out years ago there were people who truly couldn't believe that they were realtime and in-engine -- but dear God, look at Oblivion now. It's not hideous, yet, but it isn't pretty. And there's no way you could write that in pure Lua today -- not even at ten frames per second.

This post is not about Scala and/or Clojure vs. Java.

You may be missing a lot of what's happening with GHC: stacktraces, fixing GHCi and cabal, polymorphic kinds



I agree about Clojure, not so much about Scala. Clojure built on solid proven foundations of LISP and JVM. While Scala is also built on JVM, it's semantics are completely new and unproven. And b/c Scala's semantics are complex (unlike LISP), it will be very hard to prove and make it mature.

My other choice is Erlang/OTP, which is 20+ y/o dinosaur, controlled by Ericsson, not a modern language, but very stable and mature.

Who mentioned clojure?

Pompous. If you ever tried to use Yammer's "API" you wouldn't be asking this guy for an opinion on anything.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact