Hacker News new | past | comments | ask | show | jobs | submit login

I enjoyed the presentation very much, until

    "Engineers 3-10x more productive"
Citation needed.

If this were generally true (of any technology X) the dollars would be flowing that way in an unstoppable torrent, and anyone using anything else would be the subject of ridicule and/or pity. It would be like writing custom websites in C.

Since that isn't happening, I have to take assertions like that with a sizable chunk of salt. The alternative would mean that there's a gigantic arbitrage opportunity that's being missed by a bunch of highly motivated people. So yeah, citation(s) please.




The reason is simply that more powerful tools require more skill and experience to use effectively. It's not just a question of switching from imperative language X to clojure and seeing a huge productivity boost. You must also understand why clojure is designed the way it is and how to use it effectively. For awhile, you'll likely be less productive as you learn how to approach problems functionally and wrap your head around the capabilities of lisp, but you'll also be building stronger, more reliable systems, and at the end of the learning process, you will indeed be significantly more productive.

Managing a transition like this is far from trivial for a company facing a variety of immediate demands and constraints, even if it promises to pay dividends in the long run.


> It's not just a question of switching from imperative language X to clojure and seeing a huge productivity boost.

It depends on who you are. I recently used Clojure on a skunkworks feature for our product for which I had three days to complete. I am absolutely convinced that it was at least 3x smaller (LOC) and took me at least 1/3 of the time it would have taken in Java. I am an expert Java programmer and not a newbie to Clojure, but I still have to look up a lot of things.

The killer productivity feature of Clojure (LISPS and other languages) is the REPL. Trying things out, analyzing data structures, etc. is where I would have spent a ton of time in Java. Even if you're not an expert Clojure programmer, just using the REPL can save you tons of time.

So not necessarily disagreeing with you, but I also think that a non-expert in Clojure can still run circles around an expert Java programmer when considering productivity.


I'm not disagreeing with you, but exactly what is it that you're doing in the REPL that you're not able to do in, say, Eclipse's Display view while having changes pushed into your program under debug (when it's able to)? Java disallows pushing some changes into the VM, but many changes go right in during debug.


It's not about what's possible in principle, but whether the tools facilitate a certain workflow. When you work in Clojure everything is interactive. You can write a function evaluate it immediately, see the result, use it in the next function and so on.

The workflow is very natural and you practically never have to restart anything. You can build up state in the application, see how things interact, reload functions add new functions, and so on. Using a debugger in Eclipse is a very pale imitation of this.


Thanks Yogthos, that is pretty much exactly how I would have responded :)


I hadn't ever used a REPL to facilitate my programming before utilizing Gradle & Groovy to code some build files. The REPL made it really easy to experiment with syntax and facilitated my learning as much as Intellisense used to when I worked in the Microsoft stack.


>at the end of the learning process, you will indeed be significantly more productive.

That's standard process for any language.

The question is: why should we assume that Clojure pays more/special dividends in the long run?


I did a writeup a little while back reflecting on why I feel Clojure makes me more productive:

http://yogthos.net/blog/49-Why+I%27m+Productive+in+Clojure


Right, but just about every language makes this claim. Very few support it.

As far as I am concerned, they are all being deeply deceitful. If you make some "fact" up, and pawn it off as the truth, you are being dishonest.


Every café in town might claim to have the best coffee. While most of them are lying, that doesn't mean that some don't in fact have better coffee than others. It's up to you to judge each on its own merits, read reviews, or get a recommendation from someone you think has good taste in coffee.

Support for clojure's worldview is easy to find on the web. It comes both from the clojure community itself (see Rich Hickey's talks) and from the wider communities that clojure draws its ideas from (other functional languages, lisp, etc.) Decide for yourself if you find the arguments compelling. In my experience, the clojure community is not interested in promoting clojure for reasons of tribalism or vanity, but because they can indeed use it to produce better work faster.


> In my experience, the clojure community is not interested in promoting clojure for reasons of tribalism or vanity, but because they can indeed use it to produce better work faster.

Example needed. Cascalog?


While I agree the quoted assertion is bogus, I disagree with your reasoning. There are many arbitrage opportunities that are being missed, whenever technology, science, or anything that requires thinking is involved. This is due to human stupidity and/or unwillingness to learn new things.


And there's many cultural forces at play. Many managers would pay a premium for a large disposable supply of programmers, rather than a smaller more skilled supply. Bargaining power becomes a factor; the smaller labor supply may demand more decisionmaking power than you culturally want to allow.


Can you name a specific arbitrage opportunity because I wouldn't mind making some money.



Just to point out, MapR isn't a slam dunk for a couple reasons. Their HBase implementation is very fast, but it involves using their whole Hadoop distribution. In my experience:

- the management UI isn't as good as Cloudera Manager, or even Ambari in HDP2.0

- 3rd party support isn't there. If you want to use anything you find on GitHub, get ready to add support for MapRFS, and revert the API version back to 0.21

- there are bugs. Weird, specific bugs in the filesystem implementation which will bite you once every few weeks. This is one area where they lag the ASF project: many eyes do make these kind of things shallow. Their smaller install base and fewer developers means these things take a long time to notice and fix.


Their HBase implementation is very fast, but it involves using their whole Hadoop distribution...3rd party support isn't there. If you want to use anything you find on GitHub, get ready to add support for MapRFS, and revert the API version back to 0.21

The MapR team is sharp and responsive. It's founded by ex-Googlers from the search-infrastructure team (http://www.wired.com/wiredenterprise/2011/12/ex-google-man/).

M7 Tables are designed to be a drop in for HBase, with better performance and without the HBase complexity (it's much easier to run M7 Tables as the primary datastore for online apps).

M7 Tables support and certification for several key projects is coming along...

Spark 0.7 already works with MapR M7 Tables with the Hive images that were released with the Eco-1310 (http://doc.mapr.com/display/components/Hive+Release+Notes, http://answers.mapr.com/questions/7803/shark-spark-over-mfs).

MapR is in the process of certifying Titan (https://github.com/thinkaurelius/titan/wiki), and it's going is going through final QA right now (https://groups.google.com/d/msg/aureliusgraphs/RTeFVssIvoI/m...).


While I have no knowledge of whether or not Clojure programmers are more productive, a recently linked mail from Joe Armstrong on Erlang productivity was quite interesting: https://groups.google.com/forum/#!msg/erlang-programming/Oiy...


Confusing correlation and causation. Clojure coders might be 3x more productive, but they would be in any language. The chance is high that only the top 1% (or something like that) of all coders would dare to make the switch to this language, so you get a biased sample to begin with.

The problem with this of course is that when a company tries to switch their more mundane engineers to Clojure, the project will likely fail.


> Confusing correlation and causation. Clojure coders might be 3x more productive, but they would be in any language. The chance is high that only the top 1% (or something like that) of all coders would dare to make the switch to this language

It's too bad that these top coders are so bad at mundane tasks like high-school-level statistics.


I disagree. I think that top-1% programmers actually fare worse when thrown into a typical Java codebase. Sure, they're very productive on their own stuff (whether in Clojure or Java, although I still think those programmers will fare better with Clojure) but if you dropped a 1%er (or even a 10%er) into a pile of someone else's FactoryFactory garbage, he'd either rewrite it or fail completely to make any headway. Bad code is the Waterloo of good engineers.

I've seen companies end up in inversion, which means that the most apparently productive engineers are the worst ones. It's a common effect in "design pattern" Java shops with all their Visitor and Factory and Vibrator and AbstractSingletonFactory patterns. It's not pretty. Good people are either taken badly (because they keep agitating for rewrites) or become disengaged and eventually leave or are fired.


I, as a single Perl (and regex, and VBA -- laugh all you want) programmer, have done things for which an EDS contract was ruled out by a major institution because it (and however much in terms of resources and expense it would entail) would be "too expensive". (Instead, they chose to continue "throwing people" -- full-time staff -- at the problem. Use that as a measure of just how expensive that prospective EDS work was considered to be.)

The language, or its aficionado -- either way, specific technologies have made me, in my own estimation, multiple factors more productive. Yes, there's the learning curve, and understanding things that are not explicitly stated line by line throughout the code. But then, there's getting shit done.

Unfortunately, a lot of people, processes, and institutions get hung up at the sentence before last of that previous paragraph.

I also turned other people onto some useful technologies, such as pointing a consultant at some reporting software that my department didn't want to spring for but for which -- of course -- the outside consultant had budget. I heard from her a month or two later that it had greatly eased things and turned her into a sort of reporting superhero.

Throughout my career, I've seen people who, both on their own as well as through dint of better technology awareness and selection, have been multiple factors more productive than their coworkers.

And yes, often they are fighting the trend and significant inertia, rather than being embraced.


IMO the problem with the unstoppable torrent is that the people who decide where company dollars flow to can hardly tell C from Clojure so they rely on the opinion of those whose job would then be to say "ah yeah right, me and my team have been using the wrong tools all the way" which doesn't happen for obvious reasons.

I started up with clj and I am trying kinda heavy arbitrage here :)


Citation needed.

The line on slide 111, "Engineers 3-10x more productive, smaller teams", is probably in reference to...

Slide 16: "Code is typically 3-10x smaller when written in Clojure."

Slide 110: "3-10x less code. Limiting factor in programming speed is essential complexity of the problem, not typing."

I have to take assertions like that with a sizable chunk of salt. The alternative would mean that there's a gigantic arbitrage opportunity that's being missed by a bunch of highly motivated people.

That there is a "a gigantic arbitrage opportunity being missed" is essentially the thesis of the talk.

He states the reason why (Clojure is cloaked in 50 years of Lisp misconceptions) on slide 21: "Lisp is an old weird AI language no one uses anymore. It's so slow it requires a supercomputer. It's impossible to hire people who know how to use it. Functional programming has been proven impractical. Also, parentheses."

The body of the talk is designed to dispel these beliefs.


As in: the citation is a "reference" to two other assertions in the same slide set, themselves simply stated with no backing evidence.


Yeah, I think he's just confusing the economy of LoC with productivity. I might actually be 2-3X faster with Clojure over Java, but I'm sure I could crank out a CRUD web app in Rails 2-3X faster than in Clojure at the moment..


> I enjoyed the presentation very much, until > "Engineers 3-10x more productive" > Citation needed.

Oh, it's a perfectly plausible number. It increases the productivity of each developer by a factor of 3 to 10 and also increases the maintenance costs by 3^n_engineers or 10^n_engineers. This seems to be the tradeoff that startups make.


I remember when Ruby on Rails developers were claiming this. Yet, I couldn't find one to deliver a website at 1/10th the price...or even 1/5th the price.

EDIT: (yes, being a little cheeky because as we know the physical time programming isn't everything in building a website)


Here I wanted to cite the (moderately) well-known result that defect rate is proportional to lines of code, independent of language, but I can't actually find where it comes from.

Help?


3-10x is, of course, intended as a 95%-of-cases interval with some basic assumptions. Some contexts (e.g. type of problem, engineer, timeframe) will be below 3x and others will be above 10x. (The 99.99% interval is probably 0.2-100x.) Among those basic assumptions is that the engineers have the curiosity, drive, and ability to learn it. Ability is the least issue of the three; if they're not smart enough to learn Clojure, they're not smart enough to Java or C++ well, and they're mostly making messes.

Lisp has a bad reputation compared to what it is. People think it's impractical, that Lispers are arrogant (due to a few vocal but influential people in the Lisp community) and that it's archaic or weird. I've addressed those perceptions, but they're out there. Clojure is, in fact, a pretty damn practical language.

Now, the arbitrage argument: in larger companies, the concerns aren't engineer productivity so much as appearances. For a middle-manager to launch a meaningful Clojure/Lisp initiative would be to put himself out there in a major way, and since programmer productivity is only one piece of a much larger system, it might not have that big an impact on the company as a whole. It might not be worth it. (Would Google's market cap go up 3-10x if it replaced Java with Clojure? Almost certainly not. Maybe the fair value would improve by 1.05x on fundamentals, which would be drowned out by month-to-month volatility.) On the other hand, startups generally can't afford (or don't think they can afford) to experiment with newer, quirkier languages. VC-funded startups tend (surprisingly?) to have stodgy closed-allocation work cultures and boring tech stacks because they're expected to allocate 100% of the "risk allowance" to one department: business-model risk. That's not to say that there aren't other kinds of companies that can (and, in some cases, do) use more innovative tools; but the two dominant cultural forces (VC startups and large corporations) in technology don't favor it.

For the large companies, I'll admit that there's an Amdahl's Law sort of effect at play. Let's say that you improve productivity of engineers by 3-10x. Does that "3x" the whole company? As I alluded to for Google, probably not. It makes something else the limiting factor. For most firms, using the wrong programming languages is nowhere near their top problem. To use Google as the example, it might deserve a 5% bump on market cap if it included Clojure (a small gain compared to monthly volatility, so small that no one could prove causality and take credit) on the white-list but the fair value would go up 50-200% (again, on fundamentals, as I can't predict investor opinion) if it implemented open allocation. Programming language innovation just isn't the top priority for most companies. But for an individual developer, the advantage conferred by a powerful language can be pretty serious.


What do you mean by"open allocation"?


http://www.quora.com/Software-Development-Methodologies/What...

People choose what they work on. No "headcount" numbers that have to be agreed upon by people far away from the actual work. It's the one thing Google could still do at this point that would 2-3x the company.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: