
Why Every Language Needs Its Underscore - Suor
http://hackflow.com/blog/2014/06/22/why-every-language-needs-its-underscore/
======
snitko
Being a Ruby developer for quite a while, Underscore.js always struck me as an
attempt to bring some of the native functionality of Ruby into the language
that had none of it. Internal iterators being the most immediate example.

Problem is that Javascript lacks a lot of the things programmers need at hand.
So they write stuff to complement that deficiency. I write in Dart as well and
can tell you I love it precisely because it provides better OO capabilities
through its core.

Which really brings a very important question to the table: what should be a
part of the language and what shouldn't. A Unix minimalist in me wants a
language to be as small as possible. But then when you actually start writing
code, you mostly want simple things done without having to import new
libraries.

~~~
mmahemoff
It's not a coincidence that Underscore looks like Ruby, its creator has a Ruby
background and similarly developed CoffeeScript along those lines. (Both of
which are perfect for me as someone who uses Rails on the back-end.)

Yes, JavaScript never had anything resembling a decent utils library, so an
elegant and popular implementation like underscore was heaven when it arrived.
It's the usual problem of browsers evolving too slowly and fragmenting too
much for something like Java's collections to emerge as a standard.

The situation has gotten better now, with even IE moving towards auto-upgrade,
but it's still not feasible for JS core to ship with large, frequently-
updated, JS libs. The closest thing is Node's standard library, but that's not
going to be ubiquitous in browser's anytime soon.

"what should be a part of the language and what shouldn't"

In the absence of a standard library, you end up with a dozen competing
libraries trying to fill the void, and that stifles end-user applications.

We saw that a few years ago with the DOM. For the first ~8 years of JS, there
was no significant DOM library, so app developers had to suck up the
unfriendly raw DOM API. Then the toolkits started to emerge, giving way to
huge fragmentation between jQuery, Prototype, MooTools, Dojo, and YUI.
Suddenly libraries were jQuery plugins or MooTools plugins and your app was
using the jQuery stack or MooTools stack. So the next 8 years of JS saw vast
effort duplicated as developers built parallel libraries for each toolkit.

So I think the answer is to launch a language without the kitchen sink, but
watch the community and gradually standardise useful functionality as the need
emerges and adapt popular open source libraries as they becomes popular. Java
did this well in its heyday. In JS's case, a lot of the DOM conveniences are
now becoming core JS, but the nature of JS means libraries move slower than
other languages.

~~~
rnbwd
>the nature of JS means libraries move slower than other languages.

prior to node and npm maybe. The JS npm ecosystem is actually growing faster
than any other language's module/library system, it's on the cusp of becoming
#1 in the world. [http://modulecounts.com/](http://modulecounts.com/)

------
euphemize
Maybe it's just me, but I find:

    
    
      d = {}
      for k, v in request.items():
          try:
              d[k] = int(v)
          except (TypeError, ValueError):
              d[k] = None
    
    

much easier to read and understand than:

    
    
      walk_values(silent(int), request)
    
    

I need to find our what the "silent" attribute does and I'm not sure what type
of errors are caught here. Although I'm all for hiding implementation when
it's useful, I find it counter-intuitive to do it at this level in python, it
makes things harder to read.

~~~
tel
I feel like this is why types have such a benefit in Haskell. This would be
typed

    
    
        walkValues :: (v -> a) -> Map k v -> Map k a
        silent     :: (inp -> Either e a) -> (inp -> Maybe a)
    

Here, `silent` is the contextualized version of a really standard error-
ignoring combinator which simply throws away the additional error details
conveyed in `Either`.

    
    
        silent'    :: Either e a -> Maybe a
    

`walkValues` is an extraordinarily common function usually called `fmap` and
its existence tells us that `Map` is a `Functor`

    
    
        instance Functor (Map k) where
          fmap = walkValues
    

Ultimately, this pattern is really common and would be written in Haskell as

    
    
        fmap (silent' . int)
    

The types ensure it cannot be used incorrect. The notion of `Functor` happy
compresses all ideas of "mapping" such as what walk_values is doing.

~~~
tinco
I love Haskell, and I totally agree with you, but I don't think you're making
much of a case to people who don't already know Haskell. I'm pretty sure
everyone who doesn't grok Haskell reads your post and thinks "Ah there's
another Haskell weirdo spouting some weird syntax incomprehensible code."

So for them: What tel is saying is that in Haskell you annotate your functions
with type information (and if you don't the compiler will generate the info
for you), which gives you an immediate understanding of what moves into and
out of a function, often making it easier to understand what a high level
function does.

~~~
tel
Additionally, the reason why we have weird typeclasses is that they cut such a
wide swath of abstraction that you usually can recognize common functions for
various types and minimize the number of things you must memorize.

~~~
WaxProlix
It's so simple! Thanks, tel!

------
yummyfajitas
I don't disagree with the main point, but Python already has quite a bit of
this. A more pythonic version of the first example:

    
    
        @retry(DOWNLOAD_TRIES)
        def download(url):
            ...
    
        images = [download(u) for u in urls]
    

To check if a sequence x is ascending, you can use standard library functions:

    
    
        all( t[1] > t[0] for t in zip(x[0:-1], x[1:]) )

~~~
Suor
Just

    
    
        zip(seq, seq[1:])
    

Won't work with iterator though.

~~~
andreasvc
itertools.islice would work.

~~~
Suor
No it won't, unless you `tee` an iterator first.

~~~
e12e
FYI, the official documentation of itertools has its very own pairwise
example:

    
    
        def pairwise(iterable):
          "s -> (s0,s1), (s1,s2), (s2, s3), ..."
          a, b = tee(iterable)
          next(b, None)
          return izip(a, b)
    

[https://docs.python.org/2/library/itertools.html#itertools.i...](https://docs.python.org/2/library/itertools.html#itertools.izip)

~~~
baq
for the record: the official itertools docs have a lot more examples than this
one and they're all really worth knowing about if not studying.

------
zwieback
What this article also nicely illustrates is the tradeoff between readability
and compactness. As a generally experienced programmer I can easily read the
original versions and I can also easily imagine how I would make similar
improvements in a language I'm very familiar with. The condensed Python
version is only comprehensible to me after reading the rest of the article.

Is there value in code that's readable by non-experts? Sometimes yes, more
often no.

~~~
klibertp
> tradeoff between readability and compactness

What do you mean by "readability" exactly? Can you provide some kind of
definition for what "readability" is? Can you point me in a direction of _any_
research on "readability" (in programming) and it's impact on programmers'
productivity?

Sorry for the OT, but this is a pet peeve of mine: we have no way of defining,
let alone measuring "readability" in any meaningful way, yet people, including
even experienced programmers like you, talk about "readability" as if they
knew what it is, how it looks like and as if it was some universal feature of
physical world like force or speed.

~~~
JackC
> Can you provide some kind of definition for what "readability" is?

Sure. In this particular case, your parent considers an algorithm "readable"
if it can be readily understood by someone who is _not_ familiar with the
particular language being used, and is _not_ experienced with Lisp-style
languages, but _is_ experienced with ALGOL-style languages.

That's not a universal definition, but at this moment in this path-dependent
world it's a reasonable thing to consider.

~~~
baddox
I think in most cases a working programmer should be able to write code under
the assumption that it will be read and maintained by people who are familiar
with the programming language being used.

~~~
klibertp
I think you missed GP point, which I believe was that "you can write Fortran
in any language". Or more precisely that you can know a language very well,
but that won't help you much if you are confronted with unfamiliar idioms and
patterns you don't know.

------
noelwelsh
Interesting that all the examples were monads. There is another level of
abstractions waiting behind the abstractions in Underscore.js.

------
cbp
Heh, might as well just use another language at this time. Guido himself
seemed to be very pissed off at the common lispers for being forced to add
some functional language features. You can see it from the crappy
implementations of single-arity lambdas, non lazy map, filter, reduce, etc.
The language will never catch up to the times, as evident in the lame feature
list in python 3. Most python users seem blissfuly ignorant of modern
languages and Guido doesn't seem to like learning new stuff either.

~~~
wging
I'm also out of love with python for similar reasons, but please note that map
is lazy in python 3.

    
    
        >>> map(lambda x: x*2, range(100000000000000000000))
        <map object at 0x104792240>

~~~
Suor
It's not lazy, it returns iterator. There is a difference, you can't slice or
index its result anymore. Compare to Clojure or Haskell map, which are truly
lazy.

~~~
judk
Is it true that this is O(1) in Haskell? (Making up function names but you get
the idea)

    
    
        (fmap a function anArray) `get` n
    

?

How does fmap construct its value?

~~~
codygman
How fmap constructs it's value depends on how that type implements the functor
typeclass I believe.

------
buster
I think the biggest point why python does NOT need underscore is that python
has pip and for almost every problem exists a simpler helper library.

Probably _the_ reason for underscore is that it can be distributed via CDNs
and cached by the browser. This reason does not exist for other languages. If
i want to retry stuff i can use a retry library, easily downloaded via pip.
Same as i use argtools because argparse sucks. Same for database access, etc.
etc.

------
jimmaswell
In all of those python examples the non-underscore code was much more easy to
read. We can save the extreme compactness for the IOCCC.

~~~
Suor
It's easy cause it's familiar. But it's not simple.

This excellent talk by Rich Hickey explains the difference
[http://www.infoq.com/presentations/Simple-Made-
Easy](http://www.infoq.com/presentations/Simple-Made-Easy)

------
ASneakyFox
I feel like under score just makes your code compact. They're also ignoring
that I might want to be doing input validation while iterating over those
lists. Forcing me to add a loop right after that very compact code and
nullifying it only purpose. Then on top of that ill need to add comments to
explain the compact and cryptic code.

I really don't see the benefit.

~~~
Zak
You don't have to add a loop to do your validation:

    
    
        if not all(is_valid, the_values):
            inform_user("You supplied bad data")
    

Or perhaps

    
    
        bad_values = select(is_not_valid, the_values)
        if some(bad_values):
            inform_user("The following values were bad", bad_values)
    

Yes, sometimes you might want to do several things in a single iteration for
reasons of efficiency, but in that case you can compose() the functions
representing the per-element operations.

This code is only "cryptic" to people not used to functional programming. If
the intended audience is the set of everyone who knows Python, using these
abstractions will reduce the number of people who understand the code. If it's
your startup's team of three programmers, there's a good chance the time saved
is worth the learning curve.

------
heydenberk
Strikes me as more of an argument about why Python doesn't need an Underscore
library, given its expressive syntax.

~~~
Suor
It doesn't need direct port. But it can take advantage borrowing ideas or
library design principles.

~~~
heydenberk
I won't belabor the point, but I think these concepts in Python (and Ruby)
influenced the design of Underscore, not vice versa.

~~~
tel
Or that they all took influence from a bunch of languages from the 70s like
Lisps and MLs

------
noir_lord
No idea how but I'd managed to miss underscore.js up to this point (this is
one of the things that drives me crazy when I have to move into a domain that
is adjacent to mine I never know if I'm doing things the best way).

This is going to make things much simpler for me working on the client side.

Nice!

~~~
mmahemoff
You may want to check out Lo-Dash too
([http://lodash.com/](http://lodash.com/)). It's a drop-in replacement for
Underscore with some extra capabilities and claims to be faster and more
reliable. It's become a major project in its own right now.

~~~
spyder
Also there is lazy.js which is even more faster than lo-dash especially when
chaining multiple methods.
[http://danieltao.com/lazy.js/](http://danieltao.com/lazy.js/)

------
doublerebel
Debugging underscore's endlessly created anonymous functions can be quite
painful, leading to huge, nondescript stacktraces and sometimes straight
crashing the debugger. I have ripped it out of multiple projects and found
performance gains and much better readability. It seems like a great idea but
in practice native features are usually better. Only in certain browser cases
are certain underscore methods useful.

~~~
gravity13
On the flipside - I've never had to suspect _ for being the cause of a
problem.

That's a huge benefit. I actually spend less time debugging. I've reduced my
program to data manipulation strategies. If I get something wrong, it's
because I did my logic wrong, not because I typoed in a nested for loop
somewhere.

In fact, when I happen upon code that isn't using underscore, it tends to be
really long, loaded with unnecessary state and memos, and is very fragile and
not easy to reorganize. One of the first things I do is start utilizing
underscore. Once you get used to the extended language, you can do things like
convert 200 lines of code down into a few lines. It frees your mind up to
think about other things (which, for me, is the single most important thing in
programming).

I suggest learning to love the higher order data flow abstractions.

Edit: I also suggest you utilize breakpoints ability to backwards through the
call-stack (at least in DevTools), within the 'Breakpoints' tab, when paused,
you will see a trace which you can click through, and be plopped into the
scope of that execution as if you had a breakpoint anywhere along the chain of
function calls. Even if the function is anonymous, you're interacting with it
and so it shouldn't matter as much.

------
h8liu
IMHO, if the program logic is essentially to loop over some steps, it had
better be written in the form of a loop over some steps.

~~~
Suor
OTOH, if the program logic is mapping a sequence, then you better use map() on
the sequence.

~~~
h8liu
Sure, I agree. If the "map" verb is in your vocabulary, and it is the very
right verb to use, why not?

Just, if possible, please make sure it is easy to find out how the map() is
actually implemented (like please don't hide it deep in a language runtime).

------
foz
Although underscore.js is more widely-used, I think sugarjs.com is an
improvement and makes the syntax much nicer.

------
j2kun
It would help to give an explanation of what underscore is besides saying it
"makes life better." I am an avid python user and have never heard of
Underscore.js. Thought this was an article about semantic underscores in
Python, or underscores as used in variable names (which I dislike) :)

~~~
wging
It's a utility library for JS. Consider that the article mentions Python's
itertools and functools as equivalents.
[http://underscorejs.org/](http://underscorejs.org/) does a good job:

"""

Underscore is a JavaScript library that provides a whole mess of useful
functional programming helpers without extending any built-in objects. It’s
the answer to the question: “If I sit down in front of a blank HTML page, and
want to start being productive immediately, what do I need?” … and the tie to
go along with jQuery's tux and Backbone's suspenders.

Underscore provides 80-odd functions that support both the usual functional
suspects: map, select, invoke — as well as more specialized helpers: function
binding, javascript templating, deep equality testing, and so on. It delegates
to built-in functions, if present, so modern browsers will use the native
implementations of forEach, map, reduce, filter, every, some and indexOf.

"""

~~~
j2kun
Yeah, I figured that out. I just didn't like how the introduction assumed
everyone knows about it.

------
serkanyersen
Shameless plug:
[https://github.com/serkanyersen/underscore.py](https://github.com/serkanyersen/underscore.py)

Underscore.py is a direct port of underscore.js to python.

------
aikah

       s/underscore/lodash/g
    

Underscore,like Backbone feel like it is in maintenance mode.They didnt get
significant new features for years.People should better use lodash.

~~~
goblin89
> They didnt get significant new features for years.

[http://c2.com/cgi/wiki?FeatureCreep](http://c2.com/cgi/wiki?FeatureCreep)

------
Demiurge
I use underscore a lot in JS but never felt the need in Python. In the
examples given, I'd rather see explicitly what the exceptions being raised
are, or the comparisons.

------
onewland
For Java, I think Google Guava or Apache Commons are more of the de facto
standard library enhancements than the one linked in the article.

