

The Problem with Implicit Scoping in CoffeeScript - michaelty
http://lucumr.pocoo.org/2011/12/22/implicit-scoping-in-coffeescript/

======
mhansen
I'll copy below jashkenas' longer answer from the old github issue about this,
[https://github.com/jashkenas/coffee-
script/issues/712#issuec...](https://github.com/jashkenas/coffee-
script/issues/712#issuecomment-430673)

"""

Sorry, folks, but I'm afraid I disagree completely with this line of reasoning
-- let me explain why:

Making assignment and declaration two different "things" is a huge mistake. It
leads to the unexpected global problem in JavaScript, makes your code more
verbose, is a huge source of confusion for beginners who don't understand well
what the difference is, and is completely unnecessary in a language. As an
existence proof, Ruby gets along just fine without it.

However, if you're not used to having a language without declarations, it
seems scary, for the reasons outlined above: "what if someone uses my variable
at the top of the file?". In reality, it's not a problem. Only the local
variables in the current file can possibly be in scope, and well-factored code
has very few variables in the top-level scope -- and they're all things like
namespaces and class names, nothing that risks a clash.

And if they do clash, shadowing the variable is the wrong answer. It
completely prevents you from making use of the original value for the
remainder of the current scope. Shadowing doesn't fit well in languages with
closures-by-default ... if you've closed over that variable, then you should
always be able to refer to it.

The real solution to this is to keep your top-level scopes clean, and be aware
of what's in your lexical scope. If you're creating a variable that's actually
a different thing, you should give it a different name.

Closing as a wontfix, but this conversation is good to have on the record.

"""

~~~
the_mitsuhiko
> As an existence proof, Ruby gets along just fine without it.

Missing that Ruby stops scoping variables at a method and uses separate
lexical scoping rules for constants thereby avoiding this issue mostly.

~~~
cheald
Also, Ruby can shadow method names with local variable names just fine:

    
    
        class Foo
          def bar
            "bar"
          end
    
          def baz
            bar = "foo"
            puts bar
          end
    
          def bang
            puts bar
          end
        end
    
        foo = Foo.new
        foo.baz  # => "foo"
        foo.bang # => "bar"

~~~
showell30
Well, this is a case where the equivalent CS behaves exactly like Ruby.

    
    
      class Foo
        bar: -> 'bar'
        baz: ->
          bar = "foo"
          console.log bar
        bang: ->
          console.log @bar()
    
      foo = new Foo()
      foo.baz()  # => "foo"
      foo.bang() # => "bar"

~~~
cheald
Not quite; @bar is "this.bar", and is an unambiguous reference, not just
"bar". In Ruby, because self can be implicit, it's literally just "bar" in
both cases, and Ruby gives the local variable precedence over the method name.
You can still invoke the method via self.bar, but accessing the method as just
"bar" is shadowed by the variable. Additionally, even though the method may be
invoked by referencing bar, you can't redefine the method by assigning a new
method to bar. Neither Javascript nor Coffeescript has that behavior.

Here's a better example:

Ruby:

    
    
        def bar
          "I called bar!"
        end
    
        def foo
          puts bar
          bar = "I manually assigned bar"
          return bar
        end
    
        puts foo()
        puts bar()
    
        # =>
    
        I called bar!
        I manually assigned bar
        I called bar!
    

Coffeescript:

    
    
        bar = ->
          "I called bar!"
    
        foo = ->
          console.log bar()
          bar = "I manually assigned bar!"
          return bar
    
        console.log foo()
        console.log bar()
    
        # =>
    
        I called bar!
        I manually assigned bar!
        TypeError: string is not a function

~~~
showell30
When I said the two programs have the same behavior, I was simply referring to
the end result--what they output. I understand how both languages work here.
Ruby solves a problem that simply doesn't exist in CS. In JS/CS "bar" never
implicitly refers to "this.bar", so there's no ambiguity in the first place.

------
gcv
It's worth pointing out that JavaScript 1.7 resolves this mess by introducing
block scoping using the "let" keyword. It works just like it does in Scheme,
Common Lisp, and Clojure (i.e., correctly). Not supported in anything except
Firefox, unfortunately.

~~~
swannodette
... and Standard ML, OCaml, Haskell, Smalltalk, etc

~~~
draegtun
... and Perl except its called _my_ instead of _let_

------
rayiner
Scheme got this right in 1970. There is no excuse to design a new language
this way.

~~~
tikhonj
I agree with this wholeheartedly--Scheme use an extremely simple scoping model
that is nonetheless more expressive than Python (before 3, I guess) and
CoffeeScript's. In fact, I only really completely understood JavaScript's
model--and realized that, even if a little awkward, it was fundamentally
elegant--after writing a Scheme interpreter.

~~~
wisty
I think Python's is deliberately unexpressive, forcing you to use local
variables pretty much everywhere. This tends to decouple stuff, which is
_usually_ good. The model is "everything you do is local, unless you know
better, and want to jump through a lot of hoops". Coffeescript works a similar
way.

Javascript seems to have the model "everything you do is global, unless you
know better".

I'm not a big Javascript hater, but this is one very sore point.

~~~
swannodette
_"everything you do is global, unless you know better"_

That is not the JavaScript model at all.

~~~
lnanek
He is talking about if you don't do anything extra (in this case, use the var
keyword). Much like the nonlocal keyword in Python requires knowing about it
and wanting to use it in order to actually use it and get the non-default
behavior.

~~~
swannodette
var is not extra in JavaScript the way nonlocal is in Python. var is a
language construct that any good JavaScript program _must_ use.

------
limeblack
Another Issue:

Although the following examples could become unambiguous with parenthesis,
these examples demonstrates how a trivially overlooked ending delimiter
further complicated the language. Not only is the intent of the CoffeScript
code unclear in the examples below but the slight variation in the
CoffeScript, produces radically different output. The CoffeeScript differences
are so small it would be easy for someone to add accidentally while editing.
Anonymous function passing and function calling in Javascript require no
additional wrappers or edits, while in CoffeeScript you must add special case
clarity.

[http://img542.imageshack.us/img542/7379/coffeescripttojavasc...](http://img542.imageshack.us/img542/7379/coffeescripttojavascrip.png)

------
oinksoft
_@mitsuhiko Not gonna happen ;) Forbidding shadowing altogether is a huge win,
and a huge conceptual simplification._

How arrogant! You'd think he'd step back for a second and consider the
suggestion, but it sounds like he's on autopilot.

~~~
equark
Don't jump the gun. The reason is that he's already had many discussions about
this and has formed an opinion.

See: <https://github.com/jashkenas/coffee-script/issues/712>
<https://github.com/jashkenas/coffee-script/issues/238>

Whether he's right is another question. But if you don't like his decision you
can use the Coco (<https://github.com/satyr/coco>) fork which fixes it by
introducing := for nonlocal assignment.

~~~
barrkel
The reasoning basically seems to be "if you misuse this feature (i.e. have
top-level symbols), you'll get bugs (like the OP discovered), therefore don't
misuse this feature (i.e. keep your top scopes clean), and therefore you won't
get bugs, and therefore it's not a problem with CoffeeScript".

It shouldn't take a lot of thought to see why this is a somewhat user-hostile,
passive-aggressive approach for a language. If something is ill-advised, the
language should actively steer you away from it, not dissuade you with subtle
bugs a few thousand lines of code down the road.

I think this scheme would be more workable if sigils or similar conventions of
some kind were mandatory for top-level symbols. Then it would be much harder
to accidentally wander into this problem. The language he seems to be drawing
inspiration from, Ruby, does do this.

~~~
jashkenas
I'm afraid that's not quite the reasoning...

Like everything in languages (or APIs), there's a tradeoff here. By making
scoping automatic, and (hopefully) forbidding shadowing, you can make the
language conceptually simpler. Think of it as making variables be
"referentially transparent" in terms of their lexical scope. Everywhere you
see "A" within a given lexical scope -- you _know_ that "A" always refers to
the same thing. In a language with "var" and with shadowing, "A" could mean
many different things within any given lexical scope, and you have to hunt for
the nearest declaration to tell which one it is.

On the downside, you have what Armin describes: If you happen to try to use
the same name for two different things _within the same lexical scope_ , it
won't work.

Since it's always the case that you are able to choose a more descriptive name
for your variable, and gain clearer code by it, I think it's very much a
tradeoff worth making.

~~~
barrkel
You haven't said anything new to me here; I took all this for granted and it
isn't what I'm criticizing. You haven't actually addressed my summary of the
apparent reasoning for why accidental assignments to top-level symbols isn't a
pathology of CoffeeScript. The bit I'm criticizing is the _failure mode_ of
"if you happen to try to use the same name for two different things within the
same lexical scope, it won't work".

~~~
jashkenas
I see. The reason I can't address your "failure mode" with an explicit
suggestion is because it isn't a failure mode in CoffeeScript: If you assign a
new value to a variable in an inner scope, the variable now has a new value.
If you're thinking "I want to use the same name for two different variables"
... the answer is: choose a different (better) name for one of them.

But perhaps I'm still not getting at the answer you're looking for here...

~~~
barrkel
You keep stating and restating how and why CoffeeScript doesn't support
shadowing of symbols via lexical scoping.

That's not what I'm criticizing. Lexical shadowing is not what I'm advocating.
I think your choice is fine in so far as it goes. It has its logic.

What I am criticizing is how it can fail. The top scope is different,
quantitatively and qualitatively, from almost all nested scopes. It's much
larger, and spread lexically over a larger area. If you have a team of
developers, it will be modified concurrently. No one developer necessarily
knows the full set of symbols defined in the top scope while they are writing
an individual procedure.

And thus the problem: a developer thinks they've chosen a "different (better)
name" for a some variable, but in fact they've chosen one that a different
developer _also_ thought was a "different (better) name", only one of them is
in a lexically enclosing scope. This problem isn't likely to occur on the
level of nested procedures or nested blocks, because the definitions would be
visually close. But it's much more likely to happen when one of the symbols is
defined in the top scope. Here, the definition could be many hundreds or
thousands of lines away. It may even be in a separate commit, waiting to be
merged, such that there's no way for either developer to know without closely
reviewing every change.

And this is the criticism: the failure mode for this inadvertent reuse of a
variable name is subtle bugs, as what one developer thought was a global
symbol turns out to be modified and acquire strange values through unexpected
codeflow, almost like the VM was corrupted and memory was behaving unreliably.

The qualitative difference of the top scope in situations like this is the
reason why I suggested sigils or somesuch to disambiguate those scenarios.
Perhaps top-level symbols can't be reassigned from nested scopes unless you
use '$' as a prefix to their name; a visual shorthand that you are definitely
not creating a new local symbol.

The reason I summarized your argument in the way I did is because your
argument against this failure mode seems to be "don't create top scopes with
lots of symbols". That's a fine argument (or rather, exhortation), but it
isn't a realistic one. If the language is problematic with lots of symbols in
the top scope, it should be unpleasant to use with lots of symbols in the top
scope. And the unpleasantness shouldn't come from subtle bugs (the passive
aggressiveness I mentioned); it should come from awkward and ugly sigils, or
some other intrinsic way of discouraging those styles.

~~~
showell30
barrkel, I think you should create a new programming language with more
sensible scoping than CoffeeScript, so that developers can create bug-free
code while working in parallel on thousand-line codebases without close
reviews. I would start by using "$" as a sigil to prevent top-level variables
from being reassigned in nested scopes. It would be a visual shorthand that
indicates that you are definitely not creating a new local symbol. This would
prevent developers from needing to know the full set of symbols in the top
scope. In fact, by eliminating the inadvertent reuse of variable names, you
would eliminate the failure mode whereby what one developer thought to be a
global symbol actually turned out to be modified. Eliminating this unexpected
codeflow would be tantamount to eliminating memory corruptions in VM.

You should definitely go down this track. It would be a massive achievement.

~~~
barrkel
I think you think you're being sarcastic. But I do actually work for a
developer tools company, and I implement language features on a continuous
basis. I implemented anonymous methods - closures - for Delphi, a task that
involves no small amount of scope wrangling; and FWIW, Delphi almost certainly
has a larger userbase than CoffeeScript, albeit one with greyer hairs. While
my specialism is in statically typed, compiled languages, issues around
lexically nested scope are pretty much the same as in dynamic languages.

If you want to have a constructive conversation, you could try dialing back
the snark, and addressing my arguments directly.

~~~
showell30
Ok, I'll dial back the snark and just be blunt. I think all your concerns
about CoffeeScript are hypothetical exaggerations. I don't think you've
written much CoffeeScript at all, and I suspect that if you did, you'd quickly
see that your concerns are overblown. In theory, you could have thousands of
lines in a single file, and two commits from a separate branch could cause an
undetectable accidental naming collision. In practice, this rarely happens.
Files tend to be smaller, merge changes do to tend to get scrutiny, and name
collisions often have fairly obvious symptoms once you run your tests on the
code.

~~~
barrkel
I think if you have good practices - small files, scrutiny of changes, unit
tests - you can make _any_ language work. The test of a language comes in how
it bites you when you stray. So to be blunt, I think your defense is
irrelevant.

------
stoodder
Why not allow CoffeeScript to use the 'var' keyword, explicitly telling CS
that this variable should be scoped locally even though it may be shadowing
another variable? This seems consistent with their approach of allowing
(although optional) native javascript syntax such as {}, and []. This still
allows CS to stick to it's paradigm of forbidding shadowing unless we
explicitly state that we know what we're doing.

~~~
jashkenas
Because we're aiming for a conceptual simplification.

Pretend like you're a beginner, learning this stuff for the first time. If
everywhere you see a variable "A", within a certain lexical scope, it means
the same thing ... that's much simpler to understand than if "A" _means_ three
different things at three different places, because you happened to shadow it
twice.

~~~
stoodder
Yea, I completely get what you're saying and for the most part agree. To be
frank, I think the author of the article made an error in deconstructing on
the 'Math' object (at least at a global scope). 'Math' provides a namespace
for all of its methods and a similar approach should be taken to other
libraries/pieces of code. I also agree that keeping things simple and straight
forward makes sense, but you're doing it by forcing one to abide by those
standards although someone might have completely legitimate reasons for
explicitly scoping their variables.

Either way, I think it is what it is and the benefits of CS very much outweigh
the cons. Thanks for the feedback

------
tjholowaychuk
I like the look of coffeescript's assignment better, but I cant help but think
"let" is much less ambiguous, once you see it you look no further. This
reminds me a bit of Ruby, where "foo" could be a function or variable
potentially from anywhere so it's a little unclear although better looking.

------
leafo
Look at how MoonScript handles this:
[http://moonscript.org/reference/#the_using_clause_controllin...](http://moonscript.org/reference/#the_using_clause_controlling_destructive_assignment)

I didn't want to change the default semantics, but I wanted to have a way for
the programmer to be safe if they wanted to, so I created the `using` keyword
for function declarations.

You explicitly declare what you intend to overwrite in the lexical scope,
including overwriting nothing at all with `using nil`.

------
buddydvd
I found one of the referenced links in Github issue #712 quite interesting:

<http://www.rubyist.net/~matz/slides/rc2003/mgp00010.html>

Source: [https://github.com/jashkenas/coffee-
script/issues/712#issuec...](https://github.com/jashkenas/coffee-
script/issues/712#issuecomment-979127)

~~~
showell30
These links explain the thought process behind CS's current behavior:

[https://github.com/jashkenas/coffee-
script/issues/712#issuec...](https://github.com/jashkenas/coffee-
script/issues/712#issuecomment-430673)

[https://github.com/jashkenas/coffee-
script/issues/712#issuec...](https://github.com/jashkenas/coffee-
script/issues/712#issuecomment-760853)

------
gerggerg
_Considering we won't see this changed since the author has already closed the
issue and expressed his satisfaction with the current rules this article
should at least serve as a reminder for errors not to repeat with the next
language someone designs._

It's open source. Why not fork it and get some like minded coders to change it
with you?

~~~
scribu
There already is a fork that changes this (Coco), already mentioned in another
comment: <http://news.ycombinator.org/item?id=3380423>

------
danmaz74
With "var" and shadowing you can still shoot yourself in the foot, it's just
the other foot.

If you need global variables, it's sensible to just adopt a simple naming
convention, like prepending g_ (or whatever pleases you) to all your
variables. I already did that with plain JS and it's well worth the "effort".

------
latchkey
Kind of a side note to the posting, but I just have to say: Please make your
usage of parens consistent. If you aren't going to use them, don't use them
everywhere.

Here is an example of what I'm talking about:

if isAir cx, cy, cz + 1 then addPlane('near', block)

Should be:

if isAir cx, cy, cz + 1 then addPlane 'near', block

Personally, I use them everywhere because I like having the stronger visual
clue that this is a method I'm calling. I think making them optional in CS was
a bad idea.

if isAir(cx, cy, cz + 1) then addPlane('near', block)

imho, so much more readable.

~~~
MatthewPhillips
Not allowing parens would mess up the usage of anonymous functions.

~~~
latchkey
Not sure I understand you or maybe you don't understand me. I was talking
about using parens for method/functional calls, not getting rid of them from
CS.

~~~
MatthewPhillips
I understand. If you don't have parens on method calls, how could you use an
anonymous function that is not the last parameter? For example:

    
    
      doSomething(->
        # stuff happens
      ), onerror

~~~
jacobolus
First, your code snippet is wrong: you need a space before your first paren.

In coffeescript you can do any of the following, or a few other variants:

    
    
      doSomething ->
          stuff
        , onerror
    
      doSomething(->
          stuff
        , onerror
      )
    
      doSomething (->
          stuff
        ), onerror
    
      doSomething(
        ->
          stuff
        onerror
      )
    
      doSomething (-> stuff), onerror

------
showell30
CoffeeScript's approach toward top-level variables is quite elegant and
simple. When you declare a variable at top-level scope, it is equally
available to all code within that file for both reading and writing, with no
strange "nonlocal" or ":=" syntax to complicate manners.

Once you understand the reach of CoffeeScript's top-level variables, it is
easy to write bug-free code. Since you know that top-level variables have wide
scope, you simply need to be judicious about putting variables at top-level
scope. If a variable is not needed at top level scope, don't put it there.

~~~
the_mitsuhiko
Way to miss the issue. I was not suggesting ":=" for global variables at all.
I was suggesting ":=" as a replacement for the nonlocal keyword that Python 3
has to solve the issue I was demonstrating.

~~~
showell30
Nope, I get it, you were suggesting ":=" as a replacement for the "nonlocal"
keyword in Python. I'm not sure how my comments demonstrate any lack of
reading comprehension. All I said about ":=" and "nonlocal" is that they are
overly complicated once you accept that top-level variables can have wide
scope.

Obviously, plenty of folks managed to write bug-free code in Python before
"nonlocal" was invented. I'm not saying it's a bad idea, but you can avoid
bugs without it.

~~~
the_mitsuhiko
> Obviously, plenty of folks managed to write bug-free code in Python before
> "nonlocal" was invented.

Python has the inverse behavior of CoffeeScript. So that was never an issue.

~~~
showell30
Well, fine, but Python2 had the inverse problem--you couldn't mutate top-level
variables without awkward "global" statements.

~~~
notSorella
That's a whole different matter, which can't make you shoot yourself in the
foot without knowing it for any non-trivial code.

CoffeeScript's scoping _forces_ you to always keep track of whatever is
enclosing the current scope ALL THE WAY TO THE TOP. This is way too much when
your function doesn't need to access outer variables (which should be the
minority of the cases).

So, problem is, you either make all your functions have non-free variables
(but Jashkenas seems to dislike functions shadowing outer variables too, which
is just... overtly weird), or you keep track of all variables above the
current scope.

The former is not too unreasonable, until you remember it makes no sense with
closures :3

~~~
Avshalom
The problem is actually worse. You _also_ have to keep track of all the
variables BELOW the current scope as well so you don't accidentally (as in the
case of the article) turn a declaration into a reassignment.

~~~
tjholowaychuk
yeah that's a sketch thing to have to worry about :s

------
perfunctory
This is my biggest problem with CoffeeScript. And the author stubbornly
refuses to fix it. Apparently it's some sort of Ruby religion.

------
shaunxcode
Here is a ghetto "let" form in coffee

((a = 5, b = 6, log = x -> console.log x) -> log a + b)()

------
cvshepherd
> The simple solution is to either add a nonlocal keyword like Python has or
> to introduce a := parameter that works like = but explicitly overrides a
> higher level variable.

I disagree. The simple solution to this is to write tests.

~~~
Deestan
> I disagree. The simple solution to this is to write tests.

That's a step backwards. Code error checking should be done as early as
possible. In order of earliness:

* Typing in the code. (Ideal: it is clear from the syntax that the code performs X instead of Y.)

* Compiling. (Strong type checking ensures you cannot return 5.3e7 or null from GetHostName.)

* Running the code at all. (Code contracts and assertions trigger if GetHostName returns "".)

* Automated unit tests. (Check that DB.GetHostName() returns the same string given to DB.Connect().)

* Automated integration tests. (Check that the DB module can connect to and retrieve useful data from a dummy database.)

* QA ("Hey Joe, the system hangs when I give "¤;\@" as my username and press the connect button rapidly for a few seconds.")

* Customer ("Hi the system has a problem, please fix.")

The further up, the faster, more accurately and with less "noise" the error
can be discovered.

------
showell30
FWIW this is how CS works:

    
    
      top_level_variable = null
    
      f = ->
        top_level_variable = "hello"
    
      f() 
      console.log top_level_variable # prints hello

~~~
cheald
The concern is that because Coffeescript automatically scopes variables to the
scope of their first reference, it can introduce maintenance issues. Consider
the following:

    
    
        foo = ->
          bar = "woot!"
          console.log bar
    

This compiles to:

    
    
        var foo;
        foo = function() {
          var bar;
          bar = "woot!";
          return console.log(bar);
        };
    

bar is locally scoped to foo(). Now, 2 weeks later and 200 lines earlier, you
come along and define:

    
    
        bar = ->
          alert "Holy crap cheese is awesome!"
    

Which compiles to:

    
    
        var bar, foo;
        bar = function() {
          return alert("Holy crap cheese is awesome!");
        };
        foo = function() {
          bar = "woot!";
          return console.log(bar);
        };
    

Now, all of a sudden, the "bar" reference in foo isn't scoped to foo()
anymore, it's scoped globally, and once you invoke foo(), it'll replace the
function bar with a string, potentially breaking your app. It's an ease-of-
maintenance issue.

This isn't consistent behavior, though. If you define your top-level bar()
function _after_ foo, like so:

    
    
        foo = ->
          bar = "woot!"
          console.log bar
    
        bar = ->
          alert "Holy crap cheese is awesome!"
    

Then you get "correct" scoping (and the outer bar is shadowed):

    
    
        var bar, foo;
        foo = function() {
          var bar;
          bar = "woot!";
          return console.log(bar);
        };
        bar = function() {
          return alert("Holy crap cheese is awesome!");
        };
    

On one hand, it could be argued that this is a "name things better" problem,
but on the other, I have to agree that it'd be nice to be able to explicitly
scope things when needed. Given that the behaviors are divergent based on what
order the variables appear in, I'd say it's confusing enough that a way to
explicitly say "hey, I know what I'm doing, I want to shadow any outer
variables and declare local scope here" would be useful.

~~~
boundlessdreamz
Thanks for this! Way better explanation than the blog post :)

Is there anyway to make the scope explicit in coffeescript?

~~~
yxhuvud
Yes, create the variable as a function argument.

~~~
cheald
While that does allow you to use a locally scoped variable (even one that
shadows an outer variable), it's still not explicit scoping.

