
Your Language Features Are My Libraries - boilerbill
http://blog.bitquabit.com/2009/05/20/your-language-features-are-my-libraries/
======
arakyd
Here's my observation: People buy products with features, not products that
let you build features. Programmers are no different. Most of them will choose
a language that does x over a language that can easily be extended to do x,
even if the library that provides x has already been written. The fact that
their favorite language doesn't do y and z doesn't matter because they don't
know about y and z, and the fact that their favorite language prevents them
from inventing alpha and beta doesn't matter because they think in terms of
using language features, not extending the language. They want something they
can use without thinking very much about it. They don't want to change it.

Of course it is hard to do programming without doing any metaprogramming at
all, so new languages, like C#, have some basic metaprogramming features. But
you have to market them as features of an otherwise fixed programming language
or you'll scare people off. It's all about minimizing the number of things
your customer has to think about. It's even worse if management is driving the
choice/design of the language.

~~~
dan_the_welder
Is there a language other than basic that you can actually use without
libraries?

(edit)

Without eventually making your own?

~~~
randallsquared
Well, some languages have so much apparent power that it's easy to be seduced
into thinking you hardly need libraries, since you can just build it yourself
right quick. Common Lisp, for example.

~~~
gaius
But then you do need a library and run slam into a brick wall and realize you
should have just used Python...

------
jrockway
This is another thing that is nice about languages with simple syntax (like
Lisp and Smalltalk) -- language features and libraries can "look" exactly the
same. I think this is one of the big mental barriers to entry for users of
other languages. If you use Java or something, adding something like a
langauge feature would just be so difficult that you wouldn't even try -- you
would use an XML file or something. With Lisp or Smalltalk, adding something
that looks like a language feature is so easy, that you wouldn't ever think of
doing something else. As a result, Lisp and Smalltalk have a lot of languge
features ("loop") that really aren't.

From the standpoint of an editor hacker, I love simple languages. When I write
Lisp, and create some "new construct", I just follow a few simple rules and my
editor can understand the code perfectly. (Name your definition-creation
macros define-something-or-other, and emacs will syntax highlight them
correctly. Perfect.) When I am extending a more complicated language, the
editor hackery is much more involved.

A bit of backstory, Perl now has a module called Devel::Declare, which lets
you hook the parsing machinery and run your own code when you see certain
things; this basically adds macros to Perl. "It's not a source filter." A
number of people have taken advantage of this newfound extensibility to add
some great features; try/catch syntax, class definition syntax, etc. I became
a heavy user of this stuff, and I wanted emacs to understand it. 3 days of
hacking and 200 lines of code later, it does. But that's a lot of effort to go
to. When extending Lisp, I had to do nothing. When extending Perl, I had to
hack on my editor for a few days to get everything right. (And don't ask,
"what happens if you don't use the module that introduces the syntax; does
emacs detect that?" because you won't like the answer. But, try typing
"define-thing-that-is-not-defined", and emacs will syntax-highlight that as
though it's a builtin, and not as random garbage you just entered. Oh well :)

I think this must be a mental barrier for those who are not comfortable with
both the internals of their language and with the internals of their editor,
and that's why users rely on The Language Designer Gods to give them features.

~~~
stcredzero
>I think this must be a mental barrier for those who are not comfortable with
both the internals of their language and with the internals of their editor,
and that's why users rely on The Language Designer Gods to give them features.

But programmers who have that mentality probably _should_ wait for The
Language Designer Gods to give them features. These are the same people who
should not be allowed to write frameworks and should not be allowed CPP macros
and the use of #doesNotUnderstand: in Smalltalk.

~~~
jrockway
Maybe. I think if you give someone a tool, they will eventually learn how to
use it. If you give them the possibility to make a tool, though, they might
not ever figure out how to make the tool. There are two different levels of
thinking involved; more people can do the former than can do the latter. Some
people can do neither, though, I will give you that :)

------
req2
I think one of the benefits of features, rather than libraries, is that
someone can be expected to know what it does.

In C#, ?? should be understood by anyone who knows C#. In Smalltalk, ?? will
be understood by the implementer and then need to be explained or learned by
every fresh set of eyes. (Yes, they might expect it mirrors C#, but the
problem still exists. The eyes know Smalltalk, not C#.)

It is similar to the situation in
[http://www.37signals.com/svn/posts/1729-coding-style-
helpers...](http://www.37signals.com/svn/posts/1729-coding-style-helpers-that-
return-params). link_to, easily understood, rss_link, not quite sure.

~~~
stcredzero
In Smalltalk, if you're curious about ??, you'd be able to find it instantly,
and read the method. As you know from the article, it's fairly short and would
be obvious to a competent Smalltalker in seconds. If that's not enough just
take a second to type in

    
    
        nil ?? [ self halt ]
    

And right-click "Debug-it." Debugging in VisualWorks is so painless, people
actually write code in comments for people to understand by debugging -- and
people will even follow it while in another debugger session! (And even doing
that a 2nd, 3rd time is just as easy!) To heck with comments or API docs, you
can see how everything works and tinker with it!

Your division between access by the implementer vs. the "fresh set of eyes" is
a misconception you're taking from C#. There is _nothing_ in Smalltalk that
demarcates the "implementer" from the regular programmer. It's "Turtles all
the way down!" You can actually implement a debugger as an ordinary Smalltalk
app, and in 5 minutes, you can be browsing a stack. There is little difference
between programming and meta-programming.

 _In other words, Meta-programming is easy and natural, not esoteric._

You just see Smalltalk as a funny sort of Blub where no one knows what the
language/library will look like, because you see things in terms of the
programming you know: Blub. Take it from me, I've been to dozens of Smalltalk
shops over a decade: this is _not_ a problem. It's sometimes a huge advantage!
One can create very powerful domain-specific languages and tools this way.

------
dmnd
> Unlike with LINQ, if these operators did not exist in Smalltalk, it would be
> trivial to add them.

False. The SQL-like snippet of LINQ in the article is just syntactic sugar for

    
    
        people.Where(person => person.Type == PersonType.Adult && && person.Location != "New York")
    

Which is implemented via a static class extension method. C# doesn't ship with
a Zip method, but adding one is trivial:

    
    
        public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(this IEnumerable<TFirst> first, IEnumerable<TSecond> second, Func<TFirst, TSecond, TResult> resultSelector)
          using (IEnumerator<TFirst> e1 = first.GetEnumerator())
            using (IEnumerator<TSecond> e2 = second.GetEnumerator())
              while (e1.MoveNext() && e2.MoveNext())
                yield return resultSelector(e1.Current, e2.Current);
        }
    

That makes it a library feature of C#, not a language feature. Only the LINQ
syntactic sugar is a language feature, and I would argue the decision to
include such a syntax was a bad one.

~~~
gecko
Actually, still true. You're right that LINQ expressions are syntactic sugar
for C# calls, but that syntactic sugar depends on lambdas (introduced in C# 3)
and expression trees (also introduced in C# 3), among others. You cannot write
LINQ as you know it in C# 2. That's my point.

~~~
dmnd
Ok, but using the previous version of the language to demonstrate your point
is much less interesting. Additionally, there is the issue of clarity: when I
read "C#" I tend to assume that means the language as it exists today, not
some previous version.

~~~
gecko
You completely missed the point of the article, which was that _I can add
these features to the language without waiting on the language designer._
Microsoft added three key technologies to C# 3, one of which was syntactic
sugar for the other two. That's irrelevant to my point.

------
tolmasky
The central issue for me is that I don't like getting bogged down in the meta.
I hate the idea that a lot of what programmers do is program around
programming, as opposed to actually solving real world problems with their
programs. Of course the end goal is solving said problem, but you have to do a
lot of meta to get there. Its unfortunate how much of our libraries and so
forth are just pure architecture.

I like language features because they provide a clear distinction between the
meta (things that I have to do due to the limitations of my execution
environment) vs. the actual steps I'm taking to calculate some result or
provide some service. When I see a language keyword or operator its clear to
me that this is glue/implementation, and when I see a function call its clear
to me that this is an actual step related to the algorithm. Classes and
modules and templates and all these "objects" ideas have little to nothing to
do with algorithms, they are simply the way we choose to implement those
algorithms.

This varies language to language of course. Smalltalk has the advantage that I
can create library features that "look" like language features. This is a big
plus in my book. I think if I were using Smalltalk I would probably not have a
problem with this at all. However, taking an example like JavaScript, I very
much dislike this style (which many js frameworks currently use):

    
    
       classCreate("MyClass", mySuperClass, properties);
       //...
       cosine(angle);
    

In a cursory view, the actual work (calculating cosine), looks _exactly_ the
same as creating a class, there's no visual distinction. Additionally, this
encourages every library under the sun to make their own slightly different
version of the class create/subclass method which makes it difficult for IDE
writers to be able to pick up on these _semantic_ elements of your program,
since it really just is a function call and nothing more.

~~~
DougBTX
That classCreate syntax looks really bad, I've not seen it used before. A more
sane syntax would be...

    
    
        var MyClass = SuperClass.extend({ ... properties ... });
    

which has been working quite well in my current projects.

------
triplefox
I've been deep in the depths of hacking a new feature into an existing
language - adding a reflective/referential data definition language on top of
haXe. (It's called XReflect, if curious you can get it off haxelib)

Essentially, the problem I keep running into with referential data structures
is that haXe, like most popular languages, will only let you set them up in an
imperative fashion, one dependency at a time, which tends to both spread out
and bloat application code. What I want is to write my dependencies in any
order and let the resolution be sorted for me.

I'm pretty sure a language with AST access and quoting like Lisp makes it
relatively easy to write something along these lines, but in haXe, I had to
write about 500 dense lines of code, most of it going to parsing and fighting
the limitations of the type system. It still has some major bugs that I'm
trying to resolve with a refactor at the moment.

The funny thing is, I know I'm not just fighting haXe here, I'm also running
into the limitations of the platforms that haXe compiles to. The newest
strategy I'm applying to resolve references is to add a "__parent" field to
the reference stubs, which refers to the container(an array or record type) of
that reference. I use that later, after all references are resolved, to drop
the final value into the place of the stub, but as it turns out this causes
difficulty printing out the working structure. Neko will recognize the value
of __parent as "....", but Flash 9/10 will recurse into the field again and
get a stack overflow.

So I had to add more code to write a pretty-printer which is aware of such
traps.

~~~
cousin_it
Among other things, type systems make it much harder to go meta. The evolution
of ActionScript 2 into ActionScript 3 saddens me.

------
wvenable
The problem with languages where everything is possible syntactically is that
everything is possible syntactically. You're no longer dealing with a language
that everyone speaks and understands -- instead you have to learn to local
dialect just to get started. If the programmer before you was a total git,
you'll be in more trouble than if you'd use a more constrained language. This
is why most languages after C++ have avoided operator overloading.

~~~
mbrubeck
> This is why most languages after C++ have avoided operator overloading.

Except for C#, PHP, Python, Ruby, Perl, Haskell, Io, Lua, Scala, Groovy, D,
F#, and even Visual Basic.

Really the only major post-C++ languages to avoid operator overloading were
Java, JavaScript, and Objective-C.

I've seen an argument that Java learned the wrong lessons from C++. They saw
how complicated operator overloading was in C++ and avoided it, despite the
fact that the worst complications in came from mixed pointer/value/reference
calling conventions (which of course wouldn't affect Java). C# proves that
it's completely possible for a Java-like language to have safe and easy
operator overloading.

~~~
russell
And in Java you get such idiocies as == for primitives and equals for objects.
Would it have been so hard to translate == into a.equals(b)?

~~~
10ren
They have different meanings in Java

    
    
        a == b         // same object
        a.equals(b)    // same value

~~~
cousin_it
If Java had made primitives syntactically behave like immutable Objects with a
proper implementation of equals(), Russell's suggestion would've made obvious
sense and the language would've made more sense overall. The road not taken.

~~~
10ren
How does this overcome the need for two kinds of equality for objects? i.e.
whether two pointers refer to the same address vs. whether they point to
different addresses (or the same) that contain objects with the same value,
according to that object's definition of equals().

Do you mean that the present == for primitives should be replaced with
.equals(), and the == for object reference equality should be replaced by
perhaps === or .refEquality(), and then the .equals() for objects should be
left alone. and then, == be added as syntactic sugar for .equals().... Are
those assumptions what you meant?

If so, the same outcome can be achieved with by changing object == to ===, and
object .equals() to ==. However, there are reasons for treating primitives as
objects other than this.

I expect you've a model in mind from some other language that resolves this in
along different lines (C# would be one), and so you haven't felt the need to
articulate the assumptions needed to make it work. If I've guessed wrongly,
can you elaborate please?

~~~
cousin_it
The overwhelming majority of actual Java code needs at most one equality
operation defined per class. Hamming compression and convention suggest that
it be called == . Extensibility suggests that it desugar into .equals() or
something like that.

For immutable objects like ints or Strings the implementation of equals()
should compare internal values, because reference equality for passive
immutable things is practically never useful. This should already obviate the
majority of explicit calls to equals() in actual Java code, because most of
them are comparing Strings anyway.

For mutable objects the default implementation of equals() should compare
object identity (hashCode), but be overridable if the user wants something
exotic.

Algorithms and data structures relying on equality, like hash tables, should
provide the option of supplying the equality operator or hash function at
construction time, just like printf frees the user from the need to commit to
one Integer.toString() implementation in perpetuity.

This all sounds like a reasonable scheme to me. I can flesh it out in more
detail if you request.

~~~
10ren
Yes, thanks, that's what I thought. BTW couple minor points: hashCode doesn't
necessarily distinguish between objects (though most implementations would);
and an exception for mutable objects is collections (e.g. a Set is mutable,
but you want to be able to compare sets).

~~~
cousin_it
Yes, I'd want to compare sets, but using == for that feels to me a dubious
programming practice, akin to using + for set union. A special method like
hasIdenticalContents() would make me more comfortable, the longer name
removing equals-ambiguity for readers and reflecting the nontrivial
computation behind the scenes. And that would mesh nicely with other set
operations like difference and containment that would be getting non-canonical
names anyway...

Also, if you make Set.equals() depend on contents, you'll have to change the
implementation of hashCode as well, wreaking havoc on sets containing mutable
sets. So you really should only override equals() if your class is immutable
or you're feeling _especially_ exotic.

~~~
10ren
Your comment made no sense, then when I started to write a reply requesting
clarification, it refreshed, and you'd added more. I feel much better now that
it makes sense; and I'll watch out for the live-edit-comment in future (I do
live-edit myself too).

But I'll say that naming is tricky, since programming languages are really
human languages, and as such are affected by the familiarity of their specific
users as much (or, at times, more) than pure logic. Highly illogical, I know.

 _EDIT_ : The present implementation of Set.equals() depends on contents; it's
independent of hashCode, and there's no problem with sets of sets.

------
plesn
Languages are a way to communicating both to other humans and to your
compiler, and doing both well is really hard. Sometimes syntactic sugar makes
it easier for humans even in powerful and dense languages: think of the "do"
notation in Haskell.

And no, you can not put every language "feature" in a library: what about
features like static typing, isolation of side-effects... Even features like
threads can be a library, but the compiler has to know about this library and
its particular semantics...

So yeah, we all know languages like Lisp or Smalltalk are very expressive. But
current C# is also quite powerful and relatively pleasant to work in, and it's
goal is rather to bring more features to programmers and to evolve this
"blub". And of course I hope someday the average programmer will write in a
Haskell-like-but-even-better language :)

------
waldrews
[EDIT - this 1st paragraph is wrong and retracted]<wrong> The post misses a
large part of the point of LINQ to SQL, which is that it constructs optimized
and type mapped queries against an external data source, rather than filtering
an in-memory ORM object -- and it manages to do it with static type
checking.</wrong>

Adding a feature like ?? to C# isn't hard in itself, at the parser/CodeDOM
level. What makes it necessary to be careful about limiting the scope of the
language is the tooling, the IDE, the ecosystem work.

~~~
gecko
I thought I spelled this out quite clearly, but the ROE query provided does
_exactly_ that: it builds an optimized SQL query and executes it in the
database. In the example provided, the SQL generated will roughly be "select
firstName from people where people.type = (whatever the adult enum is)". To do
so, the lambdas don't actually take each row of the database; they basically
take abstract syntax trees that record what messages you send to it. At
execution time, the AST is optimized and sent to SQL. This _can_ leak at
certain places--for example, calling a real method on the object, rather than
comparing attributes--but they're identical to the places LINQ to SQL falls
apart.

~~~
waldrews
Humble apologies, I didn't read the article properly and mistook the argument
for a (much less sophisticated) one that's a pet peeve.

------
jcromartie
James Robertson wrote something _very_ similar in 2006:
[http://www.cincomsmalltalk.com/blog/blogView?showComments=tr...](http://www.cincomsmalltalk.com/blog/blogView?showComments=true&entry=3318571155)

