

The Emacs Problem  - mofey
http://sites.google.com/site/steveyegge2/the-emacs-problem
&#60;&#60;What would you rather do? Learn 16 different languages and frameworks in order to do "simple" log-file and configuration-file processing? Or just buckle down, learn Lisp, and have all of these problems go away forever?<p>It's a rhetorical question. The answer is patently obvious at this point:   Lisp is evil, and you'd damned well better write all your code in C++ and XML and JavaScript and PL*SQL and CSS and XSLT and regular expressions and all those other God-fearing red-blooded manly patriotic all-American languages from now on. No more of this crazy Lisp talk, ya hear?&#62;&#62;
======
snprbob86
Uuugh... XML... how I hate you. Everyone should need to learn Lisp, even if
you never ever program it, simply to avoid developing some of these poisonous
XML-flavored tools. He mentions Ant specifically. Let me tell you, as an
unwilling Microsoft Build expert, XML should be NOWHERE NEAR your build
system. He puts down Makefiles, but as someone who wrote a Makefile for the
first time after years of Ant and MSBuild, it was like a breath of fresh air.
The intentional lack of power is what makes it so great. If anything,
Makefiles are too powerful and too close to turing complete.

Here is what a make file should look like:

1) vpath statements for various file extensions

2) Phony targets for logical build units, including "all"

3) Generic rules for mapping _.abc to_.xyz; these rules should have _exactly_
one line of code which executes an external script/tool

4) A list of edges in the dependency graph

5) A few variables as necessary to eliminate redundant edges

If you put any logic in your Makefiles, you are doing it wrong.

If your builds are slow, add empty dummy files for larger culling by
timestamps. If timestamps are insufficient, codify early out logic into tools.

Not having logic in my Makefiles enables parallel execution and strong support
for incremental builds. If I were to use Lisp as a build system, I'd create a
DSL that had these same properties; forbidding arbitrary logic in my
dependency graph. It's about finding the right balance to inject
expressiveness without losing desirable properties of the DSL. This is why
every developer needs to understand more about programming language design.
Anytime you create any type of file format, you are doing it. And anytime you
are writing any type of file format, you are reverse engineering it.
Understanding the intended separation of logic for Makefiles helps me write
better Makefiles.

~~~
1amzave
> _If anything, Makefiles are too powerful and too close to turing complete._

"Close to"? Not a formal proof of Turing-completeness (and may only work with
GNU make, not sure), but...

    
    
      [me@host: ~]% cat fibo.mk
      dec = $(patsubst .%,%,$1)
    
      not = $(if $1,,.)
    
      lteq = $(if $1,$(if $(findstring $1,$2),.,),.)
      gteq = $(if $2,$(if $(findstring $2,$1),.,),.)
      eq = $(and $(call lteq,$1,$2),$(call gteq,$1,$2))
      lt = $(and $(call lteq,$1,$2),$(call not,$(call gteq,$1,$2)))
    
      add = $1$2
      sub = $(if $(call not,$2),$1,$(call sub,$(call dec,$1),$(call dec,$2)))
    
      fibo = $(if $(call lt,$1,..),$1,$(call add,$(call fibo,$(call dec,$1)),$(call fibo,$(call sub,$1,..))))
    
      numeral = $(words $(subst .,. ,$1))
    
      go = $(or $(info $(call numeral,$(call fibo,$1))),$(call go,.$1))
    
      _ := $(call go,)
      [me@host: ~]% make -f fibo.mk
      0
      1
      1
      2
      3
      5
      8
      13
      21
      34
      55
      89
      144
      233
      377
      610
      987
      1597
      ^C

~~~
mahmud
"The language of GNU make is indeed functional, complete with combinators (map
and filter), applications and anonymous abstractions. GNU make does support
lambda-abstractions."

<http://okmij.org/ftp/Computation/#Makefile-functional>

------
eklitzke
One solution that Steve didn't discuss is JSON. To be fair, JSON wasn't that
popular in 2005, but it's still a great solution.

The way it works is that their are no mandatory newline characters in JSON.
Whitespace between lexical elements is ignored, and any embedded newlines in
strings can be escaped (i.e. as \n). So a log format that a few people are
using today is like this:

{'kind': 'foo', 'id': 1, 'msg': 'hi'} {'kind': 'bar', 'id': 2, 'msg': 'there'}

Each log message takes up a single line in the file. You can trivially
deserialize any line to a real data structure in your language of choice. You
can (mostly) grep the lines, and they're human readable. I do this at work,
and frequently have scripts like this:

scribereader foo | grep 'some expression' | python -c 'code here'

In this case we're storing logs in the format described above (a single JSON
message per line), and scribereader is something that groks how scribe stores
log files and outputs to stdout. The grep expression doesn't _really_
understand JSON, but it catches all of the lines that I actually want to
examine, and the false positive rate is very low (<0.1% typically). The final
part of the pipe is some more complex python expression that actually
introspects the data it's getting to do more filtering. You can of course
substitute ruby, perl, etc. in place of the python expression.

I feel like this is a pretty good compromise between greppability, human
readability, and the ability to programatically manipulate log data.

~~~
kuahyeow
"XML is better if you have more text and fewer tags. And JSON is better if you
have more tags and less text. Argh! I mean, come on, it’s that easy. But you
know, there’s a big debate about it." — Steve Yegge

<http://simonwillison.net/2008/Jun/15/steveys/>

That's the problem with discussing old articles. Information gets updated

~~~
mofey
Not really. The arguments still hold.

------
panic
_You could even trivially convert it to XML and use XSLT, if you were silly
enough. But Lisp is directly executable, so you could simply make the tag
names functions that automatically transform themselves. It'd be a lot easier
than using XSLT, and less than a tenth the size._

And now anyone who can modify your log can execute arbitrary code in the
reader process…

~~~
ynniv
People always say that. Fortunately someone figured that out on the order of
34 years ago

[ <http://en.wikipedia.org/wiki/Capability-based_security> ]

(for a more recently active discussion, try
<http://en.wikipedia.org/wiki/Domain_Specific_Language>)

------
Xurinos
Did anyone else think of CL-PPCRE as they read through the "lisp does not have
regular expression support" implications? That was answered by the comparison
of elisp to modern Common Lisp, and I wonder if anyone has done any work to
make CL-PPCRE work for elisp. In spite of being someone's library, it is much
faster than perl's built-in regex support.

There is a point to be made for the idea that you are solving the wrong
problem with the log parsing. On the other hand, if you are trying to
interface with other developers' popular engines, you may not have a choice.

~~~
jrockway
Most of those benchmarks are against Perl 5.8, the version of Perl released in
2002. 5.10 had major regexp engine improvements, and 5.12 had minor
improvements. Anything is fast when you compare it to 10 years ago.

~~~
btilly
Furthermore the 5.8 engine is pretty much the 5.6 engine, which was
significantly slower than the 5.005 engine. Why the slow-down? Because 5.6 has
logic to check when regular expressions are matching slowly, and then to add
tracing when it is. This makes many of the exponential slowdowns that
_Mastering Regular Expressions_ describes be automatically caught and handled
fairly quickly.

The CL-PCRE and Java regular expression engines don't have those somewhat
expensive checks, and so are much more likely to encounter catastrophic
behavior.

~~~
jrockway
Yeah, but who cares about real-world code if you have a neat benchmark page?
Most of your "users" will never even get to the point of using your package --
after saying how great it is on Reddit, they'll turn on their porn and forget
all about it.

OK, maybe I'm too cynical...

------
pragmatic
Just to point out LINQ to XML <http://msdn.microsoft.com/en-
us/library/bb387098.aspx>.

It takes a lot of pain out of XML processing. Don't have to remember the
specifics of XPath/XQuery but you still have to deal with the pain of multiple
namespace resolution inherit in XML.

------
jcapote
It would be interesting to see an emacs implementation in language better
suited for text manipulation like Ruby or Perl.

~~~
mechanician
As an emacs newbie with a nagging desire to learn Perl, I would gladly commit
to switching to an emacs implementation in Perl.

~~~
jrockway
Not that I would use it, but there is Padre:

<http://padre.perlide.org/>

------
vtail
TextMate/Ruby = Emacs/Elisp

~~~
ynniv
But Emacs is capable of _intelligently editing_ source code which is not
Elisp. As much as I have tried, I cannot say the same of TextMate.

(How hard can it be to select blocks on double-click? Even Terminal.app does
it!)

