
Perl is no more accurate than randomly-generated language - j_baker
http://www.neverworkintheory.org/?p=197
======
celoyd
I’ll point out what a dozen others are probably typing right now: what works
for beginners is not necessarily what’s best for experts.

An analogy: I don’t know how to read sheet music with any proficiency. It
would be easier for me if note duration were shown by horizontal length
instead of the system of tails and open/filled circles. However, this wouldn’t
be as good for experts, who care more about fingering than about being able to
see the melody in the most literal way.

I don’t happen to like perl’s syntax, but this study by itself isn’t proof
that it’s objectively bad for the people who use it heavily.

~~~
jerf
"what works for beginners is not necessarily what’s best for experts.... this
study by itself isn’t proof that it’s objectively bad for the people who use
it heavily."

More than that. Three languages were chosen. One was based on what previous
studies had shown works, and one sounds like it was basically a ROT13'ed perl
(only with symbols), and from what some were saying on Reddit it was an
overcomplicated dialect of perl (though IMHO that would change little). Then,
in a study in which they pitted Perl against a language designed to do well in
studies, the language designed to do well in studies prevailed.

Even though I do question whether there may be a wee whiff of tautology about
this whole exercise, this is interesting because it shows that studies may be
able to distinguish between the features of two languages w.r.t. how easily
they can be taught and learned. This is valuable stuff, millions upon millions
of dollars to society's worth of valuable stuff. What it says about Perl is
simply that it was _not_ designed to do well in such studies, _quelle
surprise_ , and in particular it doesn't say anything at all about Perl
relative to any other real language. $YOUR_FAVORITE_LANGUAGE may well have
fared worse, and $YOUR_MOST_HATED_LANGUAGE may well have fared much better.

It is a true shame that the underlying valuable message of the study is being
missed by so many people in a tearing hurry to go "hurr hurr, perl t3h suckz,
hurr hurr".

~~~
jessedhillon
_Then, in a study in which they pitted Perl against a language designed to do
well in studies, the language designed to do well in studies prevailed._

No, it's not surprising that perl performed poorer than Quorum. The surprise
is that perl did as poorly as Randomo, which should not be sensible given that
the operators and keywords are gibberish.

I generally agree with what you're saying, but the point of the study is not
that the well-designed language did better. Who wouldn't expect that? But
there is no tautological question when you realize that the result they are
focusing on is the near-even performance with Randomo, not the failure to
exceed or match Quorum.

It's pretty clear that they aren't engaging in perl bashing, but trying to
provide data that backs up their experiential findings as teachers.

~~~
loup-vaillant
From what I've understood, Randomo and Quorum's syntaxes share the same
structure. Only keywords and operators changed. Therefore, Randomo has a much
better structure than what we could expect from a truly randomly generated
language.

So, while it still looks pretty bad for Perl, I'd say it's not _that_ bad.

~~~
Mithaldu
It doesn't look bad for Perl at all, because the entire setup of this study is
flawed to the point of making the conclusion a non sequitur.

~~~
jessedhillon
It always amazes me when people can dismiss academic papers -- the result of
(at least) several weeks of work -- with a glib comment uttered anonymously,
and no support given.

Have you read the study? If not, you can find it here:
<http://www.cs.siue.edu/~astefik/papers/StefikPlateau2011.pdf>

~~~
Mithaldu
I have read the study which is why i made that comment. In fact, i've even
contacted the author to gain more details about how they selected code samples
to serve as learning aids and how all the other code samples looked like.

Additionally, i am not anonymous. Google my name and you will find all about
me, including, if you look for longer than a minute, my full name and address.

Lastly, there was no need to give support in that comment by repeating what
others and myself have said copiously all over the comments here. This was my
main post towards that. Feel free to argue it:
<http://news.ycombinator.com/item?id=3153249>

------
cstross
Perl was designed around two slogans: "the UNIX swiss-army chainsaw" and
"there's more than one way to do it" (for any value of "it").

I first met Perl after spending some years working in the UNIX command line
environment, and it made intuitive sense ... to someone who primarily used
awk, sed, and bash (and able to get stuff done in C). I was up to speed and
productive in it within a couple of days. (This was in the days of Perl 4 --
the OO stuff in Perl 5 came years later.) At that level, Perl was basically
every UNIX text-handling tool rolled into one handy bundle oriented around
script execution rather than interactive shell work, with a bunch of POSIX
extras and things like the socket() API thrown in to take it beyond the toy
language level.

The point is, Perl isn't a beginner's language. Rather, it's an elaboration of
a pre-existing toolset that assumes you're already familiar with the UNIX way
before you get started. Pitching it against a teaching language isn't even an
apples/oranges comparison: it's like comparing a circus merry-go-round to a
helicopter (they're simply not intended for the same purpose or the same user
demographic).

~~~
chromatic
"Perl is, in intent, a cleaned up and summarized version of that wonderful
semi-natural language known as 'Unix'."

Larry Wall,
[http://groups.google.com/group/comp.lang.misc/msg/1ebc89bbc0...](http://groups.google.com/group/comp.lang.misc/msg/1ebc89bbc0a28c3b)

------
Miky
The language was not designed by chance. Some of the syntax was designed by
chance. “Some of the syntax” is a very tiny part of a language.

While this is interesting, presenting it as a “randomly-generated language” is
extremely misleading.

~~~
m0th87
Fair point. They can't, for example, randomly generate language semantics (how
the hell would that work? :)

All in all, though, this paper goes much further in trying to derive language
usability than any I've seen. And it does seem disturbing that randomly
selected characters (even for a subset of the language) should fare as well as
Perl's hand-picked syntax.

------
Mithaldu
This study is entirely useless because they invalidated themselves from the
start by providing the worst possible Perl examples they could. I quote the
salient parts:

"participants were given a code sample worksheet for the particular language
group they were in. The general idea of the experiment is to give novice users
code samples similarly to if a participant was learning to program from home
on the Internet.

This code shows one of the code samples provided to participants."

    
    
        $x = &z(1, 100, 3);
    
        sub z{
            $a = $ [0];
            $b = $ [1];
            $c = $ [2];
            $d = 0.0;
            $e = 0.0;
            for ($i = $a; $i <= $b; $i++){
                if ($i % $c == 0) {
                    $d = $d + 1;
                }
                else {
                    $e = $e + 1;
                }
            }
            if ($d > $e) {
                $d;
            }
            else {
                $e;
            }
        }
    

__This ridiculous thing was supposed to help their test subjects learn Perl.
__

If all their perl samples looked like that it should not be a surprise to
_anyone_ that their test subjects were entirely confounded. What is missing in
that paper is a description of how they actually sourced their code samples,
as well as a list of _all_ code samples provided.

------
raldi
Misleading title.

s/no more accurate/no easier for beginnners to learn/

~~~
RickHull
Yes, upon reading the obvious-flamebait title I asked myself, "What does it
mean for a language to be accurate?", and "How could something randomly
generated possibly be more _accurate_ than something designed?". I did not
expect my questions to be answered by the article, and in that I was not
disappointed.

~~~
j_baker
I didn't intend the title to be flame bait. I only wanted to get to the gist
of the matter as tersely as possible, which isn't easy. What would you have
titled this?

EDIT: And why the downvote? I would actually be interested to hear a better
title.

~~~
smokestack
"Perl's syntax is as intuitive to beginners as that of another language whose
tokens and keywords were randomly-generated" is verbose, but accurate. The
structure of Randomo was entirely based on Quorum, with certain tokens and
keywords randomized. The study really only speaks to the "intuitiveness" of
the most superficial parts of those languages' syntax, which is something that
Quorum happens to focus on:

    
    
        action z(integer a, integer b,
        integer c) returns number
          number d = 0.0
          number e = 0.0
          integer i = a
    
          repeat b - a times
            if i mod c = 0 then
              d = d + 1
            end
          else then
            e = e + 1
          end
    
          i = i + 1
        end
    

edit: I realize the title I gave isn't short enough to post on HN, but I
wanted to reply just to make a point about the study, not your title.

~~~
wazoox
Actually Quorum looks much more verbose than Perl. That may be the main
distinctive feature helping absolute newcomers to grasp the code. In that
vein, Java probably would fare way better than most terser languages, too.
This would be something to test for (on a significant number of guinea pigs
preferably).

------
chromatic
What an awful title.

What a silly thing to test in isolation; does the _syntax_ of a programming
language present a greater barrier to a novice than the _semantics_ of
programming?

A better study would have controlled for the ability of the eighteen
participants to describe their algorithms in pseudocode.

~~~
colanderman
It's not silly if it's significant.

~~~
chromatic
If you can't isolate that sole characteristic, your findings aren't
empirically significant.

------
m0shen
The provided perl is a little dated:

    
    
      $x = &z(1, 100, 3);
      
      sub z{
        $a = $_[0];
        $b = $_[1];
        $c = $_[2];
        $d = 0.0;
        $e = 0.0;
      
        for ($i = $a; $i <= $b; $i++){
          if($i % $c == 0) {
            $d = $d + 1;
          }
          else {
            $e = $e + 1;
          }
        }
    
        if($d > $e) {
          $d;
        }
        else {
          $e;
        }
      }
    

Fixed?

    
    
      $x = z(1, 100, 3);
      
      sub z{
        my( $a, $b, $c ) = @_;
        my $d = 0;
        my $e = 0;
      
        for ($a..$b){
          if($_ % $c == 0) {
            $d++;
          }
          else {
            $e++;
          }
        }
      
        if($d > $e) {
          return $d;
        }
        else {
          return $e;
        }
      }

~~~
kokey
This also clearly reminds me of one of the uses of perl. It's nice to use for
prototyping something that can be implemented in ANSI C when necessary.

------
hippich
"While they spend their time explaining how Perl can't be used for productive
work, we do commit even more code to CPAN and make our sh$t just work by the
end of the day."

------
acqq
The whole experiment is based on _learning from other code samples_ (see
<http://www.cs.siue.edu/~astefik/papers/StefikPlateau2011.pdf>). That's what
they measured -- how easy is for the beginner to learn from the code samples.

If you don't explain the logic behind the special characters, of course it's
easier to learn what the code is about from the notation which has less
special characters and less semantic behind the special characters, like their
better faring language:

    
    
      action z(integer a, integer b,
      integer c) returns number
      number d = 0.0
      number e = 0.0
      integer i = a
      repeat b - a times
      if i mod c = 0 then
    

I'd say the students were talented if they managed to make reasonable results
in any language, from my perspective, their "better" language is still way too
hard for absolute beginners.

~~~
sasha-dv
You're absolutely right.

I blame the title for turning this into _Perl sucks vs. no it doesn't_
discussion. Not going into the problems with their study (N=18, poor Perl
samples,...), all they prove is that a novice will much better understand
pseudo code designed to be easy to understand than a real programming
language.

I'm a Perl hacker and I would be the last to claim that it has an easy to
understand syntax. Perl is full of shortcuts and magic that is hard for
beginners, but it makes my life as a programmer much more easier. If they
compare their pseudo language with Perl, Ruby, and Python, Perl would be at
the bottom of the list.

------
danssig
This shouldn't be surprising. Even experts get surprised by what perl's syntax
actually does [1].

[1] <http://blog.plover.com//prs/accidental-syntax.html>

------
bradleybuda
N = 18

------
wazoox
There seems to be a number of "studies" with a ridiculously low number of
participants (here, 18), from which "scientists" (actually, students) try to
draw conclusion from quasi-random, statistically insignificant results.

Thousands of people learned programming using perl. This is much more
significant than this ridiculous study (randomo language? quorum? give me a
break would you?).

Fortunately, at least this poor-piss stuff won't make the headlines, like too
many such ridiculous studies run on 30 students do (remember lately "women
wearing make-up appear more competent", augh).

------
darrikmazey
I have spent years upon years programming in perl, and I love it dearly, but I
laughed pretty hard at this. No offense to Larry Wall, but if you've spent any
time writing perl code, surely you've had at least one moment where you
wondered if someone hadn't gotten one over on you. I don't knw about relative
ease of learning, but perl can be obscure. Of course, some think that adds to
the fun. ;)

------
wccrawford
Suddenly I'd like to know how other languages fare. I have a feeling they
might not be much better.

~~~
edsrzf
You may be in luck. The paper says:

"Given this interesting first result, we plan to test a number of additional
languages using the same procedures."

Hopefully they'll use larger sample sizes too. This one involved 18 people
across 3 groups.

~~~
Mithaldu
If they use the same procedure the result will be completely useless.

------
wedesoft
In other words: When a beginner writes code, the programming language does not
matter.

------
chc
This title is misleading to the point of being nonsensical. Attributing the
inaccuracy to Perl itself is just plain weird. It's the people programming in
Perl who failed to be accurate in their code — it's not like they wrote
correct code and Perl (or Randomo for that matter) moved things around on
them. But that doesn't get as much of a guffaw out of Perl-haters, I suppose.

------
dbbo
Unless I missed something, I don't think being easy for novices to use was one
of Larry's design goals for Perl. Nevertheless, I found Perl pretty easy to
learn, and it's the only language I ever "stuck with" long enough to build
complicated programs.

------
aneth
Shall we compare new users of TextEdit to new users of vi?

