Hacker News new | past | comments | ask | show | jobs | submit login
Desperate Perl Hacker (tbray.org)
99 points by frossie on July 24, 2010 | hide | past | web | favorite | 81 comments

I like Tim's assessment of Perl's strengths.

There's this idea in the Perl community that it succeeded due to Larry's theories about syntax, but I think that's a bit of a myth.

Everybody forgets that when Perl was created, the idea of a 'dictionary type' as a standard feature was a pretty exotic notion, and the idea of wasting such jewels of computer science on sysadmins and philosophy majors was almost scandalous.

And of course now we have entire languages built on dictionary types, like Python and Javascript, and nobody thinks it's weird.

I wonder what the Perl of our time could be. If someone dragged all the neat bits of Haskell into a language that mere mortals could use, that might be a start. And in fairness the Perl 6 people think they can bring parsers into the mainstream. Personally I think they're a bit overobsessed with syntax, but I'd be happy to be proven wrong.

I think Perl 6's Rules are a natural and important extension to the language. Perl has always been a tool of choice for text processing; but people tend to [ab]use regular expressions even where a real parser would be much more appropriate because their language -- Perl, Python, Ruby, C#, pretty much any commercially popular language these days -- gives you regular expressions on a platter, and their presence in the core language or standard library sort of enshrines regexes as a tool of choice.

Look at it this way: pretty much any "Programming ______" book will at least touch on that language's regexp implementation. How many such books currently cover implementing basic parsers for non-regular grammars?

So yeah, if Perl 6 can bring parsers to the mainstream by providing easy, visible tools to implement them, and even simply by making parsers something you learn about in the course of learning Perl, that'll be a huge win for us all.

"Programming ______" book will at least touch on that language's regexp implementation. How many such books currently cover implementing basic parsers for non-regular grammars?

(i think Expert F# might also, i can't remember)



A good point, but I think the books and languages didn't have much choice until recently. There really is a big jump in complexity from using a few regexes in a program, and going full-on into a parser using lex/yacc (or your language's port, like javacc).

I think it's only been very recently that there's been a growth in parsing paradigms that are more amenable to inclusion as libraries with ease-of-use at least vaguely comparable to regexes. Things like combinator-style parser libraries (Parsec and similar libraries in other languages), and OMeta (http://tinlizzie.org/ometa/).


When you actually get down to writing it, a top-down parser is a fairly simple beast. A programmer doesn't really need many extra tools to do it. You just create a bunch of functions which peak-at symbols, consume symbols and call each other.

The art is in having an exactly nailed-down specification concerning how these function are going to work together before you even begin. And that art comes from first analyzing and manipulate your language's grammar. That's a process that's distinct from the final code that you write and it requires knowing a little bit about abstract languages. (Your functions are pieces of the language but not the language pieces one would intuitively imagine. The parts of a language are great examples of "programming objects" which are not "object oriented" objects).

What's hard about understanding an abstract language is that it isn't a command or feature that within a language but is ... a language in itself. Creating an abstract language involves moving to a higher logical level, just using a programming language involves moving to a higher logical level than using an application. It's not that it's impossibly hard but it requires a student to interrupt any rote learning they're doing and think. It's a great thing but it's also a road block to a significant percentage of students.

Given this situation, I'm not sure how any language feature can help. Sure, you could plunk a whole parser generator into the code but I'd say it's harder to understand the combination of language specification and generated-code that results from such a thing.

I'd like to know how the Perl 6 people expect to deal with this challenge.

I agree that it would be nice to move people from complex regular expressions to reasonably simple parsers.

In the real world you seldomly have an exactly nailed-down specification of anything. If you're lucky enough to do something as basic as parsing URIs or JSON, you have, but normally not.

Writing a recursive descending parser isn't hard, as you said, but it's again something that needs testing, you need put some effort into it to get good error messages from it - and why would you repeat such a task? Also you often need to write a lexer -- another burden of which Perl 6 grammars free you.

Regarding your thoughts about abstract languages: Perl 6 provides tools to make experimenting with them rather easy, which helps learning about them.

I've made the experience that a reasonably good programmer can write some 20 lines of Perl code, and it does what he wants. That doesn't imply that he can write a 20 lines grammar that does what he wants, without doing some debugging and corrections first.

Writing grammars is a skill that just has to be learned, line any other skill too. Providing tools that make it fun and rewarding to play with parsing stuff is the best we can do.

[Ruby and Python are not] equipped with better libraries

I disagree strongly here; Perl has a lot of little utility libraries (LWP, etc) but lacks the mature and amazing frameworks like Rails and Django -- and since you can use the database components separately (at least in Rails 3's case), it's lightyears ahead of Perl's DBI. The whole point of Ruby (I can't personally speak for Python) is not just that its grammar is better-constructed and solves problems better. It's also the libraries that make me wildly productive, in a way that Perl has never allowed me to be.

That's because you're not familiar with modern Perl libraries. Out of all of the languages I've worked with, by far my favorite database library is Perl's DBIx::Class. If you want a mature, amazing web framework in Perl, look at Catalyst. Perl has a lot more than just "little utility libraries" if you care to look.

eh, from a sysadmin perspective, the real advantage is that perl5 is perl5 and it has been essentially the same for what, 10 years now? If you are stuck on a RHEL system (or anywhere else where you expect the server to remain up without major revisions for 3-5 years) maintaining the two or three versions of your scripting language is a pain in the ass. (yes, you have to do this on RHEL 5 if you support python, usually. the base system requires a pretty old python runtime, so you need another if you want the new stuff.)

I remember working on perl when you still needed to keep both perl4 and perl5 around on a system... today, though, perl5 is the scripting language you can depend on having, that you can depend on being there and compatible.

This is also the major resistance to perl6. Goddamnit, perl5 works. I don't want to go back to the bad old days when i had to write everything in sh (or test it across two major revs of perl) if I wanted it to run properly on all systems.

I know that the language writers always claim backwards compatibility is a priority, and it is, kindof, but, for example, don't try to upgrade the base python on a RHEL5 system. you /can/ but it's a /huge/ pain in the ass and it means you will have to do more work every time yum or some other python-dependent base bit is upgraded.

for us, perl occupies the niche that bash occupied in 1997. Now, it /can/ do more, but we are utterly dependent, at this point, on the bash replacement.

Perl 5 isn't going anywhere, nor does anyone claim that it is. Perl 5 is still being actively developed, and it's still going to be in the future. Perl 6 has worked out to not be so much a Perl 5 replacement as just another language in the Perl family.

Isn't that exactly why Perl 6 can't possibly do as well as Perl 5 did?

If you assume that the only Perl 5 programmers would ever have an interest in Perl 6, perhaps.

The perl community have been searching for the best way to overcome the perception that "perl is dead". (The constant bickering that it isn't the case reminds me of the holy grail http://www.imdb.com/title/tt0071853/quotes?qt0470614 )

They need to focus on "Where perl has been" while it has been out of the limelight because, believe me, perl still rocks for productivity and performance.

Moose has revolutionized the object system (a system that was so flexible that it just gave you the parts and said "go build it")

Perl::Critic has created a baseline which all production scripts should be above.

PSGI and plack have changed the way you build and deploy and perl webapps.

In fact, the perl community and the current stars of CPAN have come so far, and have had so much going on that I have difficulty keeping up with the massive strides they are taking, let alone summarize those changes here.

> The perl community have been searching for the best way to overcome the perception that "perl is dead"

Release. Perl. 6.

"perl6 is another language in the same family as perl5. It is not meant to replace or succeed perl5. They are incompatible yet complementary."

The success of perl 5 is in no way determined by perl 6. Perl 5 development continues unabated. As jrockway says here, "Think of Perl 6 as Perl 5's Clojure".

Though my personal opinion is that the naming, and the confusion it creates, sucks and makes it really easy to think that perl (5) is dead because perl (6) hasn't been released yet.

It's a shame it got called Perl 6... Perl 5 has been cracking along and is by no means stale: http://www.nntp.perl.org/group/perl.perl5.porters/2010/04/ms...

Since the announcement of Perl 6 10 years ago, there have been fifteen stable releases of Perl 5.

There have been 31 releases of Rakudo Perl 6. Pugs had half a dozen.

What more do you want?

I want to be able to go to http://www.perl.org/get.html and see two links: download Perl 5.10.xxx, download Perl 6.x.

That would be a very good idea indeed.

In meantime for Rakudo Perl6 the direct link is: http://rakudo.org/how-to-get-rakudo

Some Perl 6 eBooks.

Thanks, I have this and it's great, but it'd be a bit nicer to have something a little more fleshed-out.

Looking forward to Learning Perl 6 when it starts to come along.

I'm talking to Pete Krawczyk about "Perl 6 for Perl 5 Programmers". Is that more what you have in mind?

I totally agree.

chromatic is currently writing a book called "Modern Perl", which teaches Perl the way we know it today, idiomatic, safe, structured Perl.

We just need more of such efforts.

I heard a lot of moaning about this at OSCON.

I'm going to be blunt. I actively dislike Perl, not Perl developers, but Perl. This is because I believe it not only gives people the rope to hang themselves with but encourages them to do it in 6 different ways.

The problem that I have most though, is the reaction to this silly bit of fun. Instead of proving that Perl is awesome and showing what you can do with it, I mostly heard a lot of griping. I'd really love to see Perl 6 succeed and those guys do something amazing, new and innovative that breaks from the mold that I think of them in. But until they do, I'm not going to listen to their whining.

P.S. Tim is also right there are lot of great things we got from Perl, but that doesn't mean they should be allowed to coast on them forever.

So you don't like any programming language, then? Because given a programming language, I can definitely write bad code in more than six ways.

(Have you seen the Python that non-programmers write? Holy shit! It's like 1980s Perl, but worse! Almost like bad code has nothing to do with language, and everything to do with programming ability.)

Oh you can write bad code in every language, and I regularly do.

However, some languages make it far easier for you to write bad code and Perl is at the front of the pack.

In my own opinion each programmatic device should be expressed 1 way. Shorthands are not a good long term strategy, because they increase the chances of massively diversified coding styles.

I upvoted you because even though I disagree with what you're saying, there's nothing wrong with your having that opinion nor with your expression of it.

On the other hand, I simply cannot stand to write Python because the One Way to Do It that tends to be provided is rarely the One Way I Want to Do It.

It's okay that some communities and some languages say "There should be one-- and preferably only one --obvious way to do it", though. After all, if I believe "There's more than one way to do it", that applies to languages and communities, too. If a language whose community believes in only one way to do it makes you happy, use it. If you're like me and prefer having many options to express individual pieces of code however feels most appropriate for that specific situation(e.g., expressing a series of transformations on a list as either a list comprehension or a pipeline like series of feeds connecting higher-order functions), use a language that lets you do that. Neither is better for everyone. Nor is the programmer who prefers either type of language any better than the programmer who prefers the other.

  > P.S. Tim is also right there are lot of great things
  > we got from Perl, but that doesn't mean they should
  > be allowed to coast on them forever.
Most of the people that gripe about Perl think that nothing good ever came from it. (At least in my experience)

They ship a new language, compiler and virtual machine next week (Perl 6 Rakudo Star).

Perl 6 doesn't have much to do with Perl 5 except that the same guy created both. Think of Perl 6 as Perl 5's Clojure. Heavily inspired by the original, but a totally new and different language.

(Think of how non-radical PHP 4 -> PHP 5 and Python 2 -> Python 3 were, and how hard it was to get people to switch. Now imagine differences 100x those, and you can start to see Perl 6's relationship with Perl 5. P5 is going to be around for a long time; it's a great language, and people are going to be using and improving it for a while. P6 is a good language too.

They both happen to have the same name, though, which apparently confuses people.)

> Think of how non-radical PHP 4 -> PHP 5 and Python 2 -> Python 3 were, and how hard it was to get people to switch. Now imagine differences 100x those, and you can start to see Perl 6's relationship with Perl 5.

While the radical difference between Perl 6 and Perl 5 might discourage adoption, I wonder if the less radical changes of PHP 4 -> 5 and Python 2 -> 3 may have contributed to their slow adoption. After all, if there is no radical benefit you will receive from switching your code to a backwards-incompatible new version of your language, it can be hard to justify doing so.

P6 is to P5 as VB.NET is to VB. It's the same language but not really.

Shops that got burned on unmaintanable spaghetti Perl 5 10 years ago aren't even going to look at Perl 6.

It's interesting that he mentions how OOP is important, but not that Perl has one of the most featureful object systems in existence.

Is there a nice chart between comparisons of OOP implementations and kick-ass features between languages? I know Python, CLOS, Ruby, and Perl all can mop the floor with Java and C++ with features, but how do they compare with each other?

> Perl has one of the most featureful object systems in existence.

Having primarily used Python for scripting, I am not too familiar with Perl. Could you explain a little more what you mean by this?

* Perl has one of the most featureful object systems in existence *

That's an overstatement. Perl is in no way actually object-oriented. It's a feature that's tacked on, but it doesn't help you write better code. If you think Perl's object architecture is featureful, take a look at Ruby. It'll blow you away.

I may be wrong, but I think he's talking about Moose, not Perl's native OO style/system. (Though I suppose you could also argue that the native system is pretty nice too, if it allows you to create Moose on top of it.)

So why is Perl so unfashionable? He mentions OO and readability, but if Perl has "one of the most featureful object systems in existence", does that leave just readability? I'm curious (as someone who knows very little about Perl) what the actual reasons are for Python and Ruby having stolen so much of Perl's thunder.

It is pretty much the OO and readability. I worked with Perl for years, and, much to my shame, I really didn't understand a lot of many of the CPAN modules I relied upon. I read the poddoc and then copied the syntax they used and edited to my needs.

Perl provides many, many different ways to achieve the same thing. This is somewhat by design, in the idea that Perl should match your own intuitive style. But most people's styles are not yours, and it becomes impenetrable.

IMHO Ruby's strength over Perl is the much stronger OO, Python's strength is the focus on "there is only one way to do it", which lets most people be able to grok most of your code.

I couldn't go back to Perl by choice now. I first jumped to Python just to get some sense of clarity in my life, but now I prefer Ruby due to how its extreme openness means there are some really awesome gems out there.

  > Python's strength is the focus on "there is only one way
  > to do it", which lets most people be able to grok most of 
  > your code.
Unfortunately this is not my way of doing it, and that's why I'll always prefer Perl to Python.

That is so true. There really is never "only one way to do it" because Programmers think differently.

In other words, coding standards are always useful.

I'd say Python's strength is "explicit is better than implicit." There's usually not too much magic going on.

I really have to agree with this. The Python community has really embraced clarity of code and style as a core principle of the language, while Perl has embraced a more..."do it how you want" philosophy.

It's a bit like the subtle philosophical differences between two dialects of the same language, say Received English and an inner-city slang.

It always get cited as "only one way to do it", but the actual motto is "there should be one obvious way to do it". It's very different.

I think that the full quote is a synthesis of the two:

> There should be one-- and preferably only one --obvious way to do it.

The follow-up I think hammers home the subjectiveness of ‘obvious’:

> Although that way may not be obvious at first unless you're Dutch.

They are all good languages splitting the same market (roughly).

Perl lets you get under the hood in a way that either is frowned upon, impractical, or prohibited in other languages. So there's a language for every temperament now.

For example, in Perl, [if you wanted to], you can roll your own object system. That makes some people happy, and other people uncomfortable. But much of Perl has moved on to one very cool object system, Moose.

The Tim Bray article reads as if he's unaware of Modern Perl and all the Perl 6 stuff which has been back-ported to Perl 5. In all honesty, I had to check the date because the article had a 2002 feel to it.

Comments like this seem to miss the point:

use strict; use warnings; is not required. Moose is not the default object system.

You can't say, "Oh, this library has that feature!" Thats not good enough. It has to be built-in, not optional, core to the language. If you're working by yourself - you can write code in any language well. In groups, that is NOT the case. Writing Perl in groups requires rigorous code reviews and cooperation, and even then nightmares are hard to avoid. Expressiveness has its downsides. You cannot simply bring a Perl noob onto a project and not have him ruin it. You CAN bring a new Rubyist onto a project and they can be productive.

You cannot simply bring a Perl noob onto a project and not have him ruin it. You CAN bring a new Rubyist onto a project and they can be productive.

These assertions beg proofs.

Your requirement that certain features be "built into the core" seems to rule out a lot of languages which happen to be extensible.

* It has to be built-in, not optional, core to the language.*

This begs the question, what's core? Python's regex could go either way, for example.

You can't say, "Oh, this library has that feature!" Thats not good enough. It has to be built-in, not optional, core to the language

CPAN is core to the Perl language and deliberately so. If you don't like this ethos then Perl isn't a "language" you should consider.

One thing that people fail to mention when hyping Moose is that there is a start-up penalty for using it. You can lessen this penalty by not using all of the whiz-bang features of the object system, but that kind of defeats the purpose, no?

[ This doesn't really matter much for long-running scripts or controllers in a webapp, but it does for a lot of other applications ]

Just use Mouse, a lighter Moose. Or use Any::Mouse (whichever is installed).

My take is that the other languages simply supply more structure for how to do things, while Perl leaves you free to do things any which way. Sometimes, that freedom is invaluable. But for some people, that freedom means that they have to work to supply the structure on their own.

In short, I think some people feel that there are too many ways to do it.

Anyhow, that said, I think that people are too quick to write Perl off. Some of us are still writing Perl. Maybe I'm not doing anything big or fancy, but it's all documented, readable, and every single script on my computer uses warnings and strict. If you compare Perl to duck tape, I'd be like McGuyver: always keeping a roll of it in my pocket.

I think a lot of it has to do with the language, when written very idiomatically, tends to rapidly become hard to maintain. Part of this perhaps is the history of Perl as a fantastic one-off tool language (I'm not a developer for my company, I have other responsibilities, but when we need a fast one-off tool for some data processing, the C++, C, Java, Python and Ruby guys all come to me to hack off something to just get the job done because I can do it in a few minutes vs their half a day).

I've looked back at some of those one-offs, and for the life of me I can't tell what they are supposed to do.

On the other hand, I have written some largish projects in Perl, and when I constrained myself to a very explicit dialect of the language (almost Javaish in verbosenss), I found it to run fast, and be highly maintainable. I cracked the code open to a 20k line project I wrote 4 years ago and was able to jump right in.

In that respect it's a bit like C++ in that you can write code to be unreadable or you can write it to be readable if you constrain yourself to a non-idiomatic subset of the language. But unfortunately, the Perl community tried to embrace and open attitude towards this (there's more than one way to do things) that ultimately ended up embracing the "get it done now and in as few lines as possible" idiomatic unreadable code style.

I think other than that, OO is an important factor, Perl's OO system was so hard to do right, and other languages OO just seemed to embedded and clean to work with, but also Perl has not really found itself (for some reason) in many embedded systems, in GUI programming, on client-side systems (e.g. Javascript), etc. all places where it could have done well, but for whatever reason just never found a good comfort zone the way it does for sysadmin type tasks (and a few very small domains like computational linguistics). The Perl community seems to have spent a decade endlessly spinning around the drain of supporting backend work, and never approached the more sexy applications of the language.

One thing is that Perl 5 doesn't include by default a number of modern features that Python and Ruby have OOTB. Easy OOP, exceptions, and named function args, to name a few. If you want these things for Perl, you need to dash out to the corner store (the CPAN) for them.

Also, Perl 5 tends to be used for a lot of unglamorous admin work. So, although almost everyone's got a few utilities or cron tasks on their system written in Perl, they're not shouting about it from the rooftops.

Also, Python is an easy language to learn, and so people like to learn it and blog about how easy/nice/shiny it is: http://xkcd.com/353/ .

Ruby doesn't provide named function arguments. In my experience, Rubyists tend to do what Perlers do: use hashes to create quasi-named arguments to functions.

I still like perl despite the disdain heaped onto it. I have yet to see a good python or ruby way of doing the following:

  for $line (@array) {
    print $line unless $seen{$line}++;
So, IMO perl still has pretty useful and unique bits. It's still up to the programmer to write beautiful or ugly code (I'm not claiming I write beautiful code at all).

> for $line (@array) {

> print $line unless $seen{$line}++;

> }

The impulse to golf is strong …. How about

    $seen{$_}++ or print for @array


It's very clever.

That's why I always use the non-clever version for my code, except with foreach instead of for, just so that it reads like English.

Clever is fun, though. I do understand the temptation. I admit to having done crazy stuff like turning a JAPH into a vulnerability scanner (in retrospect, the "fork until there are 256 of you and scan an entire subnet at the same time" thing wasn't that great). But I usually find myself wanting to come back to the code and know what I was thinking.

Being too clever makes that difficult.

Golfing aside (I don't golf), I guess this pattern struck me as being natural and not really trying to be clever.

Used in small doses and when you need something quick and dirty, it has been immensely useful, much like vlookup() in Excel.

Depends on which pattern you're talking about. I was talking about inverting everything and using the magic of $_ everywhere.

If you mean just the idea of using a hash to mark things you've seen before, though, that is very, very useful. But I would write it as:

foreach $item (@list) { # Print every item in the list skipping duplicate items. print $item unless $seen{$item}++; }

Because then I don't have to think about what's happening with all the magic $_ variables Perl is tossing around. Sure, they're convenient. And I do use them some of the time. But when things break, I'm usually glad that I was explicit about what I was doing rather than relying on Perl's magic to know what I intended to do.

Maybe that's why I like Perl so much? I tend to code in as strictly disciplined a manner as I can manage, so I don't end up regretting it later.

The golf(ish) answer is by JadeNB. My original code is exactly the same as yours =)

But more importantly, I have not seen anyone post equivalent Python or Ruby code.

There was no comment and you used for instead of foreach, but that's just a nit. I did lose track of which person I was replying to :)

You're right that the overall pattern is VERY useful.

Guess no Rubyists or Pythonistas want to step up to the plate. =)

Why oh why does he write so much FUD in the post? Is all this Perl’s situation is not terribly happy and ...Perl’s somewhat beleaguered status really called for? Is he even correct?

Well here's a fascinating stat from OSCON 2010. The language with most (presentation) tracks was... wait for it... Perl!

* Perl - 17 tracks

* Python - 16 tracks

* Java - 13 tracks

* Javascript & Ruby 7 tracks each

Interesting eh? :)

The history is relevant there, though: OSCON started as a perl conference and expanded to other areas later.

(FYI, a "track" is a group of related talks, not a single presentation.)

Yes OSCON grew out of the Perl conference. The Perl conference still happens each year (see YAPC, last one being only a few weeks ago).

re: track - OK to clarify within the Perl track there were 17 presentations. See http://www.oscon.com/oscon2010/public/schedule/topic/Perl)

Perl didn't introduce dictionaries first. Awk and Smalltalk did that before Perl. Perl got the idea from Awk.

Also, note that Tim's probably right about string manipulation. A lot of times when Smalltalk was perceived as slow even when it was fast, had to do with string manipulation. (And bad programming!)

If you finished the first sentence after the bullets:

>Yes, there were other languages with some of these characteristics before Perl. But Perl had all of them...

Yeah, I know about the whole dictionary thing. I'm not saying this was the factor that caused Perl to succeed. There were a lot of reasons, one of which was that Perl was unafraid to have multiple reasons for people to join their community. ;)

Tim's current position is something any professional developer should reflect upon. Even if the tools or language one currently uses are popular (and growing more so) at the moment, there's a good chance that they will eventually enter into long-term decline, like Perl has. Ruby users may now feel on top of the world, since despite its flaws the Ruby community is vibrant, exciting, and lucrative, but one day we too may be protesting that "Ruby isn't dying", despite all evidence to the contrary.

Being honestly aware of the strengths and weaknesses of the tools one uses, and being willing and able to move to other tools if necessary, are attributes that will probably help one have a much happier career.

Readability is important.

Maintainability (and by corollary readability) is PARAMOUNT.

Post mentioned most programming is string manipulation. Well even more code must be maintained. It is the #2 tool feature / programmer skill. (#1 is being able to get shit done and thus create something needing maintenance).

$perl += 1;

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact