>>this section pretty much sums up why I've always hated working with Other Peoples' Perl.
I think that statement was really more of 'The remainder is left for the users as an assignment' kind of statement.
>>To accurately follow someone else's perl requires either an encyclopedic knowledge of the language and it's edge cases or very heavy documentation and a set standard of coding practices.
To accurately follow anything non-trivial written by anybody in any programming language will require you a good grip over the language and its corner cases. There is no magic here, if a language doesn't offer facilities to heavy lift a certain thing you will end up writing those yourself.
>> To accurately follow anything non-trivial written by anybody in any programming language will require you a good grip over the language and its corner cases. There is no magic here, if a language doesn't offer facilities to heavy lift a certain thing you will end up writing those yourself.
I'm going to have speak subjectively and relative to my own skills here. Without rigorous use of standards Perl 5 is a nightmare to read. With very clear and consistent style then it can be readable, but this has to be enforced extremely strictly for any code base beyond trivial. Perl's 100 ways to do things is not a positive in this context.
Case in point, I was given a bit of perl two weeks ago by a bioinformatics lecturer. Did not use strict or warnings; variables weren't declared in scope; no subroutines, let alone Moose objects. And this is typical of ~90% of bioinformatics perl that gets written. What's more, it was a trivial script that was simply parsing some BLAST matches from CSV, checking for overlaps and assigning a type based on the match gene combination - and was completely indecipherable. It took me two days of refactoring to pick out all the details. And he managed to use a couple of magic symbols I'd never seen before.
The first time I was given a bit of python, without learning the language at all, I was able to extend the script and get it do some new things (it was considerably more complex than the aforementioned perl script).
And even when it is well written and documented - I did work with some very good perl programmers - it's tended to end up as very fragile and difficult to extend or even substantially redevelop due to the heavy use of context for determining behaviour (and the actual context can be difficult to determine). Unit testing had to be very extensive since it was difficult to isolate changes.
I'm going to have to be precise here - I'm attacking Perl 5. It just seemed from that section that Perl 6 doesn't solve the major problem (difficult to deduce the context and work out the resulting effect) I had with Perl 5. I may be completely missing the mark and actually readability is much better. I really hope so.
>> Perl 6 isn't production ready yet. But even a cursory glance over the features tells, Python isn't even in the same league of tools Perl 6 will be.
Vapourware is vapourware. Perl 6 was meant to be production ready a decade ago. On the other hand, I'm interested in knowing what you think Perl 6 brings that will give it the edge over Python? Do you see it displacing Python as rapidly as Python replaced Perl 5 as the science "glue" language of choice? With the apparent rise of Julia as well, could it just be too late?
I'm not talking about Perl 5 per se. But I've worked with C, Java, Pig, Python, Perl and a bit of shell scripting so far. And I've never had a situation where I could simply glance over something and know all of it(This is for reasonably big applications). I'm currently working on a big Java application where logic is often deep buried below Factor classes, Interfaces, Enums and layers of classes simply passing control to level below. And when you find the code that does something finally, you will see it wrapped will kinds of exception handling code. Its simply too verbose, with pointless ceremonial code.
In pig, any reasonably large workflow has had me hooked for hours working with stub data trying to figure out how data flows and comes out.
In fact I see this nothing new to Perl. If you are working with any serious application. A good touch over the language plus experience with handling things with patience is a must. Python is no exception to this rule, neither is any language I know of.
>>I may be completely missing the mark and actually readability is much better. I really hope so.
It depends on how you define readability. I would say verbose code is equally unreadable especially if you work with large applications. If you simply don't like regular expression you have a different problem. In some cases use of regular expression is simply unavoidable. IMO Python works very well for applications that interact with standard data sources(Databases, API's, XML's). Once you get the data to some very easily ingestable form, and there is little work to do from there on, Python works just fine. But Perl works fine for other use cases, heavy lifting and other kinds of data work. Perl may not be the right language for all use cases, but neither is Python or Java.
>>Vapourware is vapourware. Perl 6 was meant to be production ready a decade ago.
No, I don't think they ever gave out a date when they wanted a finished version out. Besides Rakudo is feature ready already. They are just working out last bit of performance issues and good to have things. And plan to have it out by 2015: https://fosdem.org/2015/schedule/event/get_ready_to_party/
>>On the other hand, I'm interested in knowing what you think Perl 6 brings that will give it the edge over Python?
Grammars, Macros, Concurrency, Traits, Types, Improved regexes, Better OO, Functional programming, User defined operators, Mutable grammar, Junctions, Currying, Continuations, Multiple VM backends to name a few.
>>Do you see it displacing Python as rapidly as Python replaced Perl 5 as the science "glue" language of choice?
Perl and Python both co exist and will continue to, albeit the web frameworks market is dominated by Python and Ruby at this point in time, Perl is still pretty big in a lot of places. This isn't Perl or Python, people use tools which work for them within their project limitations.
But I guess people will use what they find useful. And that only individual projects can decide as to what works best for them.
>>With the apparent rise of Julia as well, could it just be too late?
Reading the Perl 6 spec will tell how different Perl 6 is with regards to everything else. Even with the delay Perl 6 has suffered, I don't see any other tool that has the feature set Perl 6 has to offer.
Absolutely, with any serious application, it takes time to break it down and understand it. But the majority of perl generated in bioinformatics are basically advanced shell scripts and should be immediately comprehensible. This is rarely true, mostly because there's so many ways to do things that everyone uses their own personal conventions. Learning perl 5 is a bit like collecting pokemon.
As for the Perl 6 features, I had already looked at the list and was hoping you may have some insight as to why these might be killer features for bioinformatics in particular. e.g I know what currying is, but I don't think most computational biologists will care. I actually can't see anything there that says, "Science, do it here".
Julia is a very promising language due to its focus on mathematics, readability & performance and seems to be gaining traction. R has an unbeatable set of statistical libraries. Python is now deeply embedded and has IPython, and links back end to web very nicely. C/C++ & Fortran are are miles out in front for number crunching. Java is excellent for reusable code and distributed development, and can be used in almost any layer and has proven very difficult to dislodge as the general purpose language. At the moment I don't see a niche being available to Perl 6, unless something really useful for scientists (like IPython notebooks) is brought to the table. If they really backed it as a parsing language, and provided some really sweet tools for handling biological data files - e.g. something more like an interactive IDE rather than having to write out a script in emacs. Actually give me that, I'd be very happy. But I honestly think it's the associated tools & ecosystem which will determine if Perl 6 can succeed, not a laundry list of features.
And will it ever come out? :) I remember going to a Perl learning course in 2002 where the instructor was very excited about the new version 6, which was going to be out by 2004.
Ermmm I wrote a whole blogpost on some of the killer features for Perl 6 in bioinformatics :P There are a lot of primitive types that are useful for scientific programming. A lot of my bioinformatics work at least gets reduces to bag models and set operations on big files quite often. On the computational side the stuff with native arrays is being built right now, so it remains to be seen how well that will pan out. But in theory you will get an array of native types that will be about as efficient as C. So that's like numpy included as a core part of the language. I left out the nice concurrency/parallel model and native code integration too which are really killer features in general. The ability to write first class domain specific languages that integrate fully into the compiler is kind of incredible. An example is the recent SQL Slang someone hacked together https://github.com/tony-o/perl6-slang-sql/ . You can just inline write a SQL statement and it acts like a for loop over the results returned. Kind of cool!
Thanks for the interesting reply. At the moment I guess it's a case of wait and see. I think it's fair enough to have some non-specific skepticism about Perl 6 given the history, however it's clear the designers are putting a lot of thought and love into it. And I can see how it will provide a powerful means of generating pipeline-type analysis tools.
However, it actually makes it seem to me that even more so Perl 6 makes the same "mistake" as Perl 5. The surface area of the language now seems fractal in complexity, and unless rigorous discipline & best practices are applied it's going to be very hard to build working complex projects. So great for individuals, very powerful for highly disciplined teams, but not good for the below average programmer - as 90% of biologists are. Perl 5 was full of magic, but in the hands of most biologists & less disciplined bioinformaticians that was a bad thing.
Take that SQL Slang example, great for quick scripts, but it's seems a long long way from a production ready library that can be used in a complex application. And maybe I just don't see the great advantage in being able to hide a couple of lines that make it exactly clear about what you're doing. If I'm maintaining code I want to see those explicit statements if possible. And really it's not exactly a big win over "&sql($statement)".
Anyway, it's interesting and luckily I can observe from a distance now.
I don't think they ever gave out a date when they wanted a finished version out.
"Larry Wall and other active Perl porters and Perl helpers met on Tuesday afternoon at Perl Conference 4.0 and mapped out a what is planned to become a complete rewrite of Perl that will become Perl 6 in 18 to 24 months."
-- Linux Today, byline July 19, 2000http://www.linuxtoday.com/developer/2000071901704OSSW
Perl is nothing but corner cases and magic context sensitive punctuation. If you think Python is as hard to understand as Perl, then you've never used Python.
I think that statement was really more of 'The remainder is left for the users as an assignment' kind of statement.
>>To accurately follow someone else's perl requires either an encyclopedic knowledge of the language and it's edge cases or very heavy documentation and a set standard of coding practices.
To accurately follow anything non-trivial written by anybody in any programming language will require you a good grip over the language and its corner cases. There is no magic here, if a language doesn't offer facilities to heavy lift a certain thing you will end up writing those yourself.