
Perl and Undecidability (2008) - gfredtech
http://www.jeffreykegler.com/Home/perl-and-undecidability
======
SonOfLilit
C++ is also undecidable, by the way:
[http://blog.reverberate.org/2013/08/parsing-c-is-
literally-u...](http://blog.reverberate.org/2013/08/parsing-c-is-literally-
undecidable.html)

Perl was a great language design lab experiment. They gave people 20 ways to
do any simple thing, and then Matz and Guido looked to see which ways became
popular and designed great languages that allow only those ways and maybe 1-2
more that are highly frowned upon. I'm almost as glad Perl exists as I am that
I never needed to learn it.

~~~
eesmith
I don't think the language design influence of Perl on Python was as strong as
you imply. (Not that there's no influence.)

Certainly the influence on Ruby was larger.

Also, I seem to recall the Perl5 object model was influenced by Python.

~~~
dagw
_I don 't think the language design influence of Perl on Python was as strong
as you imply._

I think the influence was mainly in Guido looking at Perl and realizing he
wanted a language that didn't look like that.

I'm pretty sure the Guido has said that ALGOL 68 and Pascal was probably the
biggest positive outside influences (together with some in-house language
called ABC that he was using at the time)

~~~
eesmith
When you say "that didn't look like that", are you presenting a conjecture?
There was only a few years between the first release of Perl and the start of
Python.

van Rossum's 1993 paper at
[http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=51C...](http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=51C37EEE34FBBA583C3615F4BF3A9CB4?doi=10.1.1.38.2023&rep=rep1&type=pdf)
explicitly lists C, ABC, Modula-3, Icon, C++, and Smalltalk as influences.

The section "competitors or precursors" also includes Perl and Tcl. It does
not appear these languages are influences.

~~~
dagw
The statement about Perl came from an interview with him I read.

------
c0m0
For the people out there that's not so into CS, and does not understands it's
implications. It basically means you could write a perl program that could
cause an infinite loop in the interpreter (or compiler). This I would presume
independant of the actual implementation of the compiler and intrepreter as
that would only mean this is a bug report for the specific implementation, and
not a problem with the language design.

~~~
knome
perl works fine, and you aren't going to unexpected trigger an infinite loop
due to some bug in the language definition.

What you can't do is tell how to parse a piece of perl without running the
program.

[http://www.perlmonks.org/?node_id=663393](http://www.perlmonks.org/?node_id=663393)

These are an expounding upon this link. In most languages, you can at least
tell what things are. This is a function. That's a variable. In perl, there
are constructs whose parsing depend on what a given bit of code was defined as
previously, and that definition can be predicated on code execution. This
leaves you in a situation where you cannot tell if something is a division and
comment or a regex without knowing whether a function takes arguments. ( see
link )

~~~
c0m0
If you can't in general parse a piece of perl without "running the program" as
you yourself mentioned then I'm pretty sure that parsing it can definitely
trigger infinite loops as perl language is definitely turing complete during
run time. So your statement is a contradiction.

~~~
rcthompson
The comment said you won't cause an infinite loop during parsing due to a bug
in the language definition, not that you can't do so at all. Perl lets you run
arbitrary code during parsing, so of course it's possible to run an infinite
loop while parsing. But you can only do that by actually writing an infinite
loop in your code, not by writing something that confuses the parser into
looping forever.

------
esaym
Perl is awesome. I bring in 90k+ a year making a living on perl while only
working remotely ( and living in a small city where the avg household income
is <$50k year).

Honestly, the only reason python "won" was because google picked it up for
internal use (and that was only because of the perl5/6 debacle since why pick
perl5 since 6 would be coming out by next Christmas?)

As far as "undecidable", its a feature and one of the reasons why it still
remains faster than most other dynamic scripting languages.

~~~
jandrese
IMHO, Perl was hurt more by its sigils than the protracted P6 development.
There was a lot of revulsion to sprinkling your code liberally with $, @, and
%. Many many comments about writing unreadable line noise when programming
Perl.

Python's genius was to force programmers to indent their code, making sure it
had at least a modicum of structure.

~~~
yellowapple
I still don't understand that revulsion. Yeah, it looks "ugly", maybe (a valid
but very subjective opinion), but it also makes it obvious that something is a
variable and - specifically - whether that variable is singular or plural at a
given moment.

The "Perl is indistinguishable from line noise" meme came about from folks
equating JAPH-style Perl Golf one-liners with all of Perl. It had very little
to do with the use of sigils specifically; such "line noise" is perfectly
possible in Ruby and Python.

If anything specific to Perl contributed to the "Perl is line noise" meme,
it's probably the fact that a bunch of special variables have names consisting
of punctuation marks. Nothing stopping the programmer from doing a "use
English;", though.

------
hpcjoe
[not trying to hijack the discussion, but I think a meta-discussion is in
order based on comments I've seen on this article]

The article is about whether or not you can actually parse Perl without
running Perl to parse itself. One of the benefits/drawbacks to Perl is that it
lets you run the interpreter at compile time, thus resulting in the
possibility of an infinite loop preventing compilation. Which makes some
people unhappy.

Oddly enough, this reminds me of the whole strongly/weakly typed discussion,
with people weighing in and asserting one position or another, without really
adequately comprehending the opposing position.

The meta-discussion here repeats almost every other discussion of Perl I see
on HN and elsewhere. First there is a claim of death, either in the past, or
present. Then there are claims that death was by sigil, line noise, etc.
Further, additional claims are that other tools have taken over its space.

The data used as evidence are StackOverflow analyses, or Tiobe scores, or,
insert additional popular, and often self-selected, data sets. You have to
make specific assumptions about some of the data presented to be able to
accept it, such that each community has about the same rate of people
searching for answers on SO, or other places. Or that the searches will have
relevant terms in the in each case.

This is a stretch to put it mildly. If I google for DBIx::Simple, and this
search is caught by one of these filters, will it show up in Perl or not? I
can't actually answer this. I have to refer to what the collectors of the data
say on their own methodology [1][2]. I am not saying that there are not
secular changes throughout the industry, or that various observed gross trends
are "wrong". What I am saying is be careful reading into these analyses too
strongly, as they may not be measuring what you think they are measuring.

In many cases, over the last 20 years or so, I've seen folks pushing Python
happily talking up why they left or abhor Perl, usually saying/quoting things
they've heard. From my own experience, Perl 4 and onward, much of what they
complain about was Perl 4 or before. Likely before many of them started
programming. That is speculation on my part, but it does appear to fit what
I've observed.

If I extend this out to operating systems, I have people tell me how horrible
Linux is and how much better anything-other-than-linux is than it on a very
regular basis. They like to list (what they perceive to be) the faults, quote
SO and other random websites as examples, and often make pronouncements not
backed up by objective fact ... in many cases contradicted by objective fact.

We as a society of technologists seem to like our tools to the point where we
feel that we can and should break into tribes with tags of honor around our
necks, and nasty comments about alternative tools, and users of said tools.
I've personally heard many an anecdote and critique of various tools from
otherwise smart technologists that were, at best, badly misinformed, and at
worst ... not simply disingenuous, but dishonest.

Look, it is great you like your tools, your operating system, your language.
It does not mean that you are "better" or "smarter" than someone else because
you use such things. And yes, I've had professional discussions as recently as
last week on exactly this issue. Which is insane.

Maybe I've spent too many years on the business side, as I look at all of
these things as tools to accomplish a goal, and as engineers, our jobs are to
select the right combination of tools to a) minimize effort, b) maximize the
possibility of success, c) enable debugging, observability, supportability.

No single tool, OS, editor (yes, I went there) has this. Moreover, if you need
to bash what someone else is using simply for self gratification, then there
are deeper issues afoot than simply a technological consideration.

Finally, I've been a user of Perl, and contributor to (CPAN) for more 20
years. Rumors of its demise, are greatly exaggerated. It is not my be-all/end-
all language ... I am comfortable and competent in 5-6 at any one time, and
can easily work in Python, C, Julia, Node, etc. w/o major issue (though with
google nearby for things I don't have on the tip of my memory).

I don't use Perl for machine learning (though I could). I don't use it for
numerics (though, again, I could). I don't use node for either of these. You
pick the right tool for the right job. And you need to keep an open mind
throughout the process on what the right tool is.

If you feel a need to bash on others choices, it might be worth reflecting why
you think this course of action actually adds any light to a discussion. From
what I've seen, it only adds heat.

[1] [http://blog.codeeval.com/codeevalblog/2016/2/2/most-
popular-...](http://blog.codeeval.com/codeevalblog/2016/2/2/most-popular-
coding-languages-of-2016)

[2] [https://stackoverflow.blog/2017/05/09/introducing-stack-
over...](https://stackoverflow.blog/2017/05/09/introducing-stack-overflow-
trends/)

[edit: fixed ref spacing]

~~~
lmm
Some tools are appropriate for particular situations. Most are better off
being entirely replaced. There are thousands of programming languages out
there, while the ideal toolbox would contain dozens at most; our industry
badly needs some pruning. We need to have these conversations, and they're not
going to be easy precisely because we're emotionally attached to our tools,
but frankly if you're really so detached as you claim you shouldn't have a
problem with people attacking Perl. These discussions are not as evidence-
based as we'd like because the evidence simply isn't there, on either side;
anecdotes are not the ideal way to learn, but they're a lot better than
nothing.

> The data used as evidence are StackOverflow analyses, or Tiobe scores, or,
> insert additional popular, and often self-selected, data sets. You have to
> make specific assumptions about some of the data presented to be able to
> accept it, such that each community has about the same rate of people
> searching for answers on SO, or other places. Or that the searches will have
> relevant terms in the in each case.

> This is a stretch to put it mildly. If I google for DBIx::Simple, and this
> search is caught by one of these filters, will it show up in Perl or not? I
> can't actually answer this. I have to refer to what the collectors of the
> data say on their own methodology [1][2]. I am not saying that there are not
> secular changes throughout the industry, or that various observed gross
> trends are "wrong". What I am saying is be careful reading into these
> analyses too strongly, as they may not be measuring what you think they are
> measuring.

No measure is perfect, but we're seeing similar trends from multiple sources
and they align with my own experience as well. I don't see any value in
throwing shade on the measures we have. If you have specific criticisms or
reasons they might systematically favour one language or another then by all
means post them; otherwise this reads like rearguard sowing doubt because you
don't like the results.

> In many cases, over the last 20 years or so, I've seen folks pushing Python
> happily talking up why they left or abhor Perl, usually saying/quoting
> things they've heard. From my own experience, Perl 4 and onward, much of
> what they complain about was Perl 4 or before. Likely before many of them
> started programming. That is speculation on my part, but it does appear to
> fit what I've observed.

Now you're the one bashing for self gratification. Is it so hard to believe
that people might genuinely have bad experiences with your preferred language?

> I am comfortable and competent in 5-6 at any one time, and can easily work
> in Python, C, Julia, Node, etc. w/o major issue (though with google nearby
> for things I don't have on the tip of my memory).

Those are pretty similar languages, or at least admit a similar style of
programming; if you want a good toolbox you'd do better to learn a smaller
number of more radically different languages. 5-6 languages is far too many to
know particularly within a narrow range like this; I'd bet that you're not
writing idiomatic code in many of them. I used to pride myself on knowing a
large number of languages; now I've realised being able to do more in one
language is much more useful.

~~~
ionforce
Where did you get the phrase "throwing shade on" from?

~~~
lmm
No idea. Picked it up from general conversation.

~~~
ionforce
You're not using it correctly. You don't throw shade /on something/, you throw
shade /at someone/.

~~~
Ultimatt
In British English at least to cast shade on is a perfectly common idiom.

------
jerianasmith
As far as undesirability is concerned, there are other languages, perl is not
the only one. But it's possible to write clean perl.

------
rurban
The same holds for every better dynamic language with compile-time evaluation.
E.g. LISP with its reader macros. Even if LISP is trivially parsable, it still
provides parse-time hooks which can lead to undecidability.

It's a feature, not a problem.

------
chubot
Related: _Parsing Bash is Undecideable_

[http://www.oilshell.org/blog/2016/10/20.html](http://www.oilshell.org/blog/2016/10/20.html)

Parsing POSIX shell is not undecideable, but bash adds a construct that relies
on dynamic parsing, much like Perl.

(I cite this Perl article and the C++ one in this thread. And there is another
one about parsing GNU Make.)

FWIW, Larry Wall has mentioned many times that Perl 6 fixes this problem. So
he very much views it as a language design flaw. I don't think you can argue
that it's not.

I watched 3 or more videos on Perl 6 and he's mentioned it at least twice.

------
cutler
It's almost 2 years since Perl 6 was released and it's still dog slow at what
Perl 5 was famous for - string parsing with regular expressions. A 19Mb Apache
log file on my 2010 quad core Mac takes Perl 6 21 secs. to search for lines
containing 15-character words compared with Ruby: 4secs, Perl 5: 2.4 secs. and
PHP7: 0.8 secs. We keeping hearing how the optimisations are coming but I
haven't seen any significant improvements in string parsing since Perl 6 was
released in December 2015. How can you market the advantages of a new version
of a language when it can't even match its predecessor?

~~~
Ultimatt
Because its faster at OO than Perl 5 and has a tonne of other builtins that
make it easier to write algorithmically performant code. Not to mention its
trivial to write parallel code. I dont think there have been any optimisations
in the regex engine in the time you're discussing. The raw IO however has seen
huge improvement, especially if you state you arent using unicode and don't
want grapheme normalisation. In 2011 just my tests took 35s to execute. Now
that's 1s and startup time is 2/3 of that. Something like CSV parsing or even
log parsing where you split instead of regex is competitive. Last I checked
faster than Ruby but slower than C. But yeah you do still have to work around
the slow bits, or be explicit with what you want like ascii IO.

~~~
cutler
Faster OO but dog slow regexes? This is Perl, right? The Practical Extraction
and Reporting Language where extraction is largely the work of regular
expressions and, now, grammars. OO languages are a dime a dozen so how can OO
be Perl 6's main priority?

~~~
b2gills
To be fair Perl 6 returns structured data from regexes, whereas Perl 5 returns
flat data. That has huge implications on how different the regex sub-languages
gets parsed. It also has a significantly different syntax, and integrates more
into the rest of the language. Basically it had to be created from scratch, so
that would mean that it couldn't benefit from the already existing
optimizations of the Perl 5 regex engine.

Perl 6 is also a mostly unpaid volunteer effort. So everybody gets to choose
what they work on, and very few people are brave enough to work on that aspect
of the compiler.

Also I would say that the main priority of Perl 6 is to integrate many ideas
from many languages and bring them together in such a way that it seems as if
all of these ideas always belonged together.

The reason why the object system sees more optimizations is because everything
is, or can be seen as, an object. A big improvement there, can for example
make regexes faster.

------
jwilk
It's worse than that. You can't parse Perl without running arbitrary Perl
code:

[https://www.perlmonks.org/?node_id=663504](https://www.perlmonks.org/?node_id=663504)

~~~
arnsholt
So? It's not that uncommon for highly dynamic languages to do this. Lisp
macros for example depend on being able to run arbitrary code at compile-time.

~~~
deathanatos
Is it? (I honestly have no idea about Lisp.) It isn't _just_ about being able
to do stuff at compile time: In Perl's case, as I understand it, you can't
determine if a string of text is a valid (in the sense of generating an AST!)
Perl program _without_ executing it. This precludes things like reliable
_syntax highlighting_.

Languages can include meta-programming/macro facilities without effecting
whether or not the parser can do its job; I am fairly certain that Rust's
macros are an example of this. Python, as a more dynamic language, also
includes metaprogramming facilities (metaclasses, decorators) but IIRC, can
still be parsed without needing to simulate a Turing machine.

~~~
kbp
> Is it? (I honestly have no idea about Lisp.)

In Lisp you can modify the read table at run time, so you can do whatever you
want. The functions the reader calls when it comes across '(', ')', ' ', and
every other character are all completely customisable.

------
grabcocque
You know there was that recent Stackoverflow analysis that concluded that Perl
was the most disliked programming language?

It got me to thinking about how Perl managed to become quite so profoundly
disliked, and I remembered these papers and thought that maybe things like
this are the reason.

~~~
pjc50
Perl used to have fans with meetups and so on. I've not heard of these for a
long time.

IMO three things killed Perl, leaving it as an unpopular legacy husk:

\- loss of ecosystem. The high points for Perl were CGI.pm and its use as a
"super awk" by sysadmins. The first was obliterated by other ecosystems,
better and worse: PHP, Rails, Node, Go, and so on. The second was obliterated
by "servers are cattle not pets": people have moved from meticulous hand-
administration to the use of containers, Ansible, etc - or away from systems
administration altogether to AWS or "serverless".

\- Perl 6 transition. Second-system effect at its highest. The long wait for
this to be completed absolutely destroyed incremental improvement because
everyone was waiting for a big bang that came very late. Python managed to
avoid this level of community damage but has still split into two languages,
Python 2 and Python 3.

\- the "two Perls" of style; one was the Wall-influenced style that looked a
bit like English. The other was sigil-heavy and incomprehensible. Eventually
people decided that it was easier to put up with syntactic whitespace than
remember all the $? and $| and so on, so a lot of the Perl audience moved to
Python.

~~~
dozzie
> IMO three things killed Perl, leaving it as an unpopular legacy husk [...]

I don't really think it was any of these. There's much simpler mechanics at
play here: Perl's competition (Python and Ruby) has about the same
expressiveness, but is much easier to learn, so for a long time very few
people have chosen Perl to learn. There was simply not enough young blood to
replace old timers that were retiring, dying, or migrating to other languages.

~~~
yellowapple
I find that interesting, since of those three, Perl was the first language
that really "clicked" for me (Ruby was almost that language, but it didn't
start clicking until after I had learned Perl; Python still ain't really
clicking, though it's gotten a bit better lately).

