
Why does OpenBSD still include Perl in its base installation? - mfontani
https://marc.info/?l=openbsd-misc&m=159041121804486&w=2
======
qalmakka
Perl 5 is in my humble opinion a really convenient to have installed. I use it
all the time as a replacement for those shell scripts that become too big or
too cumbersome, or to remove out of the equation the quirkiness of different
implementations of standard tools (i.e. GNU sed has certain flags that BSD
does not have, gawk is different from mawk, etc). A Perl onliner is often
faster and works the same way pretty much everywhere.

~~~
inglor
That's how I use Python for the last few years.

My scripts end up longer and more verbose typically (I might just not be great
at Python) but also a lot more maintainable (I am not great at writing
maintainable Perl and I use Python more often).

I also use Node.js for this a lot because of the ecosystem containing so many
accessible scripts and dependency management while not great is heaps better
than most other ecosystems.

Of course OpenBSD can't do that with Python rather then Perl (as explained in
that email). They have a different set of constraints. I am only saying in
practice I have found Python to be the better alternative for my personal
stuff.

~~~
rwmj
Python's subprocess is a bit more clumsy than Perl's built in features for
running commands, with the proviso that running commands in Perl is best
described as "idiomatic". Maybe if you didn't learn it 20 years ago like I did
you'd have a hard time with its irregularity and colloquialisms.

Is there any non-shell scripting language that makes it as easy as shell to
run external commands, redirect I/O etc.? I don't know of any. (Perhaps that
is the very definition of a shell.)

~~~
lmm
TCL ships with tclsh which is regular TCL (so a first-class programming
language) but allows you to invoke external commands as though they were
functions (so arguably is a shell in your sense, though I wouldn't advise
using it as your login shell).

------
tyingq
My guess is that it's because Perl is very comfortable for a team that mostly
writes C on a Unix like platform.

The general syntax, operators, etc, are close, and familiar things like
stat(), gethostbyname(), etc, work almost the same.

Edit: One of their Perl scripts for reference:
[https://github.com/openbsd/src/blob/b66614995ab119f75167daaa...](https://github.com/openbsd/src/blob/b66614995ab119f75167daaa7755b34001836821/libexec/security/security)

And pkg_add, which has quite a lot of Perl:
[https://github.com/openbsd/src/tree/master/usr.sbin/pkg_add](https://github.com/openbsd/src/tree/master/usr.sbin/pkg_add)

------
jedimastert
> \- you need something under and acceptable licence, so python is out.

I'm not familiar with these types of things, can someone shed some light on
this comment?

> awk would kind of work, except it's not that readable

I don't think I've ever heard someone say a plus side of perl is it's
"readability". And this is coming from someone who really enjoys perl

~~~
dijit
Marc Espie elaborates a bit on this in another post on the openbsd-misc
mailing list[0]:

> As for the license, python’s license appears fairly similar to Perl’s
> artistic license. I would worry a bit about the strong terms in

>> 6\. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

> for which no equivalent is visible in Perl’s license.

[0]: [https://marc.info/?l=openbsd-
misc&m=157781106212088&w=2](https://marc.info/?l=openbsd-
misc&m=157781106212088&w=2)

~~~
luckylion
What exactly does that part mean? If you're in violation of the license, you
cannot correct the violation and be compliant again, but the license will be
revoked immediately and there's nothing you can do about it?

It's strange though, they also have

> 8\. By copying, installing or otherwise using Python 3.8.3, Licensee agrees
> to be bound by the terms and conditions of this License Agreement.

which feels very 90ies-EULA-style. If I download ("copy") an archive that
contains Python, I should be bound by that License Agreement? Since there's
not severability clause, wouldn't that just void the limitations they've set?

~~~
maxerickson
I think in practice, if you corrected a violation you'd just be using the
software under a different grant of the license.

------
CalChris
_you need something under and acceptable licence, so python is out._

Can someone explain the difference here? OpenBSD is BSD licensed [1]. Python
is a BSD-style, permissive free software license which is compatible with the
GNU General Public License [2][3]. Licenses are always contentious, but how
does this rule Python out? (Granted, the dynamic libraries technical point
makes this kind of a moot question.)

[1] [https://www.openbsd.org/policy.html](https://www.openbsd.org/policy.html)

[2]
[https://docs.python.org/3/license.html](https://docs.python.org/3/license.html)

[3]
[https://en.wikipedia.org/wiki/Python_Software_Foundation_Lic...](https://en.wikipedia.org/wiki/Python_Software_Foundation_License)

------
fpoling
It is interesting that PHP is ruled out as insecure. I am not sure that for
system administration tasks modern PHP is particularly worse than Perl and
these days more people know PHP than Perl.

~~~
aganame
No, it's not interesting at all. PHP is fairly good at web things, but that's
it.

~~~
mtnGoat
i dunno, i wrote PHP scripts for the CLI that run a $100m+ business and
execute many millions of times per day. PHP is great for things not web-
related, i think many just didnt take the time to learn.

~~~
nineteen999
Does PHP have anything approaching CPAN? That is one of the key reasons I
still dust off Perl for quick and dirty things from time to time, as well as
the compact and accessible regex syntax.

~~~
smacktoward
Yes: [https://packagist.org/](https://packagist.org/)

------
juangacovas
A pretty amount of tools still use Perl under the hood and I'm pretty sure
some of the "granted, they just work" tools that we're used to be just there
or install, use Perl in some way.

Two examples I can give in my case are the good old rsnapshot, and CSF
firewall

~~~
mfontani
GNU Parallel is also a very useful tool, and it's written in Perl 5.

~~~
xyproto
A useful tool with a highly annoying message to the user if you're using
Fedora.

~~~
mfontani
That's odd, but interesting. What warning?

Is it Fedora-specific?

~~~
xyproto
It's specific to the tool, but most distros disable the message.

> If you want GNU Parallel to be maintained in the future, and not just wither
> away like so many other free software tools, you need to help finance the
> development.

etc

For more details, check out the Arch Linux patches that patch this super-
annoying message out:

[https://git.archlinux.org/svntogit/community.git/tree/trunk?...](https://git.archlinux.org/svntogit/community.git/tree/trunk?h=packages/parallel)

The message appears every time when using GNU parallel unless a very long flag
is given or the executable has been patched.

I think it goes against the philosophy of most open source software that comes
with Linux distros, where you are given software freely, but pay back by
contributing to the overall open source community, if you can and want. (This
is my subjective interpretation of the matter).

~~~
mfontani
I see. I mostly use Debian, and it seems that their "parallel" package removed
the notice.

More info at:
[https://sources.debian.org/patches/parallel/20161222-1.1/](https://sources.debian.org/patches/parallel/20161222-1.1/)

------
cutler
Coz Perl rocks. Long live The Camel.

------
pvaldes
If you had used Postgresql you may be familiar with psql, createdb, createuser
or pg_createcluster. All are perl scripts.

So if you remove Perl, either you renounce also to include Postgresql in your
distro, or you would need to rewrite a non trivial amount of scripts in a
different language to restore Postgresql support.

~~~
dchest
This post is about base system, which doesn’t include postgres. Obviously, if
you don’t have Perl in the base, Postgres package could just include Perl as a
dependency.

~~~
pvaldes
Then it is a different problem. Thanks for the clarification.

------
bjarneh
It's nice to read something positive about Perl again

------
pwdisswordfish2
OpenSSL/LibreSSL still requires Perl to compile.

Normally I do not have Perl installed, to conserve space.

NetBSD thankfully has always done without a large scripting language, only
recently have they included Lua. I wonder how long it would take to rewrite
the OpenSSL perl scripts in lua.

~~~
clort
> recently

10 years ago

~~~
pwdisswordfish2
First 17 years without a scripting language.

------
lmm
How is TCL less secure than Perl? I've seen a lot less weird surprises in TCL
code than Perl code.

~~~
microtherion
* In my recollection, Tcl relies a lot on string interpolation and run time evaluation, both of which carry some risk.

* Perl has a built in taint checking facility which is quite good at preventing the naive use of non-vetted user data; I don't recall whether Tcl has anything similar. [https://perldoc.perl.org/perlsec.html](https://perldoc.perl.org/perlsec.html)

------
tosh
context: the start of the thread is about the base system

> I was wondering if I need the package manager in the minimal installation of
> the system as I only use built-in deamons (httpd, sshd) and UNIX utilities
> (vi, sed)? By package manager I mean pkg_* executables as well as its
> dependencies (most notably Perl). (The size of /usr/libdata/perl5 is about
> 50MB on my machine). I just want the minimal installation without any
> unnecessary scripting language.

> Looking at FreeBSD for a moment it seems like Perl left the base system in
> May 2002 (18 years ago)

[https://marc.info/?l=openbsd-
misc&m=159005284016345&w=2](https://marc.info/?l=openbsd-
misc&m=159005284016345&w=2)

------
protomyth
I have always wondered what a scripting language designed with the values of
the OpenBSD project would look like. It would interesting to sit down and
think how pledge, veil, and privilege separation among other things would
effect the design.

------
danieldk
I have never programmed Lua, but it also seems to fit the bill fairly well.

\- Acceptable license? MIT

\- you need something that builds everywhere. It probably does? At least it
compiles with Turbo C 1.0.

\- Modicum of security: probably.

Lua is also small compared to Perl, is easily embeddable, and seems to have a
sane syntax compared to Perl.

I do understand why they use Perl though, probably more OpenBSD
developers/users are familiar with it.

~~~
tyingq
I suspect they would need the lua-posix extension, which appears to be
maintained by a fairly small number of people. Lua doesn't have some basic
stuff they would want, like stat(), chmod(), sleep().

Edit: Sort of makes me wonder why a VERY C-like, small, dynamically typed,
scripting language never emerged. Squirrel is kind of close, but not quite
what I mean.

~~~
thesuperbigfrog
>> Sort of makes me wonder why a VERY C-like, small, dynamically typed,
scripting language never emerged.

Csh
([https://en.wikipedia.org/wiki/C_shell](https://en.wikipedia.org/wiki/C_shell))
was the de facto standard shell on many Unix systems and still is widely used.

However, it has some issues:

[http://www.faqs.org/faqs/unix-faq/shell/csh-
whynot/](http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/)

[https://www.grymoire.com/Unix/CshTop10.txt](https://www.grymoire.com/Unix/CshTop10.txt)

------
Tomte
Why is TCL out? Is it about a perceived bad track record or about inherent
properties of the language?

~~~
rwmj
I guess the general answer is that _everything_ is out because they have a lot
of Perl already and not a lot of developers so why waste that time rewriting
everything?

Having said that I wrote a bunch of Tcl last year for a speech[1] and the Tcl
community is nowadays very small. It was hard to find answers to some
questions online: the active period of the community predated Stackoverflow so
there's not much on SO. And the library support is very thin compared to the
giant Perl/Python/Javascript/... communities. None of these things are reasons
why Tcl is bad - I happen to like it and my talk was a success partly because
of Tcl - but it may be a problem for OpenBSD to adopt it.

[1]
[https://www.youtube.com/watch?v=9E5A608xJG0](https://www.youtube.com/watch?v=9E5A608xJG0)

~~~
Tomte
I meant specifically about security. Because everything is a string?

~~~
rwmj
I doubt it's much different than Perl. Both languages have very loose type
systems. In fact the internal implementations are rather similar since both
use strings but optimize numbers (IVs etc in Perl which are visible to the
programmer, in Tcl the bytecode interpreter optimizes ints/floats to an
internal representation but at the language level everything still looks like
a string).

Perl's tainting feature is the only security-related advantage over Tcl that I
can think of right now, but it's not used unless you explicitly enable it.

------
eruci
Why not? Perl is an awesome tool for both sysadmin and web dev, two of the
main tasks I use it for.

------
mister_hn
Why not?

~~~
dijit
It's a good question because OpenBSD (rather famously) chases simplicity.

Perl is antithetical to simplicity, often jokingly referred to as "write-only
software". Not to mention the myriad of packages that usually end up being
brought in (thanks CPAN!).

So, it's a fair question, not just "why is there a perl interpreter?", which
is a decent question given openbsd's overall lack of shipping with much; but
also "Why are large parts of the glue code written in perl?"

 __EDIT: __I know it 's frowned on to talk about downvotes, but I received
quite a few and I'm not sure why. Please leave a comment if you're going to
flag a comment.

~~~
DFHippie
So here's a theory about the downvotes.

Some person P likes technology X, but they are aware that others consider it a
sport to dump on X. A rare article about X comes up in a popular forum. P
clicks on it with dread to see how X is being dumped on this time. But it says
positive things! P's dread is replaced with optimism. Then P reads the typical
dumping-on-X post, which says something like "X is a hideous monstrosity. I
fart in its general direction" or "X is antithetical to simplicity, often
jokingly referred to as etc". P's dark mood returns and P does as people do
when they feel like someone has been unnecessarily cruel and provoking: they
press the shun button.

~~~
hpcjoe
Alternative theory.

Perl aficionados have a long history of being on the receiving end of passive
aggressive language zealots. Usually they are Pythonistas, but others have
joined in as well.

You see this in messages that say "perl is a write once language" or "its too
complex" or "line noise" or "why is it still alive when X is better" ...
Really, its tiring after 25+ years using it. So, in an effort to express a
valid and terse critique of the comment, the respondent does the right thing,
and down votes the specific comment making those, frankly, inane points.

From there, people draw up interesting theories as to why the passive
aggressive behavior was down voted.

The reality. Perl is a modern, highly capable, fast, easy to use language. It
is comprehensible by mere mortals, while having powerful features that other
languages often attempt to replicate poorly (regex, etc.) It is portable,
running the same code across *nix, windows, macos, ... . It is fast[1] (yes
really).

It doesn't claim to solve the worlds problems, but is a tremendously useful
and portable tool, embedded in many companies products. Specifically, when
they need to get something done, and done well, perl in the toolbox is a
godsend.

Claims it is dead, hard, slow, incompatible, etc. tend to come from a vocal
minority of people with an agenda.

I don't claim my theory is correct. I do believe it fits a very long standing
pattern in language advocacy.

For me personally, I use the best tools for the job. Perl is often the right
tool, but not always. Sometimes its python. Sometimes Julia. Sometimes C/C++.
Rarely these days Fortran (moving to Julia).

This said, I am not going to bash any of the tools I use, or that others use.
If visual basic .net works for you, use it. If you need to be portable across
systems and OSes, you probably don't want to be writing things in powershell
(literally had this conversation at $dayjob a few times over the last few
months).

Languages come with dependencies and baggage. You can write great maintainable
code in Perl, and terrible not comprehensible code in Perl. As you can in any
language. Arguing that one language or the other "enables" you to do this
"more easily" is a perfect example of that passive aggressive stance that is
not helpful to anyone.

Use the right tool for the job. Perl, more often than not, is the right tool.

[1] [https://scalability.org/2020/05/on-optimizing-scripting-
lang...](https://scalability.org/2020/05/on-optimizing-scripting-languages-
and-where-they-are-useful-and-where-they-are-not/)

~~~
kbenson
> Alternative theory.

I'm pretty sure you just expressed the same theory in different words...

------
anthk
Perl5 is pledged.

~~~
notaplumber
No it's not. It has language bindings for pledge(2), which is different. There
can be no one size fits all pledge for an interpreted language like perl, as
programs can make arbitrary calls to anything they want. But those programs
can use pledge, as long as they consider the requirements of the runtime
itself.

[http://man.openbsd.org/man3p/OpenBSD::Pledge.3p](http://man.openbsd.org/man3p/OpenBSD::Pledge.3p)

~~~
anthk
That's what I wanted to mean, sorry. OFC an interpreter itself can't be
pledged.

~~~
notaplumber
Saying something is pledged has a very specific meaning. And in this context,
has no relevance to the discussion as none of the perl components in base make
use of these bindings.

------
econcon
I've never written a single line of but written python, bash and c and go.

I am not sure how important perl is.

~~~
mfontani
I would strongly recommend you do learn Perl 5 (and Raku, and Haskell, and
lua, and functional languages, and ...), even if only to be able to know
enough about them to be enlightened about what each of them gets right and
wrong.

For Perl 5, you could do worse than going through the _free_ "Modern Perl"
book, at
[http://modernperlbooks.com/books/modern_perl_2016/](http://modernperlbooks.com/books/modern_perl_2016/)

~~~
IggleSniggle
Thanks for recommending this book. Mostly I write TypeScript day-to-day, so
naturally I typically reach for nodejs when I want to "upgrade" a script from
bash to a more expressive language in a cross-platform way. I've been
frustrated with that tooling, however, for a whole bunch of reasons, at lot of
it to do with how clunky nodejs actually is in shell pipelines. I've thought
about reaching for python, but it has some similar issues to nodejs: you need
a particular environment, and some things which ought to be natural in a shell
environment just...aren't...in python. I think someone here suggested Lua, and
I just...don't care for Lua (although I haven't tried it in a scripting-like
context).

Anyhow, I've been reading this for a half-hour or so and I'm struck by how
incredibly similar common Perl idioms feel to JavaScript as it is commonly
used today. I think part of this familiarity has to do partly with type
coercion, partly with destructing assignments, partly with function
"context"(as in Perl) as it is commonly deployed in the nodejs ecosystem,
partly because of the way default arguments and function passing relates to
the default scalar variable in Perl.

I've been playing around with OCaml and Rust a bunch recently, but haven't had
any good reason to reach for them "in real life." The fact that Perl is
"everywhere" (much the way JavaScript is "everywhere" as a side effect of
browsers being ubiquitous), means that I'm likely to start reaching for Perl
instead of nodejs when I need to move beyond Bash. It really does seem to be
the right tool for the job.

So, a hearty "thank you" for the recommendation.

~~~
chromatic
You might find it amusing that "JavaScript: The Good Parts" directly inspired
the "Modern Perl" book.

~~~
IggleSniggle
That is entertaining to know!

Thank you for writing this, it seems very well done so far. And thanks for
also making it available in the way that you did: I almost certainly wouldn't
have played with Perl today without your book, made effortlessly available on
my e-reader via Pragmatic. I'm curious why it's not available in physical copy
from Pragmatic? And I'm not confident about the correct "buy" link to use to
properly support the book.

One sidenote that I hate to mention, but: I couldn't figure out how to open a
PR for this or even email someone who would care, but I found a code-typo on
the CPAN website: it's instruction to "install via App::cpanminus" and then
use with "cpanm name::module" is...an unfortunate early typo on that site.

~~~
chromatic
_I 'm curious why it's not available in physical copy from Pragmatic?_

I believe it is:

[https://pragprog.com/cart/add/skus?sku_id=821_820](https://pragprog.com/cart/add/skus?sku_id=821_820)

 _I 'm not confident about the correct "buy" link to use to properly support
the book._

It's more valuable to me if you share and enjoy. I give away electronic
versions so that it's useful, not to make money from the time I spent writing
it.

 _I found a code-typo on the CPAN website_

Is this the page you were reading?

[https://cpan.metacpan.org/modules/INSTALL.html](https://cpan.metacpan.org/modules/INSTALL.html)

It's not explained on that page, but the distribution App::cpanminus installs
a program called cpanm, not cpanminus (as one would expect).

~~~
IggleSniggle
Huh. Yes, that's what I was talking about. But `cpanm` didn't end up anywhere
in my path, and at the same time, `cpan Module::Name` successfully downloaded
and installed modules that I could use.

------
teddyh
There are many who call GNU people license zealots, and who imagine that GNU
somehow shuns BSD licences. But the opposite is true! GNU people happily use
BSD licensed code, and include BSD licenced code in the GNU “free software”
term, while it is the BSD people who, as can be seen here, reject software
like Python and GCC for having “unacceptable” licenses.

~~~
eitland
You can always combine BSD code with your GPL and release the code as GPL.

You cannot however mix GPL code into your BSD code and the resulting work as
BSD.

~~~
greggyb
As the sibling mentions, you cannot re-license code you do not hold copyright
to.

That said, you're very close to the mark. GPL applies to distributions of the
copyrighted work and executables based on it. So, the concern is around an
OpenBSD release - they are shipping a full OS with source code and compiled
work product. The GPL is "viral" in that you cannot make this release without
licensing your whole distribution as GPL. OpenBSD does not want to be GPL, and
so does not include GPL-ed code in base.

The inverse is not true. A Linux distribution can include GPL and BSD-licensed
code easily, because the BSD license imposes no constraints on redistribution.

~~~
eitland
Of course you cannot relicense someone elses git repo.

Copying BSD code and distributing it under another license (even a commercial
one) seems to be totally OK though as long as you put in the correct notices.

That is what GPL is meant to protect against.

~~~
greggyb
Right, but the GPL license's protection also means that a BSD cannot be
distributed under the BSD license (or similar) if it includes GPL components.
This is why OpenBSD excludes GPL components.

GPL enhances user freedom at the expense of restricting what programmers can
do.

~~~
eitland
> Right, but the GPL license's protection also means that a BSD cannot be
> distributed under the BSD license (or similar) if it includes GPL
> components.

I fail to see how this is different from what I wrote above (except for the
missing word in my post above):

> You cannot however mix GPL code into your BSD code and [distribute] the
> resulting work as BSD.

~~~
greggyb
I didn't pay enough attention to usernames as I was replying and thought I was
talking to multiple people through the chain.

We are in verbose (and unnecessary on my part) agreement.

