Hacker News new | past | comments | ask | show | jobs | submit login
Bsdutils: Alternative to GNU coreutils using software from FreeBSD (github.com/dcantrell)
61 points by networked on March 2, 2023 | hide | past | favorite | 82 comments



From the comments I see that there are some different coreutils implementations out in the wild, mostly done for fun.

I am very happy about this. This means that there is not much energy put into implementing new coreutils features or fixing bugs. I think this is good. This means they are more or less finished. Mature. Software that is done and usable. That is something outstanding.

This is an example of software that does not need to be in an never-ending flow of changes, not because it is abandoned, but because it is finished. I think we need more of that.


What would be an advantage of switching from GNU coreutils to BSD coreutils?

Doesn't GNU utils have more features?


Stability, readable code and unchanging flags.

GNU project code is infamously cluttered and hard to grok (mostly since it's a pretty old codebase and GNU has always favored having too many chefs in the kitchen over fewer, but competent, chefs).

FreeBSD code is also usually much more stable than the Linux counterparts; tends to be less buggy from my experience at least.

GNU also has gotten in the habit over the years of randomly shifting flags around for no real reason other than to make sure that they stay as the coreutils. They have the effective monopoly on usage (I want to say that BusyBox is the only realistic other competition due to how frequently it's used on smaller resource devices), so that way other implementers are forced to adapt. There's also have a tedency of making their software have really weird Unix pipe interop. The most infamous example I can think of is that there's one GNU coreutils command with the "human-readable" flag, which doesn't do what it usually does (adjust the output of commands displaying filesizes and the like to kb/mb/gb) but instead reformats the output to be usable in a different GNU command which also has a "human-readable" flag that works as expected.

There's a couple of reasons to not want to use the GNU coreutils specifically at least.


> GNU project code is infamously cluttered and hard to grok (mostly since it's a pretty old codebase and GNU has always favored having too many chefs in the kitchen over fewer, but competent, chefs).

I was under the impression that it was a result of 1. supporting a terrifying spread of platforms - by the time the code handles the quirks of AIX, IRIX, Minix, Solaris, all the BSDs, and oh yeah also Linux, yes it has a rather lot of clutter, and 2. specifically implementing things in a different way than commercial UNIX™ so nobody could accuse them of copyright violation.


> GNU also has gotten in the habit over the years of randomly shifting flags around for no real reason other than to make sure that they stay as the coreutils.

Do you have examples of this?


After some thinking, I do recall GNU Tar in I think the late '90s changed the bzip2 options a few times. Not sure the reasoning. Might be something in mailing lists but I don't really care to go digging.


> GNU also has gotten in the habit over the years of randomly shifting flags around for no real reason other than to make sure that they stay as the coreutils.

Sounds like they've been studying the works of The Enemy ;)


I had compared a couple of coreutils with their BSD counterparts for a personal project. Anecdotally, the BSD versions had much more readable code and less compiler/preprocessor black magic. Which also makes them easier to build with non-GNU toolchains (perhaps not out-of-the-box, but with less effort anyway). So, overall they would seem a better match for learning/diy purposes.

Poor choice of name though. Debian has a bsdutils package since forever [1].

[1] https://packages.debian.org/sid/bsdutils


I can think of a few

No copy-left license?

Compatibility on Linux with BSD-like operating systems such as FreeBSD, MacOS?

To annoy Richard Stallman?


> No copy-left license?

Are you planning to make proprietary changes to `cat` you are unwilling to share?

> Compatibility ...

GNU coreutils work fine on all the mentioned plattforms

> To annoy Richard Stallman?

Not really sure if pettiness would be sufficient motivation for me, but then again, I'm writing this response...


> Are you planning to make proprietary changes to `cat` you are unwilling to share?

Presenting to you Panthera(TM) - the Enterprise Ready cat that you were waiting for!

* LDAP support

* Office 365 integration

* SAP integration

* GDPR compliant outputs

* 24/7 SLA


Man, don't give them ideas. Have you seen systemd, pulseaudio and wayland ?


Where do I sign my soul away to get this amazing software


GPL v3 is unsable for embedded/locked hardware, e.g. Macbook, Android phones. That's why Mac OS and Android dodged GPL v3 software.


How so? If Apple chose to distribute GNU cat, they'd need to release any modifications to cat under the GPL but the license's vitality would not "infect" the rest of OSX.

Plenty of GPL software is available on osx.


No. Apple don’t ship anything that is licensed under GPL v3 [0]. In fact, they are still shipping bash 3 (the last version under GPL v2) with Mac OS [1], released 16 years ago.

The decision is also affected by the Tivoization clause introduced to GPL v3.

> If MacOS included GPLv3 software, Apple wouldn't be able to pre-install it unless they provided all users with signing keys to install their own modified versions of the GPLv3 software on-target. Check out the GPLv3's "User Product" and "Installation Information" sections for more details - they're written in plain English, and are pretty clear. [2]

[0]: https://meta.ath0.com/2012/02/05/apples-great-gpl-purge/

[1]: https://news.ycombinator.com/item?id=20102563

[2]: https://news.ycombinator.com/item?id=21819415


Apple doesn't ship bash with iOS, and you don't need signing keys to install modified versions of bash on macOS.

But Apple holds lots of patents[1], and while it's hard for me to imagine Apple enforcing patents against users of bash, I can easily imagine Apple lawyers making a strong enough case to management that the potential value of all patent licenses that would be implicitly granted by GPL 3 was likely to exceed the cost of feature-freezing or replacing affected macOS components that they would have agreed to a blanket ban.

Note that I'm not defending this decision: for the record, I detest software patents, dislike tivoization, and the only problem I have with GPL 3 is when companies that might otherwise contribute to GPL 3 projects are uncomfortable with it. Which is unfortunate, but certainly not a flaw in the license itself.

[1] https://patents.google.com/?assignee=%22Apple+Inc%22&oq=assi...


Available and used by Apple themselves are two very different things.


yes, Apple obviously hates GPL and wants it to die. But I'm confused what you're responding to. The claim was that GPL is unusable on OSX and that's obviously not the case. Whether Apple uses GPL is irrelevant to the matter at hand.


copy-left licenses are better for users.


But BSD devs have their own argument https://docs.freebsd.org/en/articles/bsdl-gpl/


Yeah. Just look what they got from that. Big corps using their work with hardly any upstream contribution (Apple, Sony, etc). At this rate BSDs will be deader than dead within a few years.


I don't know about you, but I can't eat code. That said, some people don't care about outside contributions or reuse, so I guess I find the pro-GPL partisanship more than a little misguided. Some people just have different goals, and that doesn't make them better or worse, just different.


And a project can't survive without the code.

And even then companies are famous for leeching on open source projects, with gpl you at least have some hope of companies leaving you alone or giving back something.


> leeching

How is it "leeching" if I deliberately release my source code under a BSD license because I don't care if other people use it or not? It can't be "leeching" if I don't care about their contributions either. The only other scenario that would apply is if I wanted to be paid for my work... in which case neither a BSD-like license, nor the GPL, would be adequate.

This is the problem I have with GPL partisans: They refuse to accept that projects can be completed, or that anyone can have any interests other than GPL-compatible ones.


Using a pet project nobody is expected to use as an example is a nil point, I could release a pet project in "WTFPL" or "Fuck You Nobody Is Allowed To Use" license and it wouldn't make a difference , if the concerns are that low for said project.

Monetization in FOSS is a systemic issue, with few exceptions, but even then you are far more likely that companies won't want to risk leeching with a GPL like license than a permissive one because of the code sharing requirement, which MIT/BSD at best only incentives as goodwill because of complexity maintaining the project they rely on.

I also find it funny you create a strawman, i never stated any adoration of GPL.


> And even then companies are famous for leeching on open source projects, with gpl you at least have some hope of companies leaving you alone or giving back something.

Especially when you make clear the option to dual license and let them use it under a MIT/BSD style license, or something custom, if they pay you an appropriate amount.


And that's the reason for their downfall.


> downfall

What downfall, exactly? If I want to get paid, the GPL doesn't help me with that. If I don't care about outside contributions, the GPL doesn't provide any advantage. Insofar as I have written something that is complete in itself then a BSD license makes perfect sense.


> Big corps using their work with hardly any upstream contribution (Apple, Sony, etc).

Meanwhile regular commits from / sponsored by Netflix, Dell-EMC Isilon, Juniper, Mellanox, Intel, Amazon and Microsoft (for network drivers), etc.


Netcraft confirms it.


>Yeah. Just look what they got from that.

An absolute great OS?

>hardly any upstream contribution (Apple, Sony, etc)

Sony paid FreeBSD dev's, and the Apple kernel and coreutils are opensource, it's just they have not done anything FreeBSD is interested in, not like for example Netflix with in-kernel tls.

>At this rate BSDs will be deader than dead within a few years.

Yes yes you said that already 20 years ago, it's probably time to overthink it.


Could we call it GNU-free/Linux?


There's that whole issue of the kernel.


Although licensed under the GPLv2, the Linux kernel is emphatically not GNU software.


Some people call that Alpine.


That's a subset; Alpine is a non-GNU Linux distro, but so are Android and Chimera (https://chimera-linux.org/).


People who don't know alpine does use gnu software.


The only GNU I'm aware of in Alpine is that they use gcc as the main compiler, but the built system is AFAIK GNU-free.


Not by default. Unless I overlooked something.


Precisely; that would be a good reason!


The incompatibility between the two versions of sed (I have both installed on my macOS) is super annoying.


Isn't BSD Sed a subset of GNU Sed? I can't think of any outright incompatibilities that would make a BSD Sed script invalid for GNU Sed.


Their handling of the in-place flag is different enough to cause irritation if you forget which one you're invoking.


I'd like to change it in FreeBSD, but haven't really (yet) come up with a clever way to make it work without breaking years of scripts that may be using -i.


Are you a maintainer of sed on FreeBSD?


Not sure we have anyone that could really call themselves a maintainer for sed, but I am a FreeBSD committer that has put thought into this problem in the past.


For fun, I ported much of BSDutils to WebAssembly. Code [1] and live demo [2]. It was much, much easier porting BSDutils than GNU coreutils, since the source code is often much smaller, and hence easier to read and understand with simpler dependencies.

[1] https://github.com/sagemathinc/cowasm/tree/main/core/coreuti... [2] https://cowasm.sh/


Isn't the same we have on MacOS?


Why does it need gnu cc rather than llvm?


It needs a GNU _compatible_ C compiler. You can still use LLVM's Clang.


I'm guessing that it invokes /usr/include which on Linux is probably not neutral to CPP or gnu-isms.

Your point about "compatible" is well made. I should have been clearer.

Wasn't gentoo born as Linux userspace on BSD kernel at one point?


Gentoo was born as linux. Portage, the package manager, was inspired from BSD ports. Maybe they tried to experiment with also using a BSD kernel later, as did Debian, but it was only a side project.


I think Gentoo was always about Linux with BSD-style ports. They had some additional "flavors" that switched Linux to kernels from FreeBSD, NetBSD or OpenBSD, but it seems those alternative Gentoo distros are deprecated for at least a few years already.


There’s a Rust coreutils too.


Some utils are faster already, and they are working on cp and mv progress bars. From the fosdem 2023 presentation: https://sylvestre.ledru.info/presentations/coreutils-fosdem-...

== Example #1:

$ hyperfine -r 10 \ "target/release/coreutils sort -R shakespeare.txt > shakespeare.txt.rand; \ target/release/coreutils sort shakespeare.txt.rand > /dev/null" \ "sort -R shakespeare.txt > shakespeare.txt.rand; \ sort shakespeare.txt > /dev/null" é [...] Summary 'target/release/coreutils sort -R shakespeare.txt > shakespeare.txt.rand; target/release/coreutils sort shakespeare.txt.rand > /dev/null' ran 4.63 ± 0.19 times faster than 'sort -R shakespeare.txt > shakespeare.txt.rand; sort shakespeare.txt > /dev/null'

To be fair, we can be slower too

Example #3:

$ hyperfine "seq 18446744073708551615 18446744073709551615 | factor" \ "./target/release/coreutils seq 18446744073708551615 18446744073709551615| ./target/release/coreutils factor" [...] 'seq 18446744073708551615 18446744073709551615 | factor' ran 5.08 ± 0.01 times faster than './target/release/coreutils seq 18446744073708551615 18446

===

Redox-OS is rewriting the BSD coreutils in Rust for their own OS. https://github.com/redox-os/coreutils

I wonder what the most efficient and usable alternative operating system tradeoff choice tradeoff would be, perhaps mesalock (a linux with userspace tools compiled with rust)

syllable, haiku, redox, menuet64/kolibri, serenity, icaros desktop, postmarketos, drauger, mesalock



> MIT license

How is letting me know that apple can use it without contributing back going to affect my decision on which coreutil to use?


Some people feel morally that Apple should be allowed to use it without contributing back, and are very vocal about this feeling.


>> How is letting me know that apple can use it without contributing back going to affect my decision on which coreutil to use?

Apple using those utilities could be a good thing. I've used MS PowerShell and WSL a bit, and their implementations of common CLI software is sometimes incompatible. To work around the incompatibilities you end up using proprietary features or doing something inferior. I would be better if they just dropped in the standard utilities. OTOH I understand the fear that they may provide useful proprietary extensions because of an MIT license.

Example, their version of ssh does not allow a script to hand it a password, hence my script was changed to print out "hey, type this password 'blah'" and required user interaction in the middle of an otherwise totally automated process.

Every OS should use the standard (GNU) utilities, or at least a set that aims for complete compatibility (Rust versions).


> Example, their version of ssh does not allow a script to hand it a password, hence my script was changed to print out "hey, type this password 'blah'" and required user interaction in the middle of an otherwise totally automated process.

Not sure which ssh you're referring to but openssh doesn't allow it either. There are several hacks to script a ssh login with password.


> I've used MS PowerShell and WSL a bit, and their implementations of common CLI software is sometimes incompatible

Are you implying that Microsoft would use the common CLI software interfaces if GNU coreutils weren't GPL? Because I deeply believe they wouldn't. Embrace, extend, extinguish.


They already ship curl. In doubt the license is the thing stopping it, but there is some precedent here.

WSL (v2) is just a fancy wrapper around a Linux VM though, so the tools you get there are from the distro, not Windows.



> Apple using those utilities could be a good thing.

Maybe, and maybe not. I was specifically asking why should I worry with apple's licensing considerations in my choice of tools to use.

For apple GPL license is a negative thing, they might go for a subpar solution just to avoid it. But for me? As a user, MIT or GPL license changes nothing for me.


> I've used MS PowerShell

I like the idea of setting some variable somewhere, and the Rust Coreutils output JSON, and we can use 'select' and 'where' rather than shaky scraping libraries.


If I was going to replace coreutils, I think I'd do it with the ones rewritten in rust.


30 years ago people planned rewrite stuff written in C in C++.

20 years again people wanted rewrite stuff written in C/C++ in Java.

Now people wanted to rewrite stuff written in C/C++/Java in Rust.

Meanwhile core infrastructure of humanity still runs on COBOL, C, C++, Java in that order.

20 years from now people ask AI to rewrite all stuff written in C/C++/Java/Rust and it picks ADA SPARK as implementation language.


I already thankful for an modern init- and system-management implemented in C, using plain config-files.

Sometimes being slow and careful is wise. Think of XML, hard to read and hard to parse. And an implementation language with broad support and reliability is a good thing for projects which should remain for decades.


You forgot Fortran!


20 years from now programming as we know it today will be a very specialized field and not necessary for most things, and language discussions will be a thing of the past.


I tried some of those but I'm not a fan of the frilly stuff they add by default to the outputs. I was having a hard time with the output of one of these rust tools (I think it was an ls replacement) that was pretty printing and tabulating stuff. Its nice to read I agree but it was a pain to copy and paste outputs and the option for a normal output was not default.


I think you’re conflating different projects.

There are projects that aim for a better user experience, with better command line interface, defaults, performance and UI. These are of course breaking changes and the programs can’t be used as drop in replacements. Some examples are

- ls => exa (https://github.com/ogham/exa)

- grep => ripgrep (https://github.com/BurntSushi/ripgrep)

- cat => bat (https://github.com/sharkdp/bat)

- tree => broot (https://github.com/Canop/broot)

The person you’re replying to was speaking of a different project - uutils (https://github.com/uutils/coreutils). These are drop in replacements with identical interfaces (modulo bugs).


You are right it was exa and I was conflating projects.


I was wondering how long it’d be before a certified Rust Advocate showed up. Still plugging away at it eh?


Same here. I love me some BSD but I've got too much muscle memory with the GNU userland.


whoever doing this has some serious time wasting issue. Check the previous project https://news.ycombinator.com/item?id=26303996


What's wrong with wasting time? It's their time, they have the right to waste it.


You don't get to decide how another person value their own time.


Ah yes, the infamous waste of learning, having fun, and becoming a better programmer.


Are you perhaps on the wrong website? This is Hacker News; it's a website for people interested in fun and interesting topics, mostly neat technical stuff like this project.


Says the guy who writtes nonsense on HN....bravo.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: