Hacker News new | past | comments | ask | show | jobs | submit login
Common *nix commands written in Rust (gcollazo.com)
80 points by lowonkarma 51 days ago | hide | past | favorite | 53 comments

From the ls replacement: "Why spend your time squinting at black and white text?"

Uh, because squinting at blue-on-black text is much more difficult. Human eyes focus best at yellow-green, they see that as brightest, and they don't even have long-wavelength (blue) receptors in the fovea.

I also looked at the alternatives to ps, top, cat, find, and du.

My overall feeling is that these are really not like UNIX tools at all. They are not very scriptable. Output is full of Unicode, color, column dividers, headers, underlining, and pagination. Values are in so-called "human readable" form, like "1.4M" instead of 1474560 or whatever it may be, which is a real pain for scripting.

I also notice some features that seem out of place to me, a long-time UNIX user. The tools waste space on displaying things related to git, docker, and network connections. I would expect the proper tools to be more like "git ls", "docker ls", "docker ps", and "netstat -p". (for those ones that don't actually exist, take that as a suggestion for a coding project) Not all the world is devops on a web site.

My distro's ls already does coloring by default. It turns it off somehow if it detects the output is a pipe, so that "ls | less" still works. It also turns off multi-column in that case, so "ls | wc -l" counts directory entries.

That's not fair. They typically turn off visual adornments when the output is not a tty, thus automatically supporting consumption by both human eyes and scripts.

Well, this is basically the direction that GNU tools have been growing in for a while. And I think that's because ultimately these are tools with a human interface first, script interface second (and UNIX has always been mediocre at the script interface part: shell scripts are just bad programming languages).

sd(1) looks super cute.


Very UNIX-y imo.

I think you made a typo; long-wavelength is red, and blue is short wavelength.

Yes. It's the short-wavelength receptors that are missing from the fovea. Informally, they are blue receptors, but the ranges overlap in a much more complicated way.

Terrible color combinations for text: blue/black, magenta/red, cyan/green, white/yellow.

The example uses blue/black.

There are sometimes good reasons, usually related to features although sometimes related to performance, to prefer newer utilities like ripgrep over older ones like grep. Implementation language is not one of them.

All things being equal, implementation language shouldn't matter. But they aren't equal — different languages have different trade-offs and this affects programs they produce.

A fantastic `ls` implementation in Java would not be very practical. OTOH it's not an accident that Rusty replacements of these programs tend to be multi-threaded, because that's a thing Rust encourages and makes practical.

I came here to ask, thought not outright say, what is it with all the "... in rust" stuff that keeps getting posted. It's starting to feel more like a marketing exercise than anything actually software related (happy to be corrected on this though).

“marketing” is probably not the right word since there is no money backing it, and it's not a concerted effort either. “Fashion” would fit better I think.

Rust has long been something intellectually interesting (solving memory issues found in C and C++, while keeping their low level and with equal performance is quite a feat), but it recently started to become affordable to use (libraries are maturing, there's tons of quality learning material, and even the language itself reached a level of polish that was simply not here before), then many people who have had it somewhere on their radar for a while took the leap, and now share their enthusiasm. And the interested ones who are still waiting for a good occasion to learn it upvote and then it ends up on the front page. And now, you get this kind of low-quality content, which is merely surfing on the trend to get some clicks.

This kind of things happened with Go a few years back, or with node before that, such is the fashion life of the programming world.

Rust solve memory issues of C/C++ at the cost of simplicity and with convoluted, complicated syntax. It will bring in new class of bugs when it becomes popular enough and when large parts of Firefox are rewritten in it.

I am not sure why some programmers think, complex and convoluted the syntax, the better the language, especially taking rust as an example. Go has been successful in it’s niche as systems programming language because it’s syntax is easy and can fit the brain easily. A similar simple good programming language I see is zig language which makes memory safety as a choice for the programmer to implement rather than forcing on them with complex borrow checker syntax. On top rely of unsafe C library and code to do anything useful. Zig eschews complexity and embraces C with simple syntax and semantics and provide easy tools to build memory safe code and not shoot the foot. But anyways like democracy in programming complicated languages can win due to majority by marketing or in fashion.

Zig and golang aren't solving the same problem as rust. Zig isn't memory safe, and golang achieves memory safety with the help of a GC.

Go even with GC is very popular as systems programming language in its niche of writing network servers, can’t say the same about Rust except it’s in Fashion on HN, because it looks complex and difficult for programmers. Rust rely on underlying C library for doing anything useful today and still hasn’t demonstrated any real large benefits.

Zig helps programmer with tools to write memory safe programs by not shooting in foot with sane and easy syntax and memory allocator. Still like Rust it needs to prove, so time will tell whether marketing and fashion or simplicity and sanity wins. There are enough examples where fashion and marketing with minuscule benefits did win.

> Go even with GC is very popular as systems programming language in its niche of writing network servers

Nine times out of ten a network server isn't systems programming. And even at google, golang is starting to be banned for real systems use by some projects.

> Rust rely on underlying C library for doing anything useful today

Go relies on libraries everywhere except Linux too, because that's where the stable boundaries are.

And even in Linux it was a bad plan for them not to rely on the C lib, because they miss out on other system configuration stuff like locale information because of their choices there.

> Zig helps programmer with tools to write memory safe programs by not shooting in foot with sane and easy syntax and memory allocator.

So.. the same way C does?

> * And even at google, golang is starting to be banned for real systems use by some projects.*

Quite curious. Got a source? Would like to read more on that.

When it comes to syntax, it's always a personal feeling. For instance I always found Rust syntax much easier to work with than Go's (which looks really alien to me, with both var and :=, the type annotations which are C's but backwards, the syntax for structs methods, the several for syntax, the defer statement, etc.)

Actually I don't really understand why some people complain about Rust syntax. The only alien thing is the lifetime ('a) syntax, but you don't need it that often, and the concept behind it is way more difficult to learn than the syntax itself (and it's where Rust's breakthrough comes so you have to learn this concept anyway).

>It will bring in new class of bugs when it becomes popular enough and when large parts of Firefox are rewritten in it.

It's quite hard to see how these bugs will come from 'the syntax is complicated'. For one, the syntax is simpler than C++, and secondly, complex syntax is not necessarily a cause of bugs: Rust is not prone to errors in syntax becoming runtime bugs.

People who don't like Rust like to point at Zig as an alternative, but as long as it's developed by a single guy and there's a lot of breaking changes at each release, it isn't really an option. At this point this could still change a lot and even add a borrow checker! (Rust didn't start with it, it used to have a GC back in the 0.* releases)

Yes it’s started by one man like Go by Dennis and Python by Guido and many more, only time will tell.

Only time will tell if your one-persone startup in your garage is going to kill Facebook eventually. But until it grows, it's a bit awkward to compare them.

Correct it's bit awkward given Rust is in the same league as Zig when compared wit Go language, which is widely adopted and used in many core network infrastructure systems programming, powering Internet today. So if Rust can come close to Go language, same is the case for zig language against Rust.

> * Go language, which is widely adopted and used in many core network infrastructure systems programming, powering Internet today.*

Is there a place where I can read more on this?

Look at any modern container orchestration framework most are written in golang (including kubernetes), I see many SDN control plans and data plane software written in golang. You can find cockroachdb, raft etc and many more. Rust is just a fringe language at present in fashion on HN, may be zig might replace it as fashion language one day unless there is some critical system are using it like for C, C++, Python or go.

I think it's just that Rust brings a fairly new set of semantics to system code (memory safe, but no GC), and people are pretty excited to rethink large parts of the software stack we all depend on.

That's but rust has been a known quantity for 5(?) years at least now?

Changes in the systems development space take a long time to propagate.

A year or so back I found out that I got passed over for an embedded job because I had a big emphasis for Rust on my side projects and they thought it was a toy GCed language and a weird anachronism to put on there.

known to who though? I just told a long time C++ dev about Rust during a discussion about programming languages. He'd never heard of it before, and was quite interested.

There are still quite a few folks out there who don't know about Rust, Go, or anything else that's gotten popular over the past few years here.

I assume it's just waves of popularity and people getting into particular ecosystems. A few years ago there was a similar wave of writing/rewriting software in go.

Consider this. If a program is written in Python, and becomes popular, it is usually a matter of time before it gets rewritten in, and replaced by an implementation in a faster language.

There are fast implementations (C) and there are somewhat safe implementations (Python, Java, ...). If I were to place Rust in this hierarcy - it is both fast AND safe, and there's rarely a good reason to replace it, so you've software that's already at the top of the hierarcy. This means that the particular implementation is likely to be the one that doesn't get replaced.

"...in Rust" has a lot of implications other than being an exercise in marketing.

So why not write "fast and safe implementations of popular command line utilities" rather than focusing on the implementation language? If there were a safe and fast implementation of, for example, find in Ada why should it be excluded from this list?

I don't see any reason why people who write these tools in Ada shouldn't add that to the headline as well. People who are interested in Ada would be even more likely to read the article then.

Writing fast and safe implementations of popular CLI utilities is already happening.

There's also some advocacy involved which many people obviously find distasteful. To me it's a bonus though: tell me why your tech choice makes a difference, I'll check and maybe learn it.

Anything wrong what that?

Except Rust isn't a direct replacement for c. Rust's system requirements are perhaps acceptable for what'll be normal in computers five years from now.

We don't need to re-stratify development so that only people with expensive computers with > 4 gigs of memory can compile what they run.

My laptop is a €500 bought in 2012 (it has 4Gb which is the minimum of every laptop since around 2010), and it runs rustc fine. Of course I'm not going to build a release version of any big project on it, but I won't be able to build Firefox or Chromium either on it.

You're exaggerating for effect, I assume. Rust compile does not require that much RAM.

Now, by default the "cargo" build uses all CPU threads, so if you have a 32 core Threadripper then yes, running 64 simultaneous "rustc" compiles will use 6 or 7 gigabytes.

Last I checked Rust needed more than 4GB of RAM to compile itself.

That's one of the reasons it was rejected in OpenBSD: https://marc.info/?l=openbsd-misc&m=151233345723889&w=2

clang needs more than 4GB to compile itself too.

"expensive computers"

"> 4 gigs of memory"


by my standards, $5 is not an "expensive computer".

and more to the point, the language has little to do with the compilation memory required. Firefox and Chrome require significantly more memory than that to compile nowadays; they passed the 4 GB mark that could be compiled on a 32-bit machine many years ago, and both Firefox and Chrome were written entirely in C++ at the time. furthermore, you can always enable swap. it'll be slow, but if you can't afford a $5 computer, you take what you can get.

Used computers with 4GB of RAM are cheap, but not $5 cheap. The cheapest I saw at that link was $43 including shipping, for a 2008-era system. An Ivy Bridge SFF desktop with 8G RAM is $67 shipped (I have a couple, theyre nice little systems).

The implementation language may be important if it promotes (and, with certain discipllne, guarantees) a lack of memory-access errors and a lack of data races.

This may be not as important in a replacement for `less` or `du` than in a HTTP or SMTP daemon, but still.

If the language is strongly typed and reduces opportunistic threats of mis-application or unexpected data, I'd consider that an upside.

If the compiler targets multi-cores and the language helps express implicit parallelism or asynchronous code possibilities which map to multi core, better than the original language, then thats net beneficial too.

If the language encourages coding patterns which are extensible or re-usable, then its possible the runtime code state gets smaller as increased use of shared code patterns leads in the longer term to more COW shared libraries.

Okay, so why not say "safer utilities" or "more performant utilities" rather than focusing on the implementation language? If I'm an end user of a safe and performant utility why should I care if it's written in Rust, Go, or OCaml?

Because there are a lot of programmers that read Hacker News and some of them have interests in particular things. That's why you see titles such as "Learn polynomial regression with Tensorflow" or "How do do HFT with Python".

But why do you care when it's said that it's written in language X? Does that ruin the story for you? And the potential usefulness of the tool?

This isn’t too compelling a story on its own, I’m assuming this is someone who wrote a collection for themselves and made it public on their blog.

Glad they wrote their collection, but not something I’m excited about.

In general I’d love to hear “this was better in Rust because of X, Y and Z and the tradeoffs I had to make were A, B and C”.

But again, this isn’t on the author to do, I’m pretty sure I can find other articles on the internet that talk to that.

Edit: most of the READMEs of the projects look pretty good about that on a quick skim.

What I would like to see is something like Hadoop, Kafka, Spark rewritten in Rust. Something that will fundamentally change the landscape. Rewriting these Unix cli is only good learning experience

Maybe, but consider that security vulnerabilities have been found in SSH and `sudo` / `su`, several times the last few years (and they are not the only UNIX tools, it was just the noisiest news about such vulnerabilities).

Any new programming tech that avoids my SSHD letting anyone get in because of this or that is very welcome by me.

There's Sonic, which is a searching software, although not on the level of ElasticSearch. People are working hard to bring more safety to widely used technologies, every day, we just don't hear much about it.

nushell is a better idea though - structured output, no text scraping.

And it's Rust! https://www.nushell.sh/

Yes this I think should be the future direction shells go

It's a nice collection of common CLI utilities that make one's life a bit more comfortable.

The poster is not the author, but the collector.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact