
The Piston image library is now pure Rust - tekacs
http://blog.piston.rs/2017/01/08/the-image-library-is-now-pure-rust/
======
ape4
Image libraries are exactly where we need rust-style safe code.

~~~
zzzcpan
No, image libraries is exactly where performance always beats safety and where
other approaches to secure them actually work, like sandboxes and containers,
and kind of have to be used anyway, since memory safety doesn't guarantee that
unpacking an image won't DoS the system.

~~~
valarauca1

         here other approaches to    
         secure them actually work, 
         like sandboxes and containers,
    

"It is a fallacy to think that for 30+ years people haven't been able to write
a secure OS. But virtualization, and containers are" \- Theo de Raant

~~~
nickpsecurity
GEMSOS, KeyKOS, ASOS, LOCK, VAX VMM, INTEGRITY-178B, seL4, NOVA... all in the
4Kloc-50+Kloc range for a reason. All simpler, smaller, and easier to model
than a full OS. Two survived pentesting by NSA with others during validation
way better in defect rate or correctness case than most OS's.

Theo's quote is BS by the numbers and pentest results. Sandboxing done right
is way more secure.

~~~
lobster_johnson
While true, in practical terms, those OSes are not what people have lying
around to do their image processing. I'd love a secure, formally verified,
capabilities-based microkernel OS as much as next person, but all I have is
Linux and various BSDs. It's a bit academic at this point.

~~~
nickpsecurity
The commenter countered that sandboxing was worthwhile by citing a claim by
Theo de Raadt that getting those (i.e. VM's) right was about as hard as
getting whole OS's of code right. That one proved futile meant the other
would. I countered that citation with numerous examples that disprove his
point. Then, by implication, sandboxing might be effective if it's done like
in my examples. And there are CompSci designs doing stuff like that.

Whether you want to do image processing on a Linux, BSD, or one of my kernels
is a separate topic. However, it's worth noting that even stuff like I cited
could be useful for that where it stashes the image processor into its own
protection domain with read access to memory containing input and write access
to memory that will have the output. The code itself isn't allowed to do or
touch anything else. At this point, exploits must be chained in clever ways.
Modifying this to preserve pointer integrity, lowest overhead range being
around 1-10%, would choke those kind of attackers further.

Using Rust is yet another tactic that can work well with or in lieu of a
sandbox. Ideally with for issues the language itself can't cover. Three
precedents for that are right on my list: GEMSOS's safety was improved using a
Pascal subset with call-by-value semantics; ASOS was a combo of security
kernel and Ada runtime to host embedded, Ada applications (lang-safety +
sandbox); separation kernels like INTEGRITY-178B usually support runtimes for
Ada and Java subset to get safety within components. Now, there's some OS's
like Tock and Redox plus low-level libraries written in a new, safety-
enhancing language. Such designs can benefit from the old strategies which we
see a few in Redox. On other end, Genode uses many old and recent tricks in
architecture but not safe language. It can also run a Linux desktop with your
image processing code. ;)

Note: A bit academic is also misleading. Several, separation kernels are
deployed commercially for things like secure, web browsing. INTEGRITY and
LynxSecure are among oldest. The Nizza and Perseus architectures got turned
into Sirrix's Turaya Desktop. These things aren't just academic.

~~~
lobster_johnson
It's academic in the colloquial sense that the tools you describe aren't
available to use. I'd love to have all the futurustic, secure stuff you
describe, but for a user's run-of-the-mill AWS deployment, that's not an
option.

All the other stuff you say is on point, of course, and there's an argument or
a hundred to be made that a lot of these problems are already solved, and that
we (as users and developers) are missing out on a massive amount of amazing OS
research that could have made everything better than our current insecure,
1970s-tech monolithic kernel OSes.

~~~
nickpsecurity
"It's academic in the colloquial sense that the tools you describe aren't
available to use."

I just named multiple tools available to use in commercial sector and FOSS.
Their predecessors were used by commercial and government sector for a decade
before that. That's 100% not academic even if a few started there. If
anything, more work just needs to be poured into FOSS stuff so it gets more
deployment and impact.

"All the other stuff you say is on point, of course, and there's an argument
or a hundred to be made that a lot of these problems are already solved, and
that we (as users and developers) are missing out on a massive amount of
amazing OS research that could have made everything better than our current
insecure, 1970s-tech monolithic kernel OSes."

We're definitely in agreement there. Fortunately, we're seeing some
exploration into that including on the Rust front with Redox.

------
krzat
A bit offtopic, but when Rust will have incremental compilation? I just tested
Piston's hello world and it takes 4 seconds to recompile. A bit too long for
my taste.

~~~
a_humean
It's a priority for this year with the ground work already completed last
year:

[https://github.com/rust-
lang/rfcs/blob/master/text/1774-road...](https://github.com/rust-
lang/rfcs/blob/master/text/1774-roadmap-2017.md)

[https://blog.rust-lang.org/2016/09/08/incremental.html](https://blog.rust-
lang.org/2016/09/08/incremental.html)

~~~
amelius
It would be nice if they exposed these ideas, so that programs written in Rust
can themselves become incremental.

~~~
steveklabnik
What would this mean? Rust is AOT compiled, so I'm not quite understanding
what it would mean for a program in Rust to become incremental.

~~~
dbaupp
Other programs are processing pipelines like a compiler, so the dependency
graph code that computes when things need to be reprocessed could
theoretically be extracted and published as a reusable library.

~~~
steveklabnik
Ah ha! I see, that makes perfect sense, thanks.

------
LeanderK
we are chasing more after trends than the fashion industry. The post contains
no more info than that the image library is now pure rust.

I want to like rust and understand that there are benefits to it, but why is
this on the frontpage? What important information does this post contain that
justifies this, beside that some project now has their image library in pure-
rust (does that even make sense? Not every rewrite is justified, replacing a
rock-solid c-lib with a brand new rust-lib may not be a good idea)? I just
loosely follow rust and i really don't see how i benefit from reading this.

I think we need to step back and realise that while there might be benefits to
a certain technology, it's not a miracle drug.

~~~
TAForObvReasons
> we are chasing more after trends than the fashion industry.

Larry Ellison made the same observation many years ago:

> The interesting thing about cloud computing is that we've redefined cloud
> computing to include everything that we already do. I can't think of
> anything that isn't cloud computing with all of these announcements. The
> computer industry is the only industry that is more fashion-driven than
> women's fashion. Maybe I'm an idiot, but I have no idea what anyone is
> talking about. What is it? It's complete gibberish. It's insane. When is
> this idiocy going to stop?

~~~
steveklabnik
A similar sentiment by Alan Kay: programming is a pop culture.

I usually don't agree with these sentiments, personally, or at least not with
the implications that are usually drawn from them. But it is something worth
thinking about.

------
nercury
This is fantastic news! This was the reason I kept using SDL2_image - I had
too much trouble compiling Rust image crate on windows.

------
nox_
The inflate crate is absolutely devoid of tests and full of unsafe code
though, so the foundations of all of that are quite weak.

------
bsaul
If only rust had concurrency / parallel programming primitives or library as
nice as golang or elixir,i think i'll move to this language right away. For
now i'm still holding my breath before developping with it on the server side.

~~~
ekidd
I think some of issue here is that when you ask for "concurrency / parallel
programming primitives", you're actually talking about more than one problem.
We can divide things up as follows:

\- Data parallelism. This is where you have (for example) a giant array of
data to process and want to use as many cores as possible. For this, rayon is
gorgeous:
[https://github.com/nikomatsakis/rayon](https://github.com/nikomatsakis/rayon)
Seriously, I can just not say enough good things about programming with rayon,
and how nice Rust's "no mutable aliasing" makes coding. Also at some point,
Rust will also need good libraries for talking to the GPU.

\- I/O concurrency. This is where you have a server that needs to respond
quickly to 10,000+ sockets, for example. For this, it looks like everyone is
standardizing on futures and tokio: [https://github.com/alexcrichton/futures-
rs](https://github.com/alexcrichton/futures-rs) [https://github.com/tokio-
rs/tokio](https://github.com/tokio-rs/tokio)

But it seems like you're most interested in Erlang-style actors with M:N green
threads? The issue here is that—as best I remember the history of Rust—the
Rust maintainers decided that green threads were simply too high a level of
abstraction for Rust. I think Rust will eventually move back in that
direction, first using futures+tokio+async I/O, and then by eventually adding
async/await sugar on top. That way, you'll only pay for coroutines/green
threads when you explicitly opt in. But it will take a year or so for all this
to start maturing, I think.

In the meantime, tokio's method-chaining syntax should actually be somewhat
tolerable, especially because Rust has a `#[must_use]` attribute that prevents
you from forgetting to use a future. If you're lazy, just run everything
through rustfmt to get the indentation right.

~~~
steveklabnik
> as best I remember the history of Rust

There's an RFC for that! [https://github.com/rust-
lang/rfcs/blob/master/text/0230-remo...](https://github.com/rust-
lang/rfcs/blob/master/text/0230-remove-runtime.md) (A reminder that this was
written in 2014, so statements are about Rust at that time period)

> I think Rust will eventually move back in that direction

Note that tokio is not "green threads" as they're conventionally thought of,
though they are similar if you squint, kinda.

~~~
ekidd
> Note that tokio is not "green threads" as they're conventionally thought of,
> though they are similar if you squint, kinda.

Well, I'm leaping ahead to the end of the story, here, and speaking far too
loosely about technical details. ;-) Futures + tokio + a possible future
async/await proposal would provide a reasonable enough syntax for
actors/coroutines in Rust, I suspect.

~~~
steveklabnik
Fair enough! :)

Relevant links:
[https://github.com/erickt/stateful](https://github.com/erickt/stateful) and
[https://github.com/rust-lang/rfcs/pull/1823](https://github.com/rust-
lang/rfcs/pull/1823)

------
mrich
Any benchmarks?

------
perturbation
A bit OT, but is there a C API that Piston exports? I.e., could I use Piston
from C/C++ (or another language that supports the C ABI)?

~~~
steveklabnik
I don't believe so; Piston's goals are to make libraries for use in Rust,
rather than trying to use Rust to improve other software, as far as I know. I
did a quick look through of it and didn't see anything indicating there was,
both in the main umbrella "piston" project as well as "image" in particular.

------
phkahler
Does it support higher bit-depth JPEG? Or just 8?

~~~
wongarsu
From a quick look at the source code it seems only to support grayscale, RGB
and CMYK jpegs with 8 bits per channel.

------
lacampbell
...and?

Is it faster? Is it shorter? Has it fixed bugs? Does it have more
functionality? What reason does a user or potential user of this library have
to care that it's now written in another language?

~~~
steveklabnik
There is already a top-level comment asking this question, with a few replies
[https://news.ycombinator.com/item?id=13351066](https://news.ycombinator.com/item?id=13351066)

~~~
lacampbell
My job right now is working on a C# codebase. I'd love to re-write everything
in F#. I feel like it's higher level, less prone to error, and would make the
codebase much smaller.

The thing is - no one is going to be impressed if they hear it's been re-
written in F# because I _assume_ doing so is going to make the software easier
to maintain, easier to extend and more reliable. Not management, not users,
no-one.

Now, if I did the re-write and LOC went down by half, or performance improved,
or it fixed a lot of bugs, or I was able to add features in much more quickly
because abstractions were tighter and easier to create and the compiler was
able to check more constraints for me - then people might take notice.

There is, however, no inherent value in the source language being changed in
itself.

~~~
steveklabnik
That's what makes blog posts like this tricky: this post is one for the
project itself, which means that its audience is already for people who know
what the project is, etc. Someone posted it on HN, and it got upvotes, so it
gets to a broader audience, who may not inherently know these kinds of things,
or not care.

So your point here is valid, that is, "as someone _not_ in that audience, why
should I care?" That's what the other thread, with its replies, is getting at.
Which is why I pointed to you to it.

It's possible as well, of course, that maybe that _still_ means this doesn't
matter to you. Which is also expected; nobody is going to like every thread on
HN. That's why the guidelines[1] have the section of "Please don't submit
comments complaining that a submission is inappropriate for the site." And it
gives you something to do instead: " If you think a story is spam or off-
topic, flag it by clicking on its 'flag' link." And beyond that, you could
just skip this thread and read any of the other 29 posts on the front page.
That's how I usually handle posts that I don't have any particular interest
in.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

~~~
lacampbell
Constructive criticism is nothing to shy away from or be defensive about.
Having criticism for something doesn't mean I think it's spam or off topic.

Again, for an example of a language X-to-language-Y rewrite story with some
substance, that did a lot to boost the image of the language in many peoples
eyes, see:

[http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-
ret...](http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-
retrospective/)

I'd love to something like that the next time "X rewritten in rust" etc gets
to the front page of hacker news. A re-write in of itself tells me nothing.

~~~
Manishearth
What are you criticizing here? You're ultimately criticizing the HN community
for upvoting a topic you're not interested in, which IMO is absurd, you're
telling people what to be interested in. Either that, or you're criticizing
the blog post author for not covering all audiences, which is also absur,
folks are allowed to target audiences. This isn't really constructive.

Like Steve said, this post wasn't written for a general audience. A Rustacean
will read that post and will be very happy to know that they can do certain
things using pure Rust libraries. That was the audience of the post, the post
makes total sense in that context, and has inherent value in that context. It
got shared more widely, and HN seems to like it, which means that possibly
folks who weren't the original intended audience are reading it, which is okay
too, just that they may not find it that interesting.

