
Standardizing WASI: A system interface to run WebAssembly outside the web - bnjbvr
https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/
======
kodablah
Been hoping a stdlib for WASM would spring up. So if I wanted to implement a
WASI backend (e.g. on the JVM), are the function definitions I should
implement here: [0]? I assume string, array, struct, etc layout in WASM mem is
as C expects? Also, is it a goal to provide a test suite to test conforming
backend implementations (may already be there, didn't look)? Finally, pardon
my lack of research before asking, does this mean that an LLVM-based WASM
compilation can target this instead of emscripten/libc and the final WASM
could reference all of these API pieces as a imports? Is there an expected
posix/libc-to-wasi lib?

0 - [https://github.com/CraneStation/wasmtime-
wasi/blob/wasi/docs...](https://github.com/CraneStation/wasmtime-
wasi/blob/wasi/docs/WASI-api.md)

~~~
tschneidereit
(Member of the team at Mozilla here )

Yes, that's the list.

And the layout of structs, strings, etc is up to the compiler, within the
bounds of the restrictions WebAssembly imposes.

We'll definitely have a test suite, but this is all early days, so a lot of
all that isn't yet in place.

And yes, this can be targeted by LLVM-based and other compilers. In fact,
Emscripten could use this as the foundation for their POSIX-like libc and
library packages. The syscalls are indeed exposed as Wasm function imports.

~~~
echeese
Will WASI normalize differences between platforms? e.g. convert argv or paths
to a consistent character encoding?

~~~
tschneidereit
Yes!

~~~
azakai
What about platform differences like how file permissions work on windows vs
posix? (i.e., stuff that Python does not fully normalize)

~~~
AnIdiotOnTheNet
I have a dollar that says all platform difference issues will be solved by
just doing whatever POSIX does and expecting the host OS to figure it out if
it isn't already POSIX. Whenever you try to abstract away arbitrarily
different implementations while retaining their non-common functionality you
either end up reimplementing one of them and expecting the others to work
around it, or you end up forcing the programmer to bypass the abstraction
anyway and implement logic for each implementation.

------
Others
This is really interesting. As part of my undergrad research I've built
`silverfish`, which is a tool that turns WASM binaries into LLVM bytecode.
It's currently pretty chained to how I personally compile my C to WASM, but
it'd be pretty cool to get it working with this standard!

Here's a link:
[https://github.com/gwsystems/silverfish](https://github.com/gwsystems/silverfish)
\-- although the README is pretty sparse, so you'll have to look at the code.

~~~
lyxsus1
wow, thank you. that's amazing!

------
gwbas1c
Wait a second...

I thought Java was supposed to do this with the JVM...

I thought .Net was supposed to do this with the CLR...

What's different now?

~~~
otabdeveloper2
The JVM and CLR suck balls.

They put in huge amounts of effort into things that don't need solving, and
the things that need to be solved they do in the most roundabout way possible.

What we _really_ need is a sandboxed VM for running C code securely and
efficiently.

The JVM is pretty much the opposite of that; it's not sandboxed, not secure
and certainly not efficient as a target for C code.

We're hoping WASM starts over from scratch and does things right this time
around.

~~~
Teknoman117
... 20 years later

JVM, CLR, and WASM suck balls. People have been breaking the sandbox for the
better part of a decade.

We're hoping Cool New Thing™ starts over from scratch and does things right
this time around.

~~~
timClicks
But isn't that the point? We should be getting better at building software
over time. Sometimes that means restarting.

~~~
Hydraulix989
UNIX is a good counter-argument.

~~~
JeremyBanks
UNIX is thoroughly obsolete. Its continued use is from inetia; people rarely
need anything but a tiny subset of its capabilities, for which there are more
simpler alternatives every day.

~~~
maccio92
are you kidding? do you know how many subsystems that people take for granted
are using UNIX? furthermore, how do you get simpler than UNIX? it's not
complex..

~~~
Hydraulix989
I just didn't feel like responding since it was clear OP didn't know what he
was talking about.

------
hathawsh
Solomon Hykes, co-founder of Docker, believes WASM+WASI could take the place
of Docker.

[https://twitter.com/solomonstre/status/1111004913222324225](https://twitter.com/solomonstre/status/1111004913222324225)

~~~
talkingtab
This. I don't have a clear enough mental model for either docker or wasm+wasi,
but I immediately thought of docker. What is docker except a bunch of purpose
build operating system instances? And why do we need Linux installations (as
much as I am a fan of Linux) when we could just run on WASI? And Electron,
where each instance runs on an individual instance of a browser?

~~~
simcop2387
This is the same reason that things like unikernels have been built for. They
haven't taken off because of how difficult it is to rebuild your application
to also contain it's kernel, but this kind of sandboxing with WASM and WASI
doesn't require that and could likely make things a lot simpler to deal with.

~~~
pjmlp
Yep, just like JEE servers deployed bare metal, oh wait.

~~~
talkingtab
I'm not sure I get the reference? Could you explain a bit? I know what Java EE
servers are, but don't recall anything about bare metal deployments. Comparing
my experiences with using JEE and Docker was that they are nothing alike. Just
unclear as to your point.

~~~
simcop2387
Java was at one point going to have a bunch of CPUs[1] built around the JVM
bytecode, that would have allowed J2EE and such to run on the bare metal
hardware, but they largely didn't go anywhere.

The most successful attempts that i'm aware of are Jazelle[2] and Java-card[3]
plans to augment the software solutions

[1]
[https://en.wikipedia.org/wiki/Java_processor](https://en.wikipedia.org/wiki/Java_processor)

[2]
[https://en.wikipedia.org/wiki/Jazelle](https://en.wikipedia.org/wiki/Jazelle)

[3]
[https://en.wikipedia.org/wiki/Java_Card](https://en.wikipedia.org/wiki/Java_Card)

------
Svoka
After reading about all the AMP and Google, Mozilla's blog is a welcome
relief. Always a pleasure.

------
ironchief
These Mozilla blogs are some of the best edited, articulated and illustrated
explanations I've seen on the web. I wish more people could put in the
resources to help communicate their mission like this.

~~~
humblebee
The author, Lin Clark[0], is the person behind Code Cartoons[1]. She does
absolutely amazing work. Unfortunately, there isn't really a single place to
find everything she has done. She has done quite a few conference talks[2] as
well.

[0] [https://twitter.com/linclark](https://twitter.com/linclark)

[1] [https://code-cartoons.com/](https://code-cartoons.com/)

[1] [https://hacks.mozilla.org/category/code-
cartoons/](https://hacks.mozilla.org/category/code-cartoons/)

[1] [https://twitter.com/codecartoons](https://twitter.com/codecartoons)

[2]
[https://www.youtube.com/playlist?list=PLIIHcC8epcPqhBrQ1dRmI...](https://www.youtube.com/playlist?list=PLIIHcC8epcPqhBrQ1dRmILg5e5okOVnER)

~~~
hsndmoose
Adding a podcast Lin was on, detailed just how much work goes into creating
Code Cartoons. It's pretty amazing and highly appreciated.

[https://changelog.com/podcast/294](https://changelog.com/podcast/294)

------
steveklabnik
This year is extremely interesting times for WebAssembly, there's a number of
these different runtimes popping up. The effects of all of these decisions,
including stuff like this, won't be felt until next year... I wonder how much
adoption this will get, but the lineup looks _extremely_ strong.

------
naikrovek
ok, so I hate to be this guy and I'm going to do it anyway because it has to
be said:

Did they just invent Java again? Aren't they promising what Java promised?
Won't they hit the same problems that Java hits during its write-one-run-
anywhere promises?

I'm asking honestly - why is WebAssembly _outside of the browser_ needed?

~~~
jzoch
The idea of isolation that java espoused is not a bad one - it just wasnt
executed well. Its not a java only idea either - see Native Client[0]. As with
all things, execution and marketing matter. Java was lacking in both. [0] -
[https://static.googleusercontent.com/media/research.google.c...](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/34913.pdf)

~~~
naikrovek
OK, that's all correct and valid.

Java _still exists_ and is a valid deployment platform.

Why is WebAssembly __outside of the browser __needed?

~~~
JeremyBanks
Using Oracle's Java Programming Language is an immeasurable legal liability.

~~~
naikrovek
There are open source and non-Oracle Java virtual machines you can buy support
from if Oracle isn't your cup of tea.

It's not really a legal liability either. Lawyers for companies who use Java
go over those agreements with a fine tooth comb and approve them before
they're signed. There are no surprises (if your attorneys aren't frauds.)

~~~
JeremyBanks
Google's was not using Oracle's virtual machine, and their attorneys were not
frauds, but they've been tied up in litigation for most of a decade and are
facing potentially ten billion dollars in penalties.

[https://en.wikipedia.org/wiki/Oracle_America,_Inc._v._Google...](https://en.wikipedia.org/wiki/Oracle_America,_Inc._v._Google,_Inc).

Java is not a responsible or ethical choice for anybody to use for any
purpose.

~~~
pdpi
Google's case is less one of "using" Java, and more one of reimplementing it.
It's an entirely different legal beast.

------
maxgraey
WASI layer interface for AssemblyScript from Frank Denis:
[https://github.com/jedisct1/wasa](https://github.com/jedisct1/wasa)

~~~
tschneidereit
Oh, this is amazing!

------
droobles
Excuse me if I come across as naive, I'm kind of a noob on these architectural
topics, but how is this different/better than what the JVM is/accomplishes?
When WASM first popped up I thought about the similarities with Java Web
Applets and Flash.

~~~
jxcl
1\. It's an open standard that the major browsers have agreed to, so it's not
a plugin that you have to install. A WASM app will work seamlessly in your
browser without any additional software.

2\. WASM is designed from the ground up as a compile target, not a language.
We already see many languages with support for building to WASM. C, C++, Rust,
and eventually when WASM supports garbage collection we'll probably see
Python, JS, Go, all with support for compiling to WASM.

~~~
arcticbull
(1) could be resolved by shipping Java, let's say hypothetically.

(2) the JVM was designed from the ground up as a compile target also, for
Java, but is also used by Kotlin, Clojure, Ruby (JRuby), Python (Jython) and
Scala, among others. Even JavaScript (Rhino, Nashorn and Graal).

~~~
Rusky
The JVM was designed as a compile target for a very particular kind of
language. One with a Java-like object model, with some later concessions for
more dynamic languages.

C, C++, Rust, etc. do not fit into that model at all. There's a lot of
software written in C, and there's a lot of performance to be gained by
dropping down to that level when necessary.

~~~
fulafel
By later concessions do you mean invokedynamic? How much did it help? Clojure
doesn't use it even now, 8 years after it came about, I think. Is it mostly
for JRuby?

The JVM is (and was from the start) a lot more dynamic than Java.

------
equalunique
I love cross-over episodes like this.

Patiently waiting for when eBPF gets it's turn inside the browser.

------
writepub
I would love for WASI/WasmTime to emphasize on:

1\. Backwards compatibility with existing libc. (Maybe pick musl)

2\. Platform agnostic wasm generation: the same wasm file should run in the
browser (with emscripten polyfills) and across ALL OS-es, mobile included.
List: iOS, Android, Mac, Linux, Windows, FreeBSD

#1 shall enable decades of legacy programs to work with minimal porting, while
#2 shall enable true cross platform capabilities without multiple codebases.
Specifically, don't repeat what node did: digress from browser js semantics,
instead of polyfilling them

~~~
jcranmer
> 1\. Backwards compatibility with existing libc. (Maybe pick musl)

Honestly, that ought to be a non-goal. We already know that POSIX often
enforces models that we don't want--filesystem permissions and the fork model
are two good examples of things that are broken. So why start with "implement
POSIX"?

~~~
Crinus
POSIX != C and from what i understand from the official docs, they already
have a (musl derived, even) C library.

~~~
AnIdiotOnTheNet
The C stdlib is so married to POSIX they may as well be the same thing.

~~~
pjmlp
Actually no, go read ISO C and compare it with POSIX expectations.

------
_bxg1
I've heard people predict that the future of the web is to replace userland,
to make every program cross-platform and sandboxed close to the metal, and for
JavaScript to fade into a family of "web"-targeted languages. Looks like it's
happening.

~~~
millstone
In this scenario, what's the point of having more than one OS?

~~~
int_19h
Same as with POSIX - different implementations of the same standardized
interface that target different hardware, and have different quality of
implementation (possibly optimized for different use cases).

------
gouh
This sounds great. Basically the web created a cross platform & sandboxed API
to access everything we need (Camera, Sensors, OpenGl through WebGl, etc.) but
it was very opinionated (for historical reasons) as it imposes the slow JS,
the DOM, CSS/HTML etc.

Now this is the same idea but, this time, one layer lower so that we can have
performance, the language we want, etc. This is kind what I always wished
would happen.

It will be interesting to see how the sandboxing will work with already
existing sandboxing solutions. For e.g a WebAssembly app packaged as a
Android/Flatpak/UWP/etc. , there will be the need to make mappings between the
permissions of the two sandbox systems. Or even maybe one day we'll have the
Webassembly sandbox as the only sandbox (like in
[https://github.com/nebulet/nebulet](https://github.com/nebulet/nebulet))

------
VikingCoder
WASI will have some interesting milestones ahead of it:

* Self-hosting (WASI can run inside WASI)

* GCC can run in it

* Linux can run in it

* Quake can run in it

* Chrome can run in it

~~~
dboreham
Doesn't Quake traditionally come first?

~~~
TheAceOfHearts
Isn't Doom the standard go-to?

------
lachlan-sneff
This is super cool! We're going to have to rush to implement this in the
wasmer runtime!

~~~
gbraad
This is what I was thinking. How will this affect wasmer.

------
_jn
Can't help being reminded of that talk by Gary Bernhardt: “The Birth & Death
of JavaScript”[0] — exploring a hypothetical future where JS takes over
everything without (most) anyone using it of their own volition.

0: [https://www.destroyallsoftware.com/talks/the-birth-and-
death...](https://www.destroyallsoftware.com/talks/the-birth-and-death-of-
javascript)

~~~
tokyodude
Except entirely irrelevant as WASM is not Javascript.

~~~
_jn
Sure, though it’s still a web technology taking over an otherwise unrelated
space ¯\\_(ツ)_/¯

~~~
int_19h
I don't think it's unrelated. Despite the name, WASM isn't really a "web
technology" \- it's a sandbox technology and a compile-once-run-everywhere
technology, and there has always been demand for that outside the web, even
before the web existed. It might be that the web is what created enough demand
for it to happen in the end, but what do we care?

The problem with JS was never that it's a web technology. It's that it's a bad
technology that happened to be in the wrong place at the wrong time to get a
first mover advantage.

------
dmitriid
Should they standardize to run WebAssembly on the Web first? There's like
hundreds of things to do:
[https://webassembly.org/roadmap/](https://webassembly.org/roadmap/),
[https://webassembly.org/docs/future-
features/](https://webassembly.org/docs/future-features/)

~~~
tschneidereit
We're working on that, too :) See this post from last Fall where we laid out a
way to think about where WebAssembly is going, which use cases to enable, and
how: [https://hacks.mozilla.org/2018/10/webassemblys-post-mvp-
futu...](https://hacks.mozilla.org/2018/10/webassemblys-post-mvp-future/)

------
pspeter3
How much of the Fuscia model apply to Web Assembly? It seems like they could
share similar security models

~~~
tschneidereit
We've mainly based the current design on CloudABI/Capsicum, but it's all early
days, and Fuchsia is on our list of systems to at the very least take heavy
inspiration from :)

~~~
writepub
Will this be backwards compatible with existing libc?

~~~
sunfish
This tutorial gives an overview of how compatibility with existing portable C
code works:

[https://github.com/CraneStation/wasmtime-
wasi/blob/wasi/docs...](https://github.com/CraneStation/wasmtime-
wasi/blob/wasi/docs/WASI-tutorial.md)

------
justinmchase
Love it. This could be a huge innovation and really push the dream of cross
platform development to the next level.

I suppose this would have pretty big implications for Electron or a similar
successor to aid in the UI portion of this endeavor.

~~~
titanix2
Yeap, so innovative it existed already for decades (virtual machine executing
some fixed bytecode).

~~~
rapsey
Properly sandboxed VM that you can compile practically any language into?

~~~
pjmlp
AS/400 TIMI comes to mind, and CLR as well.

~~~
int_19h
CLR sandbox should not be considered a meaningful security boundary at this
point. CAS and partial trust are both officially obsolete.

The problem is always complexity. It can be difficult to reason about safety
with so many features on bytecode level. There have been exploits using
dynamically-generated methods, exception filters, and vararg handling - note
the near-lack of any common patterns here other than they're all obscure
features of the platform, the security impact of which wasn't fully analyzed.

For example, a long time ago, I reported the vararg exploit, which boiled down
to the fact that you can have ArgIterator over an argument list containing
another ArgIterator, allowing the latter to be mutated via the former. Thus,
you can stash away an ArgIterator to the arguments of a function call that has
already returned. This is basically the same as having a managed reference to
a managed reference, and it's exactly why CLR prohibits this - but they forgot
that ArgIterator is also a kind of a managed reference.

~~~
pjmlp
And WASM remains to be battle tested in the wild, as in the juicy target of
millions of black hats out there.

Forcing internal corruption of code produced from C compilers (like doing a
stack out of bounds data overwrite due to incorrenct params size) is perfectly
viable.

Yeah the exploit doesn't leave the sandbox, so what. It can still be used to
direct the sandboxed code to produce other outcome from the called functions.

------
jasonhansel
Since it's based on web tech, is this going to be focused on async, rather
than synchronous, calls?

------
skybrian
This technology will become more compelling once there is a deployment target
that runs WASM+WASI binaries but won't run native binaries.

Nothing like that exists (it's too early) but it would be interesting to know
if anyone is planning anything.

------
throwaway66666
And here I am still writing asmjs by hand. Do we know if that's fully
deprecated or are there any plans to keep it fast and loved like wasm?

------
wolfspider
If there are any WIP implementations utilizing CloudABI and Capsicum they
would be interesting to look at.

------
naikrovek
The amount of hatred in this thread for non-web technologies is insane.

~~~
krapp
Funny, I see the exact opposite. People dismissing WASM as unecessary, or just
another Java, or just another Flash, or just another Node.js, or something
that shouldn't exist outside of the browser.

If it didn't have "web" in the name, I think people would hate it a lot less.

------
thosakwe
What does this mean for host bindings?

------
SubiculumCode
There is already a WASI and its a standardized IQ test. I did a double take.

------
Touche
This is nice and I'm excited about it, however there is a concern that people
are going to start building code that targets WASI and then shim it to run in
the browser. This happened to JavaScript when Node.js was released; people
began using APIs intended for servers and then shimmed them to also run in the
browser resulting in code bloat. I'm worried that the same is going to happen
here; the video actually encourages doing just that. The browser is not a
platform that should be treated as secondary.

~~~
AnIdiotOnTheNet
I think the browser shouldn't be a platform at all, frankly, and WASI is one
possible way we can finally stop trying to shoehorn it into being one. If
everything runs a WASI runtime, which is designed to run applications from the
ground up, there really isn't any need for the browser to run applications
anymore, is there?

~~~
tokyodude
I think the world in which the browser disappears and all we have is apps is a
worse world than the one we have now.

With HTML/CSS we get the relatively easy ability to add extensions to dig deep
into the data of a page. One of my favorite extensions is rikaikun which adds
popup translation of Japanese words. It can only do this because it can
inspect a standardized data structure (the DOM) with standardized APIs.

I also love the ad blockers. They let me not only block ads but also hide
parts of websites I find distracting. For example I hide the "Hot posts from
other stack exchange sites" section of stack overflow because I find I get
sucked into 20-40-60 minute distractions if I don't hide it.

I can also copy and paste nearly any content and link to almost any content.

An a world of native apps none of this is really possible. Every native app
will use its own internal representations of data. Different UI kits or
libraries. Much of it is un-linkable, un-copyable, It drives me nuts when I
can't copy and paste of phone number, or address in a native app.

I don't believe getting rid of the web for a WASI based app world would be a
good thing for users.

PS: I think WASI is a great idea. I just think trying to replace HTML based
apps with WebAssembly apps using WASI and rendering with WebGPU is the wrong
direction for most web apps.

~~~
saagarjha
What you’ve mentioned about HTML/CSS is true for native apps too: there is a
standardized API for drawing the UI (GTK+, Cocoa, whatever Windows has) and
standardized data formats that back them. The issue is that apps are not
necessarily required to use these (ironically it’s Electron et al. that break
these) and they don’t use code that is readily inspectable, unlike pure
JavaScript. On the other hand, though, this has been slowly been becoming true
of the modern web: websites are ditching the DOM for React/Vue/Angular “SPAs”
that use horrible selector names and minimized and nigh-impossible to
comprehend JavaScript so they can reimplement things that the platform
provides for free. So it’s not great either way…

