
WHLSL: Web High Level Shading Language - bangonkeyboard
https://webkit.org/blog/8482/web-high-level-shading-language/
======
kvark
The article is full of inaccuracies, unfortunately. It mixes up the objective
criteria W3C CG agreed upon with subjective claims of Apple, leading to WHLSL
as a collaborative product of the group... it is not.

> Last year, Apple established the WebGPU Community Group inside the W3C to
> standardize a new 3D graphics API which provides the benefits of these
> native APIs, but is also suitable for the Web environment. <...> All of the
> major browser vendors are participating and contributing to the
> standardization effort.

The browser vendors negotiated and agreed to proceed with W3C group (as
opposed to Khronos/WebGLNext). It's a collaborative effort from day one (where
all of Google, Mozilla, and Apple had prototypes), not exactly Apple-led like
the article attempts to present it.

> Forking the SPIR-V language means developers would have to recompile their
> shaders, possibly being forced to rewrite their source code anyway.

This (and the whole section about SPIR-V) reads like FUD to me.

> Weirdly, this would mean on those two platforms, the starting point and the
> ending point are both human readable, but the bit in between is obfuscated
> with no benefit.

Inaccurate. The end point is always GPU machine code, and on the way there
_all_ the APIs introduced their own intermediate binary formats (SPIR-V, DXBC,
DXIL, AIR). It's just that not all of them are clearly specified. We should be
able to convert SPIR-V to DXBC/DXIL without going through HLSL, and the AIR
limitations are up to Apple to unblock.

~~~
pizlonator
> This (and the whole section about SPIR-V) reads like FUD to me.

It's really not. These are legit arguments:

\- SPIR-V does not yet have a documented security story that is suitable for
the web. Maybe it could have one eventually, but this does not exist right
now.

\- There is not yet a story for what web-SPIR-V would do about all those
optional capabilities and other issues of SPIR-V that are a bad fit for the
web. That includes, for example, SPIR-V's underspecification of most
operations to allow for UB-style optimizations.

\- It's easier to write a code generator that targets a text-based language
than a binary one. If I want to write a JS program, or even an offline
program, that emits shader code that a browser can run, then: (A) to emit
WHLSL all I would need is strings and appending; (B) to emit SPIR-V,
realistically I would need some SPIR-V library. Sure there are many SPIR-V
libraries, but string appending is easier every time.

It's not FUD. It's just arguments that you don't agree with.

> Inaccurate. The end point is always GPU machine code, and on the way there
> all the APIs introduced their own intermediate binary formats (SPIR-V, DXBC,
> DXIL, AIR). It's just that not all of them are clearly specified. We should
> be able to convert SPIR-V to DXBC/DXIL without going through HLSL, and the
> AIR limitations are up to Apple to unblock.

Binary formats that are underspecified. What a shocker. Maybe it's because
text-based languages are easier to clearly specify.

Creating a clear specification of a text-based language is easier for lots of
reasons. Probably the biggest is that the specification has an easy way of
citing code examples. Binary specs always have a hard time with that. No
question in my mind that trying to glean the C/JS/Java syntaxes from those
specs is easier than trying to do the same for x86/arm/wasm/spir-v "syntax".

~~~
kvark
> It's really not.

Let's take the biggest offender (that I quoted):

> developers would have to recompile their shaders, possibly being forced to
> rewrite their source code anyway

Those are likely to hit WHLSL harder than SPIR-V, given that WHLSL is
attempting to be a whole new specification of the format _and_ execution
model, while SPIR-V is only missing the execution model for the Web. The
compatibility story of WHLSL (towards HLSL) is still to be figured out, yet
the blog focuses on the issue only in the context of SPIR-V (compatibility of
existing SPIR-V versus WebSPIR-V)...

I agree there is work to be done for SPIR-V to be adopted on the Web. From the
last call, I thought there is also work to be done on WHLSL. This blog post
tries to show it as the latter is ready, and SPIR-V is likely to fail, hence I
called it FUD.

> It's easier to write a code generator that targets a text-based language
> than a binary one.

It's easier to s/one/two/, but if you are doing anything more complex, you'd
still build a logical representation in memory that is non-textual (hello,
AST). From there, emitting binary output isn't more complex. Needless to say,
there is already a large ecosystem around SPIR-V, which can be targeted from
other SLs like HLSL and GLSL, and even output directly from Rust.

> Binary formats that are underspecified. What a shocker. Maybe it's because
> text-based languages are easier to clearly specify.

Is HLSL specified? Is MSL? They are documented, but not strictly specified. I
only see an actual specification for SPIR-V (among the current-gen graphics
shading languages), which is binary. So I don't buy the argument that text-
based languages are easier to specify. Let's take a floating point number for
example. In binary, you'd just say here come 32bits denoting IEEE 754 float
(basically, piggy back on the existing spec). In text you need to say what
symbols are accepted (exponent form, dot-based form, sign, how many digits are
allowed, etc), which doesn't appear simpler to me.

> Probably the biggest is that the specification has an easy way of citing
> code examples.

Examples are not required to specify a format, they are basically
documentation. In that sense, I agree with you - text formats are easier to
document.

~~~
Gankro
To elaborate on the float example, see JSON, where text-based representations
of numbers has led to massive incompatibilities among different parsers and
generators. The original "spec" didn't even define if numbers were doubles or
infinite precision or what. This is much harder to mess up with a binary
format.

[http://seriot.ch/parsing_json.php](http://seriot.ch/parsing_json.php)

~~~
pizlonator
Programming languages and intermediate representations are about a lot more
than representing floats. I think that it’s true that the best specification
for how to store floats is a binary one. But programs have a lot more than
just float literals and text wins at most of the other ones.

------
xxgreg
"There is active debate in the Community Group about whether or not this
human-readable language should be the one that’s natively accepted by the API"

[https://lists.w3.org/Archives/Public/public-
gpu/2018Nov/](https://lists.w3.org/Archives/Public/public-gpu/2018Nov/)

~~~
skohan
IMO there's no reason to adopt a text-based standard. WASM/SPIRV seem to be a
step in the right direction: assembly is by default more compact and less
ambiguous than a high level language represented as UTF-8 characters.

Let programmers work with whatever high-level language they prefer, and
compile it down to a consistent, compact assembly representation to transmit
it over the network.

~~~
pizlonator
binary code formats are not more compact than text formats. SPIR-V is
particularly fat - you need four bytes to refer to a variable, which is less
than what you need in minified source.

WHLSL is a great compilation target so it doesn’t prevent you from using
whatever language you like.

------
muizelaar
It's weird that Apple says "The WebAssembly Community Group expected that
parsing a bytecode would be more performant than parsing a text language.
However, that turned out not to be true; Asm.js, which is JavaScript source,
is still faster than WebAssembly for many use cases." and then links to a page
that shows WebAssembly being 3x faster.

Also the page they link to is not just measuring parse time but also
compilation time.

------
jcelerier
> The language is insprired by HLSL, the dominant shading language for
> graphics app developers.

sorry wat ?
[https://trends.google.com/trends/explore?q=glsl,hlsl](https://trends.google.com/trends/explore?q=glsl,hlsl)

GLSL is used for multimedia apps, user interfaces (Qt at least uses it quite a
bit), 3D authoring apps. ISF (Interactive Shader Format
[https://www.interactiveshaderformat.com/spec](https://www.interactiveshaderformat.com/spec)),
fairly used in all the software chain around audio-visual installations, is
based on GLSL. It's used on android, on embedded devices, etc etc...

> GLSL is the language used by WebGL, and was adopted by WebGL for the web
> platform. However, reaching cross-browser interoperability was extremely
> difficult due to incompatibilities in GLSL compilers. There remains a long
> tail of security and portability bugs with GLSL still being investigated

and how will implementing a whole new set of compilers for WHLSL solve this ?

> And since Windows and macOS/iOS don’t support Vulkan,

now this is just FUD. you can entirely write vulkan code and have it run on
windows and apple platforms.

> the incoming SPIR-V would still need to be translated/compiled into another
> language

yes, that's the freakin point of SPIR-V

Also...

> In Metal Shading Language, you could, for example, write a shader that casts
> a pointer to an integer, adds 17, casts it back to a pointer, and
> dereferences it. This is a security problem because it means the shader
> could access any resource that happens to be in the address space of the
> application, which is contrary to the Web’s security model. Theoretically,
> it could be possible to specify a dialect of Metal Shading Language that
> doesn’t have raw pointers, but pointers are so fundamental to the C and C++
> languages that the result would be completely unfamiliar. C++ also heavily
> relies on undefined behavior, so any effort to fully specify each of C++’s
> numerous features would be unlikely to be successful.

followed by

> The first is the safe pointer. Some form of reference semantics, which is
> the behavior pointers allow for, are used in almost every CPU-side
> programming language. Including pointers in WHLSL will make it easier for
> developers to migrate existing CPU-side code to the GPU

so instead of reusing an existing thing and just add one tiny dot of safety
semantics on top of it, well no let's just NIH and reimplement a _whole new
language_. Seriously, they don't want to use a "safe metal" because "pointers
are so fundamental to the C and C++ languages that the result would be
completely unfamiliar".

So instead, they write a freakin new language - why the hell do they use the
familiarity argument then ?

~~~
pjmlp
> GLSL is used for multimedia apps, user interfaces (Qt at least uses it quite
> a bit), 3D authoring apps. ISF (Interactive Shader Format
> [https://www.interactiveshaderformat.com/spec](https://www.interactiveshaderformat.com/spec)),
> fairly used in all the software chain around audio-visual installations, is
> based on GLSL. It's used on android, on embedded devices, etc etc...

Google and Samsung are developing HLSL compiler for Vulkan, because a large
majority of game studios don't want to bother with Vulkan on Android if they
cannot port their shaders.

There are plenty of HLSL compiler status talks at the Khronos web site and
YouTube channel.

> > And since Windows and macOS/iOS don’t support Vulkan,

> now this is just FUD. you can entirely write vulkan code and have it run on
> windows and apple platforms.

On the contrary, this is quite true.

Apple does not support Vulkan, you need VulkanMK, which is basically a Vulkan
portability layer making use of Metal API, and only supports the subset common
to both APIs.

Microsoft does not allow OpenGL ICDs on UWP/Store Apps, which is the mechanism
used by Vulkan drivers on Windows, thus it is not possible to run either
OpenGL or Vulkan code on UWP and Store apps.

Which is why Microsoft ported Angle for UWP, however they haven't bothered
with similar support for Vulkan.

[https://github.com/Microsoft/angle](https://github.com/Microsoft/angle)

~~~
cztomsik
and who will make microsoft support whlsl? w3c? (serious question)

I'm not against new standards, but I'd rather stick with something already
adopted. Vulkan seems better than metal (proprietary). And I really don't have
enough experience with HLSL and GLSL but they seemed fine both.

~~~
flohofwoe
WHLSL is primarily relevant for the browser. So if Microsoft adds WebGPU
support to their browser, they'll add a WHLSL compiler to it.

I'm also quite sure there will be translation tools to translate WHLSL to
HLSL, GLSL and MetalSL (probably going through SPIRV as IR, see for instance
[https://github.com/KhronosGroup/SPIRV-
Cross](https://github.com/KhronosGroup/SPIRV-Cross))

