VectorWare founder here. We are working with them and stoked they are investing more in Rust. I just gave a talk at RustConf about our different takes (https://rustconf2026.sched.com/event/2KNQj/making-gpus-feel-...). The video isn't up yet but you should check it out when it is. The efforts are complementary.
It's just a way for us to add minutia and details that most don't care / need to know about. There are three audiences we try to make the posts accessible for: Rust people who don't know about GPUs, GPU people who don't know about Rust, and non-Rust non-GPU people. The toggle lets knowledgable readers go "wait, what about..." and hopefully the toggle answers it.
We never mention anything about superiority nor compare with other languages or programming models. This post is about making existing Rust CPU code work on the GPU.
What is vectorware's business model? Are you planning to sell support/consulting to companies using your stack? Or are you looking to sell licenses to your tool? Or something else?
The tentative plan is to open source all the compiler and `std` bits with our products built on top (compilers are not good businesses). More about our products coming in the next couple of months!
The post is kind of vague on the IR you're targeting. Can you give some examples of what the SIMD-ized IR looks like, and how it maps to the target PTX?
Each family of operations is a trait parameterized by the operation itself:
pub trait EvaluateReduction<Operation, T>: LaneEvaluator {
/// Reduce one distributed definition to an ordinary uniform scalar.
fn evaluate_reduction(&self, value: LaneValue<Self, role::Distributed, T>) -> T;
}
Call sites name the operation:
let one = evaluator.splat::<Splat, _>(1_u32);
let two = evaluator.splat::<Splat, _>(2_u32);
let three = evaluator.binary::<Add, _>(one, two);
let total = evaluator.reduce::<Sum, u32>(three); // a uniform u32
let running = <Executor as EvaluateScan<Scan<Sum, Exclusive>, u32>>::scan(&evaluator, three);
Operations like Sum, Max, ReduceXor, Inclusive, and Exclusive are all distinct types.
As mentioned in the post, execution shape is typed too. A static shuffle takes its control as a type-level constant, and the shuffle mode constrains which controls are expressible:
// Shift down one lane, keeping our own value where the source is inactive.
let down = <Executor as EvaluateShuffle<Shuffle<Down>, DownOrSelf<1>, u32>>::shuffle(&ev, v);
// Broadcast from lane zero.
let bcast = <Executor as EvaluateShuffle<Shuffle<Broadcast>, WarpLane<0>, u32>>::shuffle(&ev, down);
// Butterfly exchange with the neighbor one bit away.
let bfly = <Executor as EvaluateShuffle<Shuffle<Xor>, Butterfly<1>, u32>>::shuffle(&ev, bcast);
For an example of errors caught, a warp-scoped executor for a device-scoped barrier is a compile error:
<ScopedWarpExecutor<'_, WarpUniform> as EvaluateBarrier<Barrier<Device>>>::barrier(evaluator)
// error[E0277]: the trait bound `Device: NvptxBarrierScope` is not satisfied
// help: the trait `NvptxBarrierScope` is implemented for `Warp`
Strip mining is typed on the amount of work and the lane capacity, and it hands back one chunk at a time along with the predicate saying which lanes live in that chunk:
// Six work items across four active lanes: two chunks, based at 0 and 4.
<Executor as EvaluateStripMine<StripMine, (WorkItems, ActiveLanes<StripMined<4>>), i32>>::
for_each_strip_mined(
&evaluator,
(WorkItems::new(6)?, ActiveLanes::new(4)?),
|index, active| {
// ...
},
);
I'm confused too. How does this fit between these approaches for paraellization:
- CUDA kernels and Tiles (e.g. Cudarc, cuda-oxide, rust-gpu etc) - SIMD on the GPU. (E.g. as in the title...)
- CPU SIMD using avx or SSE instructions (And probably thin wrappers for vectors so you can have sane syntax). Or the maybe-upcoming core simd which should abstract over architecture-specific instructions. Magic floats etc which do 4-16 computations at once, but are a bit clumsy to work with
- Rayon thread pools - arbitrary parallel computations, including SIMD, one per CPU core.
It looks like from the code samples like maybe a cleaner syntax for writing code on the GPU than CUDA kernels? E.g. without mucking with serialization, host and device by abstracting over it? And inspired by core::simd. (Good choice if so, in the interest of standardizing on syntax; I did this for my x86 SIMD vector/quaternion lib as well)
This is really cool! It sounds like y'all have a compiler fork that you are using to make this work. I wanna tinker with this, is your compiler available?
If you have to express your computation using an "array programming DSL" with things like scan and gather anyways - why not opt to use torch/tensorflow/jax or anything else that targets MLIR? An example of writing a relu using an embedded array DSL is really not helping your case either - that's exactly the problem that these other solutions mentioned above are successfully solving for the past ~15y (starting with theano etc). Not sure what this brings to the table - doing that AoT instead of at runtime?
The goal of this work is to run existing unmodified CPU libraries (which may use core::simd) on the GPU. If you are manually writing ML-shaped workloads, it doesn't add any value over writing with tech like torch/tensorflow/jax which are custom built for those use-cases (except maybe familiarity if you are a CPU programmer).
Really like that the barrier scopes and shuffle controls are type-level, that is a lot of footguns turned into compile errors. Once you lower to PTX, does any of that structure survive for ptxas to optimize on, or is it fully erased by then? Also curious whether the strip mining abstraction fights the register allocator at higher lane counts, or if it stays friendly.
Oh, this again. I should put a website with this up...
I was the person who personally ran 10.6 security updates at Apple (10.6.1+), the "DRI". My team in the Updates Program office and I reviewed every single bug to determine if it should go in a security and stability update or wait for the next major version. Seriously, every morning we group triaged all Mac OS X bugs, both incoming and those nominated internally for us to look at and determine if it should go in an update. I packaged and audited the builds and tuned the delta vs full updates. I built the system that largely automated diffing "trains" for software updates (automastering).
The new version of the OS was always being developed in a branch/train, and fixes were backported to the current version as they were found. They weren't developed linearly / one after another. So, if you are comparing the most stable polished/fixed/stagnant last major version with the brand new 1.0 major version branch, the newer major is going to be buggier. That would be the case with every y.0 vs x.8. But if you are comparing major OS versions, Snow Leopard was different.
Snow Leopard's stated goal internally was reducing bugs and increasing quality. That is a fact, not marketing. I am not sure why people on the internet don't believe that, but I was there. If you wanted to ship a feature you had to get explicit approval from leadership and the bar was high. In normal feature releases it operated bottom up "here is what we are planning to ship" and in Snow Leopard it was top down "can we ship this?".
AFAIK Snow Leopard was the first release of this kind (the first release I worked on was Jaguar or Puma), and was a direct response to taking 8 software updates to stabilize 10.5 and the severity of the bugs found during that cycle and the resulting bad press. Leopard was a HUGE feature release and with it came tons of (bad) bugs.
The first .1 or .2 ALWAYS fixed critical bugs, because:
1. You had to GM / freeze the software to physically create the CDs/DVDs around a month before the release. Bugs found after this process required a repress (can't remember the phrase we used), which cost money and time and scrambled effort at the last minute and added risk. This means the bar was super high, and most "bad, but not can't use your computer bad" bugs were put in software updates...which was developed concurrently with the end of the main release (hence why .1 came out right away)
2. Testing was basically engineers, internal QA, some strategic partners like Adobe and MS, and the Apple Seed program (which was tiny). There was very little automated testing. Apple employees are not representative of the population and QA coverage is never very complete. And we sometimes held back features from seed releases when we were worried about leaks, so it wasn't even the complete OS that was being tested.
Software updates are always needed, though the issues they fix became less severe over time due to larger seeds (aka betas), recovery partitions, and better / more modern development practices. But I can tell you FOR A FACT that Snow Leopard had fewer major bugs over its lifetime, coalesced very quickly, and was extremely solid when Lion was released.
I was there too, but this blog post agrees with my memory.
I agree with you that Snow Leopard was billed as no USER VISIBLE features. But people did NOT just fix niggling bugs, it was an architecturally huge release. E.g., it introduced GCD. And unless I’m confusing the year, it rewrote the Mail backend to use it and mail hasn’t had reliable search since.
My memory is that quality was so low that we pushed back the release. With the extra time we did manage to fix enough bugs that it was well received. But internally it was more destabilizing than most releases due to the amount of architectural churn.
Snow Leopard shipped a new 64-bit Cocoa Finder too, but no one noticed because it was almost a pixel perfect and behaviour perfect copy of the previous one.
There was something else going on during that time period though.
Consumer operating systems in the 1990s were pretty bad. Windows 9x and MacOS Classic didn't even have protected memory. They crashed a lot.
Microsoft fixed this with Windows NT (delivered to consumers as Windows XP), Apple with BSD/NeXT (delivered as OS X). It was the point when ordinary people first got computer operating systems that were fully baked. But the earliest versions (Windows NT/2000 or NeXTSTEP/OS X pre-Snow Leopard) still had the problems that come with new code, so the later releases were better. Hence, the best version of MacOS was Snow Leopard and the best version of Windows was XP (or, depending on who you ask, 7). Because that's when they finally made a decent release.
Once we had operating systems that do essentially what operating systems are supposed to do, since then it has felt like nothing but lateral moves and deterioration because there wasn't much left on the table for newer versions to win but every update is another chance to break or lose something we used to have.
The impression of XP being the best version of Windows came after at least two service packs (patch bundles / minor releases, for those unfamiliar) and the Vista shitshow. First Windows release that was good from the very beginning was 7.
Ahem, the best version of Windows was 2000 / 2003 x64 server. It had all the improvements from real world experience gained by NT 3.5 and 4.0 and none of the consumer crap pushed in XP by the abandonment of the Win95 line.
That indeed was the best business edition of Windows. Consumers weren't supposed to have that though. The then-consumer edition was Windows ME, which... wasn't as good.
It's rewriting history to say that OS X didn't have a decent release before Snow Leopard. Panther and Tiger were both excellent releases and Panther was widely considered the first truly usable OS X release. That's when it got decent.
It was definitely a great release. I think that’s a big part of why it’s become legendary.
To me the myth is the “no new features”. Apple may have said that in places in marketing but there were new features. Grand Central Dispatch for example.
Yeah as someone else that worked on Snow Leopard I found this article and ones like it a bit silly. They're a lot like the old "debug code" posts on Mac forums back in the day trying to explain why Puma was slow (no, you've got a G3 and an unaccelerated video card).
Tiger and Leopard were sprawling releases. Tiger covered 32-bit and 64-bit PowerPC, then 32-bit x86, then 64-bit x86 Macs. Tiger was also what I guess I'd say is the first "modern" OSX release, with subsystems like launchd and a more mature OpenDirectory replacing older NeXT subsystems.
Leopard in my experience was a train wreck. It shipped a lot of new features, both user facing and back end frameworks, but the development was seriously impacted from teams losing engineers to iOS. Leopard had features planned assuming 100% availability of SWE's resources but ended up with 40-50% of SWE's resources (made up numbers based on my feels at the time). Many new frameworks in Leopard did not land or get stable until very late in the development cycle which meant everything downstream had to scramble right before GM. It was 100% the case that Leopard was not really stable until 10.5.8 and still sucked IMHO.
There was a lot of clean-up needed in MacOS and that was Snow Leopard. It was not a perfect release but 10.6.3 (the second disc pressing IIRC) and onward were very stable. Dropping PowerPC support really helped focus QA since they didn't need the massive array of test hardware Tiger and Leopard required. You also didn't need to wrangle builds for four architectures, including multiple compilers as a few libraries (if not whole frameworks) were compiled with ICC on x86 for performance reasons.
The "Snow Leopard wasn't that great" meme is weird. I'm sure it had its problems like every OSX release did. Unfortunately most OSX releases broke someone's workflow[0]. Snow Leopard ware no different. But it was always billed internally as a "no new features" release and those features that it did ship faced a very high bar to get in the release. I definitely look back on Snow Leopard as a high water mark for macOS quality in terms of a release.
From the outside, Tiger always seemed insanely multifaceted. Supported the four architectures you mentioned AND was the only release to have both Rosetta and Classic mode. Despite all that, it seemed pretty stable.
> 10.6.3 (the second disc pressing IIRC) and onward were very stable.
10.6.3 actually introduced a CFNetwork/NSURLConnection crash with HTTP 301 redirects that plagued many people including me and lasted way too many OS X updates afterward. (This was originally Radar 7816615, now FB6066729, see also https://openradar.appspot.com/7841731)
> Snow Leopard's stated goal internally was reducing bugs and increasing quality. That is a fact, not marketing.
That being a stated goal is not necessarily contradictory with claims that the release introduced bugs and had quality problems. Fixing bugs can introduce new bugs, or unveil existing bugs that were previously hidden by other bugs. It's quite possible that users like the author encountered bugs in their workflows that were the result of fixing bugs that didn't impact their workflows. It's also possible that they used applications that relied on behavior that was considered bugs in the OS, and thus broke when the bug was fixed.
> If you wanted to ship a feature you had to get explicit approval from leadership and the bar was high.
What was defined as a feature? The first new thing I noticed in Snow Leopard was that Quick Look could now preview Adobe Illustrator files (it was not the case in Leopard). I very much welcomed that, but it doesn’t seem like the kind of feature addition that would clear a high bar.
> So, if you are comparing the most stable polished/fixed/stagnant last major version with the brand new 1.0 major version branch, the newer major is going to be buggier. That would be the case with every y.0 vs x.8.
I ran Leopard and upgraded to Snow Leopard on a 2006 Core Duo MB Pro as well. My memory is very hazy on details, but I had remember no issues at all with either one. It was a difficult time in my life, as I my employer essentially laid everyone off in late 2009 and I spent the next 18 months unemployed. During that time my Mac was an utterly reliable means of job hunting, studying for certifications, and applying for unemployment.
1. Rust CUDA is over 5 years years old, was dead for 2(?) years, but was rebooted and works. It enables both rust on the GPU and controlling the GPU from the host. The host library (cust) predates cudarc. If it was started today it would just use cudarc for the host side (and indeed, you can use the device side with cudarc). Rust-cuda is based on nvvm, which was the supported layer in the past but Nvidia is moving away from.
2. Rust-gpu (vulkan) works but isn't 100% complete of course. The dimforge folks are using, there are some crypto folks using, and we have llms written in it. Rust-gpu is only the "running rust on GPU part", the host is left to wgpu or ash or whatever.
The Nvidia projects are great (we gave pre-release feedback on them), but they were not announced when the call for presentations went out. They were also released as experimental with only 2 people working on them so Nvidia was cautious about marketing and over-committing.
We are the only people trying to bring rust natively to the GPU rather than just making the GPU work with rust (which others like cubecl, wgpu, cuda-oxide handle well). This has some interesting considerations from a rust language and compiler standpoint, which is what was talked about at the conf.
We sponsored an unconf room and got as many folks from all the various GPU projects together, including nvidia. The big problems right now are a) everybody has different needs, b) there are very few people working in the space c) the entire rust project is generally indifferent to GPUs. I expect this to change in the next year or two.
That is great context and promising for the ecosystem as a whole. I owe it to myself and you to try it again. I think I will take an existing code base (I have a molecular dynamics engine in rust) that uses CUDARC, and I will write similar Cuda Oxide and Rust-Cuda implementations to compare. Would love to see what each does well. (Bearing in mind Cuda Oxide in particular may change its API heavily; IIRC I had an issue installing it in the past which blocked me, but I don't recall what specifically)
There is also a Cuda-Tile rust lib which I have been meaning to try, but I haven't used tiles so far.
Quite honestly I would use cuda-oxide over rust-cuda for anything new right now....while rust-cuda is more complete, cuda-oxide is more aligned with nvidia's future direction (when we were rebooting rust-cuda we were told to by nvidia to stay with the nvvm layer but their plans changed and it is clearly not where their focus is). It's of course just rust so you could just feature-gate off differences as 80% will probably be the same if you wanted to try both projects.
cuda-tile is great...I would use it for anything written from scratch as it is closer to how the hardware (and projects like triton) work. But there are tradeoffs as it requries your code to be structured a certain way.
Egraphs are for optimization passes, which operate on the same IR, and (without egraphs) may undo and/or prevent each other and are repeated for more optimization.
Nanopass is compiler passes, which each have their own IR, and run once in a fixed sequence.
Egraphs also require the IR to be defined a specific way, which prevents some optimizations. My understanding of https://github.com/bytecodealliance/rfcs/blob/main/accepted/... is that cranelift’s egraph optimizations are pure expression reordering/duplication/deduplication and rewrite rules.
There is no reason why compiler passes cannot be in an egraph, they are more general than optimizations. When you think about it, traditional compiler concerns like instructions selection are sort of a optimization problem if you squint.
Egraphs fundamentally require a first-order language, i.e. no lambda functions, because they do not handle scoping well. There are workarounds that work in some circumstances, but no general solution (that I'm aware of).
Additionally, egraphs express rewrites on a single language. Compilers are fundamentally translators between different languages; how do you write typechecking as an egraph rewrite rule? Or conversion to assembly?
