It's 24 x64 cores and 228 CDNA compute units on a common 128GB of memory. Personally I want to run constraint solvers on one. The general approach of doing lots of integer/float work on the GPU and branchy work on the CPU, both hitting the same memory, feels like an order of magnitude capability improvement over the current systems.
> Unfortunately the BoM on these things is probably super high.,, even for a datacenter GPU
You would save money on improved yields of chiplets and on using cheaper nodes where appropriate. I imagine that would help offset the increased packaging costs.
However, they are moving to compete in a market where the profit margins are so obscene that a small increase in the cost of materials really wouldn't be an important factor.
> Nvidia Makes Nearly 1,000% Profit on H100 GPUs: Report
And that's the consequence of giving the market leader a decade and a half head start. No one and I mean no one was preventing AMD or Intel from building a competitive hardware and software ecosystem other than themselves.
It was a David against two Goliaths situation until recently.
Five years ago when AMD was recovering from near bankruptcy, it was battling Intel (100K+engineers) in CPUs and Nvidia (20K+ engineers) in GPUs
It was doing that with just 10K engineers and a fraction of the R&D budget of the giants.
Things have changed now. AMD has more than doubled their headcount and R&D budgets.
Within the last year the company has pivoted to AI as their main focus.
In the case of AI computing, my layman's perspective of the last decade is that NVIDIA created a market where one did not previously exist "at scale" as a long bet that is paying off significantly.
It seems unfair to criticize AMD or Intel for placing losing bets on other market segments when things could have just as easily turned out differently.
It could be argued that Nvidia sabotaged OpenCL. Part of the reason that nobody used OpenCL was because that it didn't work very well on Nvidia hardware, which was the dominant hardware. So unseating Nvidia was an impossible chicken/egg problem. To unseat Nvidia you needed OpenCL to succeed but OpenCL couldn't succeed without good support Nvidia.
OpenCL sucked for other reasons too, but it likely wouldn't have succeeded even if it didn't suck.
That'd be a tough argument. Intel and AMD aren't responding to CUDA by pumping resources into OpenCL, they're building completely new libraries. That suggests NVidia was frustrated by its partners in the OpenCL design process and just built their own system that worked.
Yeah, NVIDIA was frustrated by disabling memory channels (3 concurrent transfers for CUDA, 1 for OpenCL) and not providing real support while advertising support for newer OpenCL versions (builds failing with cryptic compile errors). Causing OpenCL code to perform poorly and preventing it from running on their own hardware in some cases.
Other than locking out the technologies they see threatening, they’re completely a fair company, if you want to believe in that.
Yeah, I can't reasonably buy one. But rent one to do some maths? Good odds.
There are some existing APU systems - I use low wattage ones as thin clients. Currently thinking it should be possible to write code that runs slowly on a cheap APU and dramatically faster on a MI300A system. Debug locally, batch compute on AWS (or wherever ends up hosting these things).
Bill of Materials should really only be used for things that require a list of items/labor, and aren't sold individually such as a datacenter, a building, a hardware integration project, a wedding, etc. For things that are sold individually, "cost" will suffice, and BoM is an example of incorrectly using a more complicated term for the sake of seeming smart.
As a shorthand for expected retail cost, that is terribly misapplied. Or even for internal cost.
The chip would be one line item on the BOM for the entire thing you plan on shipping (but not packaged yet). Even in the case you are "just" selling a chip, the BOM is likely more complicated,and in this context primarily the manufacturing side cares about that. The COGS (cost of goods sold) is something the company as a whole will care more about - this is what it actually costs you to get it out the door. You will hear "BOM cost" referring to the elemental cost of one item on the BOM, but that's not the BOM itself.
None of these are related to the retail (or wholesale) cost in a simple way, either than forming a floors on long term sustainable price.
The GGG-whatever post is using this sloppily to suggest that the chips are going to be very expensive to produce, therefore the product is going to be expensive.
You'll also see BOM in a materials and labor type invoice, (like when you get your car serviced) but that's not relevant here.
What I really meant was total production cost + reasonable amortization cost for the development/tape out, which is of course is huge even if this chip was mass produced.
The point kinda stands though, this thing is collectively way too hard to fab + assemble to sell to consumers at any reasonable cost, especially on top of the massive R&D AMD, TSMC and everyone along the huge chain put into it.
Its not like a 4090 or Apple Silicon where the production is reasonably cheap, but margins are super high because they can be super high.
What tool can you advise me to find the best combination of dividers and multipliers in PLL clocks and dividers in the UART and SPI peripherals so that the deviation from the assumed baudrate is as small as possible and that the dividers meet a number of constraints, including e.g. those from the chip errata which says that one PLL must be 2 times faster than the other?
Not that AMD doesn't deserve any credit, they have considerable multi chip experience under their belt and undoubtedly served as a guinea pig/pipe cleaner for TSMC's advanced package.
I think the other parameters are way more difficult to get right than any optical system alignment considerations. Things like statistical process control make photolithography look less scary.
You can check if you correctly patterned the wafer almost immediately. You won't know if the layer is any good until many process steps later. Maybe not for sure until EDS. Tuning the interactions between manufacturing processes is the actual secret sauce that all the manufacturers are trying to protect. How much dose on the EUV machine depends a lot on how you intend to etch the wafer. Imagine iteration cycles measured in months for changing individual floating point variables.
The contraints all depend on the scale of the alignment required.
I wonder what is the wavelength range used for their interferometers, and what kind of mecanical engines they use (probably piezo electric based engines).
What exactly are you missing when hipifying your CUDA codebase? For most of the software I've looked at this has been a breeze, mostly consisting of srtting up toolchains.
Or do you mean the profiler tooling?
I hear everyone say that AMD doesn't have the software, but I'm a little confused --- have you tried HIP? And have you tried the automatic CUDA - HIP translation? What's missing?
I think they're referring to support in general. Technically HIP exists, but it's a pain to actually use, it has limited hardware support, is far less reliable in terms of supporting older hardware, needs a new binary for every new platform and so on.
CUDA runs on pretty much every NVIDIA GPU, this year they dropped support for GPUs released 9 years ago, and older binaries are very likely to be forward compatible.
Meanwhile my Radeon VII is already unsupported despite still being pretty capable (especially for FP64), and my 5700XT was never supported at all (I may be mixing this up with support for their math libraries), everyone was just led on with promises of upcoming support for 2 years. So "AMD has the software now" is not really convincing.
I suppose if you're talking about consumer cards, I agree, support is often missing.
But if we're talking datacenter GPUs, the software is there. Data centers is where most GPGPU computing happens after all.
It's not ideal when it comes to hobby development, but if you're working in a professional capacity I'm assuming you're working with a modern HPC or AI cluster.
Well, to provide my own experience bringing GPU acceleration to scientific computation software, AMD got passed on because even if the software was there at the time (it wasn't), there was no way to justify spending a bunch of money buying everyone workstation AMD GPUs when we could just start by picking up basic consumer NVIDIA cards to work on for anyone who didn't happen to have one already and then worry about buying more suitable cards if needed.
Of course the end goal was to run on a large HPC cluster we had access to, but for efficient development, support on personal machines was necessary. My personal dual 3090 setup has been invaluable for getting through debugging and testing before dealing with the queueing system on the cluster (Side note: it also ended up revealing another important benefit of consumer side support for GPGPU, a 3090 was easily matching the performance of a single node of the CPU-only version of the cluster, thus massively bringing down the cost of entry to an otherwise computationally restrictive topic).
This is a very valid point. 100% AMD need better support of consumer cards to get research, university and everyday folks using their SW. The high end consumer7900 xtx are supported now, and a lot of other cards not officially supported actually work. (I've a colleague who tried it on his home computer and got it working, despite that card not being on the official list) .. . Still, AMD need to be getting more cards working ASAP
The chip industry is sure. But are the customers? The customers who cared are jaded by nearly 15 years of intel and amd utterly failing to make a compelling alternative and likely have a large existing investment in CUDA based GPUs.
> but no customer wants to give Nvidia monopoly money forever either.
From a consumer perspective, I agree. From a datacenter, edge and industrial application perspective though, I think those crowds are content funding an effective monopoly. Hell, even after CUDA gets dethroned for AI, it wouldn't surprise me if the demand continued for supporting older CUDA codebases. AI is just one facet of HPC application.
We'll see where things go in the long-run, but unless someone resurrects OpenCL it feels unlikely that we'll be digging CUDA's grave anytime soon. In the world where GPGPU libraries are splintered and proprietary, the largest stack is king.
I support this product that uses off the shelf nvidia GPUs to do <thing> on the computer they use for <big machine>. I see a lot of IT deps and MSPs asking if they can use AMD gpus because apparently a lot of dells come with em or something. I always have to tell them no and to just buy a P1000 or one of the other cheapo quadros instead and I hate it.
Whether it's Microsoft with Win32, AMD with x86-64, or Nvidia with CUDA, the winner is always the guy who enable people to do things with their computers.
Meanwhile, priests like Intel with Itanium, Microsoft with WinRT, FOSS nerds, or AMD with GPUs will continue failing because most people ain't got no time to be preached to about how they achieve something.
It feels like playing software catchup in this fast moving sector is very challenging. NVIDIA+CUDA is kind of a standard at this point. AMD's CPUs still ran Windows. AMD's GPUs still ran DirectX and OpenGL. This feels different.
Eh, 16 years ago CUDA was the cheap option, compared to other HPC offerings.
And there wasn't a parts shortage (modulo some cryptocurrency mining, but that impacted both GPU vendors)
And ML models weren't so large as to make 8GB of vram sound meagre.
And there weren't a bunch of venture capitalists throwing money at the work, because the state of the art models were doing uninspiring things. Like trying to tag your holiday photos, but doing it wrong because they couldn't tell a bicycle helmet and a bicycle apart.
The memory isn't designed to connect to a lot of different chips. So there's no reason to put it in the center.
The core systems of interconnect are at the base most center-most (well, there's a passive interposer too, but it's just wires): the IOD. These intermediate connections across the chips on top of them, the IOD next to them, and the memory.
HBM is designed for the wire lengths that you're talking about. The core protocol isn't so terribly different from good old DDR4.
The die-to-die links, on the other hand, have extremely short wire length limits (less than a centimeter iirc for UCIe). At the physical layer, you just waste a bunch of power and area driving a high voltage to make the signal go further. Do that enough and you basically wind up with a PCIe PHY.
Each memory chip is connected to a single DRAM controller. Those DRAM controllers are what you'd want to be close to each other to minimize NUMA effects.
AMD's compute oriented cards used to come with displayport output, but I haven't seen one of those in a long time. These cards are definitely GPUs in that they can handle graphical workloads, but I don't think anyone is trying to make them work for video games or the like.
I heard a rumor somewhere that Stadia ran on MI25s - not sure if that's true but certainly there have been a lot of them floating around on eBay in the past year.
ROCm, which is AMD's equivalent of CUDA. The thing is you don't have to directly interface with CUDA or ROCm. Once the framework you want to use supports these, you're done.
AMD is consistently getting used on the TOP500 machines, and this gives them insane amounts of money to improve ROCm. CUDA has an ecosystem and hardware moat, but not because they're vastly superior, but because AMD both prioritized processors first, and NVIDIA played dirty (OpenCL support and performance shenanigans, anyone?).
This moat is bound to be eaten away by both Intel and AMD, and compute will be commoditized. NVIDIA foresaw this and bought Mellanox and wanted ARM to be a complete black box, but it didn't work out.
Ethernet consortium woke up and got agitated by the fact that only ultra-low latency fabric provider is not independent anymore, so they're started to build alternatives to Infiniband ecosystem.
Interesting times are ahead.
Also there's OneAPI, but I'm not very knowledgeable about it. It's a superAPI which can target many compute platforms, like OpenCL, but takes a different approach. It compiles to platform native artifacts (CUDA/ROCm/Intel/x86/custom, etc.) IIRC.
TOP500 has had all sorts of "odd" systems near the top. Unusual or custom architectures and fabrics are far more viable for "this is our nation's nukeputer" or "this is going to run the nation's weather forecasting model" type of systems compared to e.g. a commercial HPC system where you wanna run all sorts of commercial/proprietary software. AMD being successful in the former niche doesn't mean their software stack is viable for the latter niche.
The systems running AMD cards are not "Nukeputers", and not all "Nukeputers" and "Fusionputers" are running on custom silicon.
When you look to latest list [0], #1 system, Frontier, is a Nukeputer, but it's not only a Nukeputer. #5, LUMI is definitely not a Nukeputer. They are very close to us, we work together under a project with them (we're equals. They operate under a different consortium, we have stakes on another computer which is also very famous and also in top 10 in TOP500, and is not a Nukeputer). We also have our smaller systems in our own datacenter.
We operate mostly in the long tail of science, and this means we see heavy use of popular software packages, and many special software packages optimized for these long tail problems. ROCm was invisible in this niche before, but with Frontier and LUMI, and with AMD's announcements, this started to change very quickly.
ROCm libraries are more open w.r.t. their CUDA counterparts, and they started to land to mainstream distribution repositories directly. This is important for our niche. AMD is sponsoring integration of their libraries to popular packages and it started to payoff already.
Also, small-guy-programmers start to optimize LLM training routines for AMD cards, getting 99% of the performance of NVIDIA counterparts with way less power consumption.
As a result, AMD is already much more visible and more capable position when compared to last year.
Everything is possible, especially what they have in mind, where they will do a custom implementation. CUDA is important for the existing ecosystem, but that doesn’t make it the only show in town.
AMD has released HIP and a tool called HIPIFY which kind of behaves like this but at the source level¹. Rather than try and just translate CUDA to work on AMD compute they are more focused on higher level tooling.
Currently they seem to have a particular focus on AI frameworks and tools like PyTorch/Tensorflow/ONNX. They have sponsored and helped with a lot of PyTorch development for example, so PyTorch support for AMD is much better than it was this time last year².
Nvidia could build that but doesn't want it to exist. Outside of nvidia you'd have to reverse engineer their machine code which would be a massive undertaking. The ISA ISA is published by AMD and Intel, you could build tooling from the docs alone if you wish.
Video is getting a lot of direct ASIC blocks (look at the latest VPE block in AMD GPUs).
I guess those chips are for the movies/video industry, online or not. Because for the "consumer", the CPU is already very efficient, and I don't think we would save interesting about of battery usage in a "real usage" perspective. I may be very wrong, but I don't watch hours and hours in a row of ultra high quality videos on a small screen, that off the AC plug, the battery is unusable in a matter of a few years anyway... it not less.
Encoding video in real time is expensive. You can run your blog on a 5€/month machine. You'll most likely need ten times that money for a single real time stream using nothing but software decoding, rendering and encoding. If you think, "hey I'm going to use a GPU for this" then congratulations, you've increased your cloud bill by a factor of 20.
Lastest AMD GPU does realtime encoding with AV1. It has directly an ASIC block for that. Just mmap the video engine command circular buffer, get some dma buffers, get a interrupt circular buffer, and you are good to go (I think the dma-buffer sync framwork between drivers and user space is still a WIP, and I dunno how the interrupt circular buffer is handled).
If you're looking for fair comparisons don't ask nVidias marketing department, those guys are worse than Intel.
What AMD did was a true comparison, while nvidia is applying their transformer engine which modifies & optimizes some of the computation to FP8 & they claim no measurable change in output. So yes, nvidia has some software tricks left up on their sleeve and that makes comparisons hard, but the fact remains that their best hardware can't match the mi300x in raw power. Given some time, AMD can apply the same software optimizations, or one of their partners will.
I think AMD will likely hold the hardware advantage for a while, nVidia doesn't have any product that uses chiplets while AMD has been developing this technology for years. If the trend continues to have these huge AI chips, AMD has a better hand to economically scale their AI chips.
Not my area, but isn't a lot of NVIDIA's edge over AMD precisely software? NVIDIA seem to employ a lot of software dev (for a hardware company) & made CUDA into the de facto standard for much ML work. Do you know if AMD are closing that gap?
They have improved their software significantly in the last year, but there is a movement that's broader than AMD that wants to get rid of CUDA.
The entire industry is motivated to break the nvidia monopoly. The cloud providers, various startups & established players like intel are building their own AI solutions. Simultaneously, CUDA is rarely used directly, typically a higher level (Python) API that can target any low-level API like cuda, PTX or rocm.
What AMD is lacking right now is decent support for rocm on their customer cards on all platforms. Right now if you don't have one of these MI cards or a rx7900 & you're not running linux you're not going to have a nice time. I believe the reason for this is that they have 2 different architectures, CDNA (the MI cards) and RDNA (the customer hardware).
That's what I have (RX 7900XT on Arch), and ROCm with pytorch has been reasonably stable so far. Certainly more than good enough for my experimentation. Pytorch itself has official support and things are pretty much plug & play.
> Given some time, AMD can apply the same software optimizations, or one of their partners will.
Except they have been given time, lots of it, and yet AMD is not anywhere close to parity with CUDA. It's almost like you can't just snap your figures and willy-nilly replicate the billions of dollars and decades investment that went into CUDA.
That was a year ago. AMD is changing their software ecosystem at a rapid pace with AI software as a #1 priority. Experienced engineers have been reassigned from legacy projects to focus on AI software. They've bought a number of software startups that were already developing software in this space. It also looks like they replaced the previous AMD top level management with directors from Xilinx to reenergize the team.
To get a picture of the current state which has changed a lot, this MS Ignite presentation from three weeks ago may be of interest. The slides show the drop in compatibility they have for higher levels of the stack and the tools for translation at the lower levels. Finally there's a live demo at the end.
It's 24 x64 cores and 228 CDNA compute units on a common 128GB of memory. Personally I want to run constraint solvers on one. The general approach of doing lots of integer/float work on the GPU and branchy work on the CPU, both hitting the same memory, feels like an order of magnitude capability improvement over the current systems.