> AMD doesn't need to duplicate everything NVIDIA provides, they just need to duplicate the parts relevant to most of the $3T spend the market seems to be expecting.
In effect, they already have. Both AMD, Apple and a number of smaller OEMs all wrote GPU compute shaders to do "the AI inference stuff" and shipped it upstream. That's about as much as they can do without redesigning their hardware, and they've already done it.
Nvidia wins not because they have everything sorted out in software. They win because CUDA is baked into the design of every Nvidia GPU made in the past decade. The software helps, but it's so bloated at this point that only a small subset of it's functionality is ever used in production at any one point. What makes Nvidia fast and flexible is integration of compute in hardware at the SM-level. This is an architecture AMD and Apple both had the opportunity to reprise, even working together if they wanted, but chose not to. Now we're here.
I tend to steelman the idea that it was AMD and especially Apple's mistake for eschewing this vision and abandoning OpenCL. But apparently a lot of people tend to think that AMD and Apple were right despite being less efficient at both raster and compute operations.
With CUDA a researcher can use C++20 (minus modules), Fortran, Julia, Python, Haskell, Java, C#, and a couple of other stuff that compiles to PTX, have nice graphical debuggers for the GPU, IDE integration, a large ecosystem of libraries.
With OpenCL, C99, some C++ support, printf debugging, and that is about it.
For good C++ experience, one needs to reach out to Intel's Data Parallel C++, which has Intel's special sauce on top of SYCL efforts, which only became a reality since a British company specialised in compilers for game consoles decided to pivot their target market and produce ComputeCpp, followed by being acquired by Intel.
In effect, they already have. Both AMD, Apple and a number of smaller OEMs all wrote GPU compute shaders to do "the AI inference stuff" and shipped it upstream. That's about as much as they can do without redesigning their hardware, and they've already done it.
Nvidia wins not because they have everything sorted out in software. They win because CUDA is baked into the design of every Nvidia GPU made in the past decade. The software helps, but it's so bloated at this point that only a small subset of it's functionality is ever used in production at any one point. What makes Nvidia fast and flexible is integration of compute in hardware at the SM-level. This is an architecture AMD and Apple both had the opportunity to reprise, even working together if they wanted, but chose not to. Now we're here.
I tend to steelman the idea that it was AMD and especially Apple's mistake for eschewing this vision and abandoning OpenCL. But apparently a lot of people tend to think that AMD and Apple were right despite being less efficient at both raster and compute operations.