>The chips are essentially highly parallel processors
Right. AFAIK we already were doing SIMD, Vector Processing, VLIW etc. to speed up parallel processing for numerical calculations in AI/ML. What then is the reason for the explosion of these different categories of chips? Are they just glorified ASICs designed for their specific domains or FPGAs programmed accordingly? If so what is their architecture i.e. what are their functional units and how do they work together?
Realistically, a good AI chip will have provisions for high throughput IO (the most important thing, and the key differentiator) and the actual processing really doesn't matter because with enough engineering effort you'll be able to saturate the chip.
GPUs have a high speed output in the form of an HDMI link. However, there is no high speed input. Reads/writes to/from the GPU are slow. The Cerebras wafer chip for example has 8-16 FPGA driven IO chips that directly read from TCP/IP onto the chip and off again in parallel. Each FPGA connects to its own ethernet port. So you can get the data on/off the chip as fast as possible. That's it really.
As for the processing engines. They're usually just standard cores with a high speed interconnect and maybe some matrix multiplication optimizations. Some, like Groq, have a high speed fabric with specialized processors at various locations.
Right. AFAIK we already were doing SIMD, Vector Processing, VLIW etc. to speed up parallel processing for numerical calculations in AI/ML. What then is the reason for the explosion of these different categories of chips? Are they just glorified ASICs designed for their specific domains or FPGAs programmed accordingly? If so what is their architecture i.e. what are their functional units and how do they work together?