Zen 5 really is nice. I swapped my RAM from a Xeon W Sapphire Rapids machine into a Threadripper Pro 9000 series machine and I get almost double the memory read performance, plus it's a heck of a lot faster in single and multi core performance, and it runs cooler and quieter. Huge win all around. Aside from the price... (I went from Xeon W5-3435X to TR Pro 9985WX, eep.)
I went the opposite way and went back to Intel after many many years on AMD. I snagged a Xeon 654 Granite Rapids ($850) for a new workstation geared towards local inference. I don’t need a ton of CPU cores and all Granite Rapids have 8 memory channels where as you need very high end Threadrippers (9975wx for $4000) for an equivalent due to their CCD design.
As a bonus Granite Rapids is super power efficient and runs much cooler than my previous TR and Ryzens. Intel seems to be doing good things again! My only minor complaint is that P2P doesn’t work on my multiple GPU setup because every PCIe5x16 lane has its own dedicated root to the CPU, but all that bandwidth is useful for MoE models that are offloaded to RAM.
You could get a PCIe switch backplane, stick your GPUs on there and then have them communicate locally through the switch using p2p. For some GPUs you may have to use a hacked driver (google p2p 3090 for instance).
C-Payne makes excellent stuff but it is very pricey and supply is more than a little spotty, I've used pretty much all of his stuff by now and I am a very satisfied customer if I can order what I need, which more often than not is not the case.
A good alternative is ADT, they make excellent boards, there is a 4 and a 5 slot PCIe expander with a 88096 on it that works extremely well. I have two of these connected to my rig with their own power supply and four GPUs in each (and another two in the main machine). It's not exactly a portable affair (to put it mildly). Note that these won't work in a standard PC case due to the slot spacing so you'll have to rig something for that yourself (I use 2020 + some custom 3D printed fixtures).
Yeah, I had to go high-end to get one with enough CCDs to make use of the memory channels. I'm curious to know what all-read bandwidth you see when you run intel mlc on your system. My Xeon W5 was doing ~180GB/s after a lot of tuning, which was very disappointing. The Threadripper does ~320GB/s. From what I've heard, Granite Rapids is supposed to have solved the bandwidth problems that ailed Sapphire Rapids, so your numbers ought to be similar.
I have the complete opposite experience. Got a 9550x with 64gb ddr5 and a fairly high end mobo about two year ago. Just running the memory at stock speed. About 50% of the time I’d reboot and one or both of the sticks would only be detected as 2gb. Would need to do a hard shutdown to get it back.
I eventually gave up and turned off memory context restore and now I just deal with the minute plus (!) time to Post.
Not sure if amd memory controllers are just garbo or what but I’m going back to intel next chance I get.
On the desktop Intel has better memory controllers last time I looked, but it’s not that big a difference. Something in your setup is just broken. Failures will happen with any brand, so I wouldn’t chalk it up to AMD vs Intel… figure out where your problem is and get the faulty part replaced if you can.
I feel you though, regardless of the cause, that is an extremely frustrating place to be.
Yea I’m just bummed because it’s a real expensive time to be swapping memory parts around. Like I said, things seem to just work without memory context restore so I’m fine with just paying the cost when I reboot once a week or whatever. I feel like the fact that disabling MCR fixes it should narrow down the issue to some part, I’m just not sure which.
Did you set it to do memory training on every boot? That's a big stability help.
Should also run Memtest86+ a few times. With some stick swapping you can determine if you have a bad stick or slot.
Yea I’ve done endless googling around the issue and it seems to be a not uncommon issue with ddr5 but there’s no way I’m buying new parts now so I’ve come to peace with it.
