I think the intention of the phrase, is that most cs programs include a systems course. you will learn von neumann arch, with cpu including {registers, clock, ALUs, etc}. Obviously modern CPUs have a lot more complexity, but even the basic coverage of CPUs is more coverage than GPUS.
Everything you learn about CPUs in your CS undergrad program is applicable to GPUs. Both are examples of von Neumann architecture, both have registers, ALUs, ISA, instruction schedulers, cache hierarchy, external DRAM, etc. The main difference is how they are being used - GPU, while a general purpose computer, is typically used as an accelerator for specific workloads, and it needs CPU to function. The distinction between the two is getting blurred as GPUs get better at executing control flows, and CPUs get more cores and wider vector operations.
