On that kind of process, you could make a 1024-core Z80 machine, leaving half the area for memory, interconnect and I/O. With suitably smart programming and an embarrassingly parallel problem, it might even be able to beat a Pentium III in performance... although it looks like the single-core Pentium III can run 128-bit SSE instructions at 2 cycles per instruction.
I think those two combined already will cost you a factor of around 100 in speed versus that pipelined Z80, much more versus a Z80 proper.
Things get worse if you want to add or subtract 32- or 64-bit integers (another factor of 2 or 4, ballpark)
If you want to do integer multiplication and division of any size and all floating point operations you will have to do those in software, and likely lose whatever speed advantage you might still have.
O, and each core will be limited to 64kB of memory. Those interconnects better be fast and use DMA, so you can keep computing while you shuffle data around.
Yes, there are definitely challenges, and there must be a reason no one is seriously selling 1024-core versions of old 8-bit processors... but perhaps on an Intel 130 nm process you could make a faster Z80 than just 50 MHz. Quick googling didn't reveal what process Zilog is currently using for eZ80.