By the time you get to 2400bps you're starting to stress how fast the abysmal disk interface on the C64 could write data, you had better not hit too many seeks.
The 1541 drive can write at ~400 bytes per second, much faster than the 2400bps modem could deliver (240 bytes/s). And buffering was a thing back then. A simple ring buffer would suffice. I had no problems downloading at 2400bps after I fixed the serial code.
Faster, but not "much faster". Depending on where you were writing it could be as low as 300 bytes/second. You're starting to cut close to the margins, especially if the C=64 is having to also manage the modem communications at the same time. So yeah, doable but that's about the max you can expect out of the hardware. A 4800 baud modem would be right out.
The C64 doesn't control the disk drive directly, the 1541has its own CPU, about as powerful as the C64.
> A 4800 baud modem would be right out.
Not with a "Warp Speed" cartridge. It does write speeds on a stock C64/1541 up to 2900 Bytes per second. I'm pretty sure reading serial data faster than 2400bps is easily doable too.