The autoencoder that maps between that and the 512x512x3 RGB space was trained together with the model, so it is specialized in upscaling the 64x64x4 info to pixel space for this particular purpose. It's "just" a factor of 48 (de)compression..
I think it’s considered an implementation detail so it’s just not mentioned. A lot of models still work if you round the values to FP16, which personally I find surprising.
The way I think of it, we have a 512x512x3 target, so that's 48x the information. I don't think it's unreasonable to say that far less than 1/48th of the space of 512x512x3 outputs are natural images (meaning an image that might actually exist, rather than meaningless pixels). So if we think about that 64x64x4 tensor as telling us what in the smaller space of natural images we should draw, it seems like plenty of information. Especially since we have the information stored weights of the output network also.
The amount of information in a 64x64x4 array would depend on the precision of the numbers in it, right? For example, a 512x512 image in 24-bit colour could be completely encoded in a 64x64x4 array if each of the 64 x 64 x 4 = 16,384 values had 384 bits of precision.
So, I wonder — what's the minimum number of bits of precision in the 64x64x4 array that would be sufficient for this to work?
According to a anecdotal test on one of the images found elsewhere in this thread, JPEG comression at 80% quality can chop a factor of 16 off the size of a 24bpp .bmp file.
384 / 16 = 24bits per [64x64x4-] array value. Integer range of 32bit float is 2^24. So "literally just a jpeg packed into 16K floats" is a option.
