If you do so, add ability to reduce # of colors to speed up refreshes. Clients to whose machines I am connecting to often have lousy upload bandwidth.
I'm not sure how hard is it to send events (window move/resize, mouse move etc.) instead of compressed image of invalidated screen region. Even if that's usable only in special cases, it's worth doing.
