This assumes that only messages from controlled sources will be received though, right? If you're receiving messages over a TCP socket or something similar, that seems like a potentially flawed assumption; I'd think anything parsing messages coming from the network should be written in a way that explicitly accounts for malicious messages coming from the other side of a connection.
EDIT: I'm also still not any more convinced that four bytes per message would ever be a bottleneck for any general purpose protocol, but I'd be curious to hear of a case where that would actually be an issue.
> This assumes that only messages from controlled sources will be received though, right?
I don't think so. The question of whether you trust the length indication to be correct (you almost certainly shouldn't) seems to me to be independent of whether the length indication comes from inside the message or from some outside wrapper.
I might have misunderstood what you were suggesting; from rereading, it sounds like you're suggesting to rely on something like the end of a TCP packet rather than having an explicit length. If this is what you mean, my concern would be that requiring a protocol to map 1:1 to the transmission protocol's packets (or a similar construct) can be limiting; it would require all messages to fit in a single buffer, which could prevent it from working with clients or servers configured to use a different length and might make it difficult to use with other transmission protocols.
My question in the beginning of this thread was intended to be specifically about general purpose formats like protobuf; I think relying on the semantics of TCP or something like that might be a good choose for a bespoke protocol, but it doesn't seem like a great idea for something expected to be used in a wide variety of cases.
For a slow protocol like protobufs that is rarely streamed, I agree a length prefix should be the default
One way to make streaming work is just to allow the length value to be bigger than needed and add a padding scheme at the end of the message. This is overhead free in terms of processing time since fields must be decoded sequentially anyway.
EDIT: I'm also still not any more convinced that four bytes per message would ever be a bottleneck for any general purpose protocol, but I'd be curious to hear of a case where that would actually be an issue.