I mean the cost of hitting the disk/socket. The actual IO. There is also the syscall overhead but that doesn't matter /that much/ with IO usually, in comparation with the actual IO (unless, again, you are doing something very wrong - very small buffers etc ...). This of course depends on the deatils. And yeah, if it uses mmap then that syscall problem is no more. Problem I was refering was actual IO cost per message (fsync/msync per message?) to mantain consistency instead of caching for example or batching and grouping IO at the expense of consistency guarantees (maybe) or coherent data re. messages in realtime.
I agree that the stack is usefull and I see your point. But I would almost never resort to alloca. That is not a solution IMO. If you have that kind of costs with the joining of keys there are other solutions, I think. What about a dynamic string library, that over allocates (maybe a big chung on the first time in the 'new message' path) and then just uses the extra memory until you run out. Classic (and silly) strategy of allocating like n << 4 bytes every time you run out of space. This way you really just have a few allocations instead of hundreds maybe making it a non-issue (at the cost of prob. unused extra memory). Not as efficent as alloca but very close.
I you want to get fancy you can have efficent malloc pools to. But I think a semi-clever dynamic string lib would be able to handle the problem best.
I mean the cost of hitting the disk/socket. The actual IO. There is also the syscall overhead but that doesn't matter /that much/ with IO usually, in comparation with the actual IO (unless, again, you are doing something very wrong - very small buffers etc ...). This of course depends on the deatils. And yeah, if it uses mmap then that syscall problem is no more. Problem I was refering was actual IO cost per message (fsync/msync per message?) to mantain consistency instead of caching for example or batching and grouping IO at the expense of consistency guarantees (maybe) or coherent data re. messages in realtime.
I agree that the stack is usefull and I see your point. But I would almost never resort to alloca. That is not a solution IMO. If you have that kind of costs with the joining of keys there are other solutions, I think. What about a dynamic string library, that over allocates (maybe a big chung on the first time in the 'new message' path) and then just uses the extra memory until you run out. Classic (and silly) strategy of allocating like n << 4 bytes every time you run out of space. This way you really just have a few allocations instead of hundreds maybe making it a non-issue (at the cost of prob. unused extra memory). Not as efficent as alloca but very close.
I you want to get fancy you can have efficent malloc pools to. But I think a semi-clever dynamic string lib would be able to handle the problem best.