
VRB – Virtual Ring Buffer - ingve
http://vrb.sourceforge.net/
======
wrl
For a similar end result (a circular buffer from which all returned ranges are
contiguous) but without OS support for mmap()ing, there's also Bip Buffers[0].
I did an implementation in C [1] designed to be used for zero-copy IPC over
shared memory (by way of having a structure with a flexible array member for
the actual data).

[0]: [http://www.codeproject.com/Articles/3479/The-Bip-Buffer-
The-...](http://www.codeproject.com/Articles/3479/The-Bip-Buffer-The-Circular-
Buffer-with-a-Twist)

[1]:
[https://github.com/wrl/wwrl/blob/master/src/bip_buffer.c](https://github.com/wrl/wwrl/blob/master/src/bip_buffer.c)

~~~
loeg
This has additional restrictions:

    
    
      * (Maximum) item size must be known in advance, and
      * Buffer items must be smaller than half the buffer size, and
      * If items are not suitably sized, buffer fragmentation may result
    

There are also some cases where the padded size requirement may exceed
available non-wrapped space with bip vs VRB or similar. Anyway, most OS
support mmap(2) today :).

~~~
wrl
You're right on the second point and the third, though I reckon the
fragmentation isn't appreciably worse than in a regular wrap-around circular
buffer. Same goes for the first point. I'm keen to see examples where bip
buffers exhibit particular issues relative to circular/ring buffers, however,
and if you've got any then please send them along.

The chief use-case for my bip buffer implementation is using a shared memory
segment for inter-process communication, and adding mmap() for address
mirroring on top of that just seems like a can of worms I'm not particularly
interested in opening.

------
loeg
(2006)

The basic premise is to mmap() the same physical memory twice in a row (in
virtual address space). Then you can always point to a contiguous chunk of
memory to reference a given object in your queue.

------
clord
There should be (is there?) an OS designed around zero copy principles. I want
to get packets from the network delivered into a block of memory, and then all
subsequent parsing and reading occurs on that memory, with OS provided framing
and fine grained policy determining how things are interpreted by userspace.

Something like the memory trick used in this virtual ring buffer but far more
powerful.

~~~
colanderman
See Intel's DPDK and Tilera's mPIPE.

You can't really do this with TCP though, without changing the hardware. Most
modern network adapters support only a scatter/gather list mechanism for
receiving/sending packets, but this doesn't work for receiving TCP payloads
into contiguous locations, since TCP's data model is a byte stream.

With UDP and SCTP you can do full zero-copy just fine.

~~~
gpderetta
An option is to expose the the fragmented packets to the application and let
it handle it. For some applications, application levels packets are rarely
fragmented in the tcp stream so 0 copy handling of the incoming tcp stream is
possible and fall back once in a while to a compacting copy is an option.

A bigger problem is that on the send side you can't reuse the a 0 copy buffer
until the remote ack has arrived for that tcp segment which complicates the
interface between the application and the tcp stack.

------
Kristine1975
More detailed explanation plus more in the comments:
[https://fgiesen.wordpress.com/2012/07/21/the-magic-ring-
buff...](https://fgiesen.wordpress.com/2012/07/21/the-magic-ring-buffer/)

