

Microsoft`s Open Source Subsidiary Releases First Deliverable (Redis) - ks
http://www.eweek.com/c/a/Linux-and-Open-Source/Microsofts-Open-Source-Subsidiary-Releases-First-Deliverable-535785/

======
wickedchicken
> "Redis on Linux uses an OS feature called Fork/Copy On Write. This feature
> is not available on Windows,

The memory-mapped file APIs on Windows are seriously lacking. Last time I
dealt with them, there was no documented way to write to a constantly-growing
mmaped file (unless you knew its size already). All the workarounds I tried
ended in segfaults :/. Curious if anyone here has had success with that.

~~~
beagle3
How do you do that on Unix / Linux?

Don't you have to set the size when you map the file?

~~~
wickedchicken
You can change the size of the allocation on the fly, and just keep writing
bytes as you go. Pages aren't alloc'd on disk until they're touched. When you
hit the end of the file, you call a final resize to trim off the virtual bytes
at the end.

You can see an example of the technique here:
[http://www.gnu.org/software/libc/manual/html_node/File-
Size....](http://www.gnu.org/software/libc/manual/html_node/File-Size.html)
(scroll to the bottom).

~~~
beagle3
> You can change the size of the allocation on the fly, and just keep writing
> bytes as you go.

How do you do this technically? mremap(), I assume?

You _can_ do something very similar on windows, by VirtualAlloc()ing at the
end of the previous allocation, and if that fails, be committing,
VirtualFreeing and remapping at a different address.

Yes, I agree mremap() is much cleaner. I am mostly a Linux guy myself. What
I'm missing in Linux is "mremap2()", which would map an _existing block of
memory_ onto a file (whereas mmap maps an existing file onto a _block of
memory_ ). redis achieves similar functionality (and more) by fork()ing and
write()ing instead, but it would be nicer if it could be done with one call
and no fork()ing. a "copy this file from here to here" could then be done by
mmap()ing the source into memory, mremap2()ing the memory into the destination
file, and forgetting about it.

added: This would also allow btrfs/zfs to have O(1) file copy, even between
independent media if possible (i.e. source file already exists on target
media, even if in a different directory) - without any consideration from the
_cp_ implementation.

~~~
wickedchicken
> mremap(), I assume

Yes, with ftruncate(), see the link above. The problem is Windows doesn't have
an ftruncate() that works while the file is open and being written to. (Note:
my memory is a little fuzzy here, it might be that there was no implementation
that took a file descriptor instead of a filename, either way I couldn't pull
it off). As for your 'zero-copy dump memory into a file after-the-fact' idea,
I'm sure it's done somewhere. My brain is too frizzled right now to scrounge
up how though :).

~~~
beagle3
When I was still following the kernel mailing lists, there were discussions
about how to achieve this for sockets, and none about files, so that would
surprise me.

However, if there is a way to achieve that without forking, I would be very
happy to hear.

------
pragmatic
This is good news, I've been waiting for the MS supported version of Redis so
I can play with it on a Windows box.

I wonder why they hosted on Github vs Codeplex. Codeplex supports Mercurial,
Team Foundation Server or Subversion and Git.

Microsoft recently posted the source for _parts_ of ASP.NET (minus web forms)
on codeplex.

(just curious, not trying to troll.)

~~~
barranger
I'm pretty sure it's a case of Codeplex is going through a massive
overhaul[1]. The addition of git support seems to be the first step.

[1] here's a preview of the new look they are moving to:
[http://blogs.msdn.com/b/codeplex/archive/2012/03/30/new-
code...](http://blogs.msdn.com/b/codeplex/archive/2012/03/30/new-codeplex-ui-
coming-soon.aspx)

