
Fastsocket – A highly scalable socket for Linux - nicolast
https://github.com/fastos/fastsocket
======
bscanlan
The performance data looks interesting, but this is work based on a pretty old
kernel (originally released in 2010 or so). There have been many changes and
improvements added to the 3.x kernel that may overlap with this work.
Publishing the code and details on github is great, but working with the
kernel community and merging into the mainstream kernel is the only way for
work like this to have a long-term meaningful existence - Google in particular
have been doing a great job getting networking improvements in.

That said, it's interesting to have this kind of thing come out of large-scale
production web environments in China.

~~~
sillysaurus3
Is it possible that by using an old kernel like this one, you'd expose
yourself to security vulnerabilities?

I'm new to kernel programming. Is this submission suggesting that you
downgrade your kernel to a 2010-era release in order to take advantage of the
performance improvements, or is the submission showing some kind of modular
component which you can integrate into your current kernel?

If it's the former, then wouldn't you be pinning yourself to the old version
of the kernel, so you'll have to integrate all updates by hand rather than
receive them automatically during the normal update process?

~~~
greglindahl
This kernel is what Red Hat Enterprise Linux 6 is currently using. Red Hat
maintains it, and writes patches for security vulnerabilities. It's no
surprise that Sina developed, tested, and deployed this patch against what
they were running in production.

~~~
sillysaurus3
By using this kernel, will you be able to automatically receive security
upgrades in the future? Or will you have to apply them manually and then
recompile and install the kernel yourself?

Is "developers have to apply security patches manually, then recompile and
reinstall the kernel themselves rather than automatically" not a big deal in
practice?

~~~
snus
All distro vendors backport critical fixes for the lifetime of the operating
system. RHEL6 is supported at least until 2020.

2.6.32 is also supported by kernel.org still.

------
edsiper2
Looks like its based on 2.6.32 series. I would hope they start working with
the upstream Kernel otherwise this project will stay stuck in Limbo as
previous initiatives to improve TCP handling at kernel level (e.g: Megapipe).

This version do not support TCP_FASTOPEN, SO_REUSEPORT, TCP_AUTOCORKING, etc.

~~~
peterwwillis
It's RedHat's 2.6.32 kernel, which is not the vanilla Linux 2.6.32 kernel.
It's been getting backported fixes since the tree began. RedHat does not
release what patches they include in their kernels, but luckily for us, Oracle
maintains a project called RedPatch which publicly documents the patches going
into the RHEL kernels.

As an example of how this kernel is not the 2.6 tree: On April 14, 2014, in
RedHat's 2.6.32-431.23.3.el6 kernel tree [1] , there was recently a patch
included [2] that affects the ipv4 subsystem. You can find that same patch [3]
was originally applied to the Linux kernel on April 14, 2014. This is common
practice, and so RedHat kernels more closely resemble modern kernels like 3.12
than anything else.

[1]
[https://oss.oracle.com/git/?p=redpatch.git;a=shortlog;h=rhel...](https://oss.oracle.com/git/?p=redpatch.git;a=shortlog;h=rhel-2.6.32-431.23.3.el6)
[2]
[https://oss.oracle.com/git/?p=redpatch.git;a=commitdiff;h=c0...](https://oss.oracle.com/git/?p=redpatch.git;a=commitdiff;h=c01663823d746852b7632199c57ad77a84907540)
[3]
[http://www.serverphorums.com/read.php?12,912195](http://www.serverphorums.com/read.php?12,912195)

~~~
desdiv

        RedHat does not release what patches they include in their kernels
    

Stupid question here: aren't kernel patches considered derivative works? If
so, then isn't RedHat legally obligated to released them under GPL?

~~~
logic
They provide the complete source, as required by the GPL. However, they do not
provide patch sets neatly broken out like they used to; that's what the parent
is referring to.

------
crazydoggers
Why would the evaluation charts look the way they do?

[https://github.com/fastos/fastsocket#online-
evaluation](https://github.com/fastos/fastsocket#online-evaluation)

The "before and after" CPU series have nearly the same exact fit. If the data
was from separate 24 hour periods, wouldn't you expect the graphs to look
different? I recognize that with a large service, you'd get repetitive load
patterns, but the similarity here look a little extreme.

~~~
teraflop
I find the first graph peculiar on its own. Supposedly, each line is the load
on one of 8 cores on the same machine. Why would some cores experience heavier
load than others, very consistently, over the course of a day? I've never seen
a workload exhibit that kind of long-term, core-level affinity on Linux.

~~~
yxhuvud
Well, the obvious reason for such a graph is that the network load balancing
between several waiting worker processes isn't symmetrical.

~~~
teraflop
Even if that was the case, there isn't normally a stable mapping between
processes and physical cores. There would have to be something within the
kernel itself that gives higher priority to some cores than others.

Not saying that's impossible, but I've worked on machines with more than 8
cores and never seen it happen.

------
Sir_Cmpwn
Is this being considered for merging upstream? What's the tech behind it, what
makes it faster?

~~~
corbet
The developers have not posted it to the relevant mailing lists or asked that
it be merged, so, no, it is not being considered.

~~~
harry8
Maybe it's up to the more sensible in the kernel "community", to reach out to
the developers of code known to be interesting to discuss what's in it for
them to do the work required to get it merged, the probability of doing a ton
of work, and then being ignored etc etc.

There's a sense in the above of "They haven't submitted us so we don't care."
It might not be the best way to make the kernel as good as it can be, if that
is the goal of anyone active in the kernel "community." (And maybe it is).

I have a lot of sympathy for someone publishing their code and their results
and then saying "I won't play stupid kernel politics, your move." I don't know
if that's what is happening here or it's cultural differences or something I
haven't thought of. Nor do I know if this particular development is worthwhile
merging, but hey, neither does the kernel "community" right?

------
sandGorgon
does this occupy the same functional space as zeromq or nanomsg ? are there
any comparisons?

~~~
Twirrim
This is much lower level than those. This is all about the TCP stack.

This is the OSI model, from the top down:

7) Application 6) Presentation 5) Session 4) Transport 3) Network 2) Data link
1) Physical

ZeroMQ fits in neatly at the top, layer 7 (arguably it is the presentation
layer too because it uses its own protocol).

What this is talking about relates to network sockets which is around layers 4
and 5 (you can find lots of debate around the subject). Any speed improvements
at lower levels in the stack would be seen by stuff on layers above it.

------
theyoungestgun
Better yet - avoid the kernel altogether!

Onload + Solarflare is a wonderful thing.

------
haosdent
Great job!

