
Serve2d – A protocol detecting server - wakaflockafliz
https://github.com/joushou/serve2d
======
rogerbinns
We did something similar in our Tarantella product many years ago. However we
quickly discovered that many companies operate network infrastructure that
verifies protocols. For example they would check that whatever happened on
port 443 was valid SSL and nothing else.

In the end we modified our clients to include a decoy cipher suite in the SSL
negotiation. That kept the network happy, and was enough for our multiplexer
to then internally route to the correct backend.

~~~
joushou
Yup, that's also why I added the ability to tunnel anything over a transport
(the only implemented one being TLS).

You can get the SSH client to connect over this either by using an openssl
s_client trick, or by just using my little tunnel tool
([https://github.com/joushou/tunnel](https://github.com/joushou/tunnel)).

------
nly
haproxy can do this out of the box. I've been using it for some time to put
SSH on port 443 as a backup (not an alternative).

    
    
        frontend mux
            mode		    tcp
            bind                EXTERNAL_IP_HERE:443
            acl                 is_tls req.ssl_ver gt 3
            acl                 is_ssh payload(0,8) -m str SSH-2.0-
            tcp-request         inspect-delay 5s
            tcp-request	    content accept if is_ssh
            tcp-request         content accept if is_tls
            use_backend         ssh if is_ssh
            use_backend	    https if is_tls
    

where 'ssh' and 'https' are tcp mode backends.

------
joushou
Uh, wow, the popularity just skyrocketed. It's 00:17 and I just came home from
a concert, so bear with me maybe being a bit incoherent.

First of all, I'd like to thank my wife, who helped motivate me, my parents,
who you know, did the thing that brought me here, my work, for paying me to
browse news and occasionally code...

So, why Go? Because it was easiest that way. Try to take a look at how simple
the ProxyConn thing is, which is the core of the entire thing. Go is a tool
that I found suitable for the task. It also brings things like easy cross-
compilation after the code has been written (Seriously, GOOS=windows go build
on my mac and I get a PE32+, GOOS=darwin go build on my linux desktop and I
get a Mach-O binary). You also get to interact easily with the wonderful Go
infrastructure, in case you don't want to redirect to external services.

Why make something that already exists? Well, I didn't know that there were so
many solutions, I just thought "Hmm, I wonder if I can make this work in a
nice fashion...". That's not going to stop me from continuing development of
my own, however. I have ideas I want to try, features I want to implement.
It's powering everything on my own servers, and the stealthiness of SSH over
TLS ( _NOT_ just SSH on port 443) have been (ab)used multiple times already.

Is my solution better than the others? Well, I of course like my own solution,
but with few exceptions, I think I bring some interesting flexibility to the
game. A lot fo solutions seem to be fixed for a certain purpose, such as SSH
and HTTPS, usually just in their regular variants. I don't really care _what_
you want to serve, as long as you can write down some unique bytes in the
config for serve2d, or write a more complicated handler directly for serve2.
Add transports if you want. Want SSH over TLS over WebSocket over IP over
avian carriers? Add the transports.

I'm going to sleep now, and I hope the world haven't self-ignited before I
wake up. It's really exciting and motivating to see how people react to
private projects.

------
BillinghamJ
Why? The entire point of ports is so you don't have to do this.

Don't get me wrong, it's kinda cool, but it seems like a really bad idea to
actually do it.

~~~
falcolas
There are a number of work and wifi proxies which won't allow out a lot of
traffic, based on port. A multiplexer like this can work around such (assumed)
well-intentioned but poorly implemented protections.

~~~
BillinghamJ
Wouldn't a VPN or tunnelling solution be better? You pass all your traffic
through a fixed port on one host, then unwrap it and use the web as though the
restriction wasn't there.

~~~
cbhl
Generally these well-intentioned networks block the usual VPN and tunneling
solutions (think kid and a school network with a firewall that only allows
FTP, HTTP, and HTTPS).

~~~
revelation
At which point you simply use these ports for your VPN.

OpenVPN can use TCP, it can use UDP, it can use whatever port you like, it can
even use a fixed key to turn all traffic into random noise (obfuscating
protocol structures).

~~~
falcolas
Thus the value of having a multiplexer which allows you to run these on 80 and
443, while still running a web server.

------
sschueller
Or use the good old sslh:
[http://www.rutschle.net/tech/sslh.shtml](http://www.rutschle.net/tech/sslh.shtml)

~~~
wakaflockafliz
That is certainly part of the inspiration and legacy behind serve2d.

This advantages of serve2 is that it is 100% go and can be integrated and used
without any extra dependencies or multi-setup :)

~~~
IgorPartola
So here's what hopefully won't be considered a trolling question: I have seen
a lot of "100% Go" projects over the past several years and that's usually
presented as a big feature. Some pretty trivial things have been redone as
brand new in Go, and then suddenly gain lots of attention. What is so magical
about a project written in Go vs C, Python, Ruby, Rust, JS, etc.? As a user of
the software I won't care what it's written in, if it's done well. If it's
done poorly, I am much more likely to look for better alternatives than to fix
it (if only I had about 240 hours in a day...), so what's the advantage?

To me, an advantage in usability is having a PPA with properly built .deb
packages. If I have to use a language-specific package manager that I don't
already use regularly, you've likely lost me, unless I really need this
functionality. If it doesn't come with a proper daemon mode (correct forking,
PID file support, proper file or syslog logging), sample config file, man
page, or an init file, that's even worse. I am much less likely to use this in
any type of "production" environment if I have to maintain those pieces
myself. Running things in a screen session is so "I'm running a Minecraft
server".

That is not to criticize your work. You've done a great job! serve2d looks
very interesting and I might actually have to give it a try sometime.

~~~
rogerbinns
Go produces a single static binary. Consequently deploying go apps is as
simple as copying a single file around, that just works.

~~~
poizan42
That's a very weird way to look at it. Static linking was there before
everything else. gcc/ld and, well, any other C/C++ toolchain can do that as
well. There is a reason this isn't usually done. It's like you are trying to
spin a bad thing into something good.

~~~
marssaxman
The reason this isn't usually done is that executable size was significant
relative to storage capacity up until the early 2000s or so, and people tried
to economize by deduping common parts of their executables via shared
libraries / DLLs. This worked well enough to catch on, but came with an
extremely high cost in added complexity, and over the years a whole layer of
additional infrastructure was created in order to manage it. The industry
progressed, storage capacity grew dramatically, and executable sizes stopped
mattering, but the use of shared libraries / DLLs continued out of inertia. As
time passed, people started asking - why are we doing all this? And some of
them invented a reason, which was the idea that one could swap out pieces of
existing executables after installation, and thereby fix security problems in
an application without needing to involve the application's developer in the
process. This works about as well as you'd expect if you had spent years
trying to fit all the rough edges of various third-party libraries together
with varying degrees of success, but the idea caught on as a popular post-hoc
justification for the huge layer of complexity we're all continuing to
maintain long after its original justification became obsolete.

As is no doubt obvious from my tone, I'm not buying it and am very happy to
see signs of a pendulum-swing back toward static linking and monolithic
executables.

~~~
scrollaway
There are other important downsides to static linking. Namely, critical
security updates to shared components. It's better to have to update your tls
library once per system, than to update every app that came with it. And
that's the best case scenario where the developer notices and the app is
actually repackaged.

~~~
marssaxman
Yes, that's the argument I was referring to, and as I said in the comment
you're replying to, I don't believe that the cost is worth the benefit.

------
ZenoArrow
serve2d looks promising, can see some good uses for it.

If anyone's interested, Corkscrew is an alternative solution, allows you to
disguise SSH as HTTP/HTTPS traffic, useful for getting through restrictive
firewalls to administer remote machines:

[https://github.com/elia/corkscrew](https://github.com/elia/corkscrew)

~~~
joushou
I guess the best way to find alternative solutions is to write one and have
people tell you there's other like it! I genuinely didn't know there were so
many doing equivalent things.

I do think serve2d is considerably more flexible than Corkscrew, though, being
able to tunnel _anything_ over TLS if needed. Or other transports, if more are
added. I also let you use the same ports for other things.

------
dutchbrit
Funny, my company is called Serve2 :)

~~~
joushou
Hopefully your "What's up with the name" section is better than mine!

------
aayala
Is possible to for MySQL ?

~~~
joushou
MySQL seems to use a server initiated protocol (which I always find to be a
terrible idea, as it means that an evil client has to do very little to
trigger a larger amount of traffic in return, potentially to a spoofed
address).

Postgres (which I would recommend over MySQL any day) seems to have a saner
client-initiated protocol. I only read part of the spec, but I'll try to see
what pattern would need matching later today.

