
Show HN: Hydrogen, a multithread non-blocking Linux server framework in Rust - nathansizemore
https://github.com/nathansizemore/hydrogen
======
yazaddaruvala
May I ask,

Why not Hyper?

Why not Mio?

Don't misunderstand, I'm all for competing implementations. I just feel like
it would be helpful to lay out a comparison for everyone else's
understanding/decision making process.

~~~
15155
Why not Boost? / Why not libuv?

Why not EventMachine? / Why not Celluloid?

Why not Twisted? / Why not gevent?

This is a really sad situation unfortunately. I was hoping Rust wouldn't head
down this road.

To preempt: a systems language shouldn't _dictate_ these high-level matters,
but there certainly should be one "blessed solution" vs. a sea of incompatible
libraries.

Go doesn't have this problem, Node.js doesn't have this problem, Elixir
doesn't have this problem. Sad deal.

~~~
pcwalton
> To preempt: a systems language shouldn't dictate these high-level matters

But you cited a whole bunch of languages that _do_ dictate this. Those
languages don't have those problems because you _can 't write_ competing
implementations of low-level async I/O in those languages.

And there certainly are competing, incompatible implementations of relatively
low-level functionality in Golang, for example fasthttp:
[https://github.com/valyala/fasthttp](https://github.com/valyala/fasthttp)

I'm all for standardizing on one solution when the time comes. Rust will have
an async I/O story. But right now nothing is mature enough yet to begin that
process. Rushing to standardize something that is not yet ready is the worst
of all possible options.

~~~
15155
> But you cited a whole bunch of languages that do dictate this

I don't believe the language needs to dictate something. It might help, sure,
but there's no reason that is a requirement.

For the sake of compatibility, a blessed option should be offered.

The AIO situation today in Rust looks _identical_ to Ruby, Python, and C++.

~~~
pcwalton
No, it doesn't. Basically all async I/O code in the Rust ecosystem uses mio.

And eventually a standard interface will be offered. Just not before it's
ready.

~~~
15155
Basically all async I/O code in the Ruby ecosystem uses EventMachine.

That's not the point: more of an ecosystem is built around the stdlib's
blocking I/O than mio. New projects everyday are built around things that will
eventually be incompatible.

------
Matthias247
Just some comments from looking very briefly at it:

The send API lets me wonder how it copes when the socket only allows to send
half a frame and then returns EAGAIN/WOULDBLOCK? There is not a number of
bytes that were really sent returned, and the ownership of the data to send
was given up, so it can't be retried later on.

On the receiving side it looks like the framework is allocating the memory and
then pushing the received bytes instead of the user allocating the memory and
fetching it (after readiness notification). That gives up some potential for
memory reuse.

~~~
nathansizemore
Thanks for the feedback! This means I need better docs :)

So, the framework itself does not implement any of that, it is on the
consumer. It just calls the proper methods on the trait object provided.

[https://nathansizemore.github.io/hydrogen/hydrogen/trait.Str...](https://nathansizemore.github.io/hydrogen/hydrogen/trait.Stream.html)

In the README example, I'm using it combined with another streaming crate I
made, which does provide all of the internal buffering and state management
for non-blocking I/O.

------
txutxu
In software context, for me, the name is taken.

    
    
        Hydrogen: advanced drum machine for GNU/Linux
    

And a very good one, with a long road in the back.

[http://www.hydrogen-music.org/hcms/](http://www.hydrogen-music.org/hcms/)

~~~
amelius
How about "proton"?

~~~
gkya
What about a proper, useful name that hints to the functionality?
"libhttepoll" (because built on epoll)?

~~~
kachnuv_ocasek
Sooo boring.

Just kidding, I'd also welcome if more projects came up with sensible names
that hint at what the program/library does.

It's happened way too many times to me that I saw a cool project with a
generic name, and wanted to use it at a later time to make my own cool stuff.
Of course, I'm then unable to look it up because the names get all mixed up,
and I only rediscover it a few months later when it gets posted on HN again.

------
onion2k
It's great to see people talking about how they're using Rust in production.
One of the biggest barriers to introducing new tech in to a business is a lack
of successful examples.

~~~
kibwen
We just began working on an official site to showcase companies using Rust in
production, here's an early look: [https://www.rust-
lang.org/friends.html](https://www.rust-lang.org/friends.html) (mouse over
logos to see links to code and testimonials, we're still working on the UX).

~~~
ekidd
As somebody who works with one of the companies on that list, I just want to
say that, yes, using Rust in production is a great experience. (At least once
I made friends with the borrow checker, which took some experimentation.)

We especially like the way that Rust code is simultaneously high-level,
strongly-typed and fast. The code feels "crisp", and if the compiler doesn't
return any errors, the program will almost always work correctly on the first
try. The code refactors easily. Plus the tooling (cargo, unit testing, etc.)
is well thought out. And we like the ability to build static binaries against
musl-libc and copy them into an Alpine Linux container.

I would happily use Rust again, at least if I wanted C++-level performance and
most of the right libraries were already available for a given task. (I can
find most basic, useful libraries on crates.io, but if I'm looking for
something like an HTTP proxy server using async I/O, I may not necessarily
find a mature implementation yet.)

~~~
nathansizemore
Yeah, learning a more safe way to program via the borrow checker can take some
time to get ingrained. Bad habbits are hard to break ;) I do know that async-
hyper is coming out soon (hyper built atop mio). Personally, I enjoy not
having support libraries around for everything - it gives me a chance to
implement myself and learn a ton along the way.

------
loonattic
Is it built such that a different event notification thing such as libev,
libevent, kqueue could be swapped in without too much effort? Well, it's not
even that much code.. looks nice

~~~
nathansizemore
Thanks, but it only uses epoll :)

mio is a crate that offers more platform agnostic async I/O.

------
cyphar
I like the fact that you chose MPL. Not enough people are using copyleft these
days. :D

------
Svenstaro
Alright, when would I want to use this? It reads like I would use it for a
high-performance and safe network daemon of some kind. So basically it could
be used as the lowest layer of, for instance, an IRC server?

~~~
nathansizemore
Exactly. It is ideal for any type of high performance server that needs to
handle many long lived concurrent connections.

------
bradhe
Happy to see projects like this pop up! As a long-time go user, these types of
fundamental projects really are what drove the community early on.

~~~
nathansizemore
Thanks! Was lucky enough to be able to use Rust for a production application,
and hope it can be useful to others in providing the tedious mundane parts of
socket server building :)

~~~
dengnan
Nice work! But you may want to remove src/#lib.rs# :)

~~~
nathansizemore
Thanks! Yeah, only a few days into using emacs. I keep forgetting about those
files.

~~~
kyllo
Just use this as your .gitignore file for every project
[https://github.com/github/gitignore/blob/master/Global/Emacs...](https://github.com/github/gitignore/blob/master/Global/Emacs.gitignore)

~~~
adrusi
Really such editor-related gitignore entries should go in the user's
core.excludesfile, not repository's .gitignore

Its unlikely that anyone would try to make a file that conflicts with an
editor-related gitignore entry, but it's nice to have the .gitignore file
cleanly delineate exactly what garbage the build system and application
create, without being cluttered up with entries for every editor that any
developer might ever use.

[http://stackoverflow.com/questions/7335420/global-git-
ignore](http://stackoverflow.com/questions/7335420/global-git-ignore)

------
dberg
So i assume this is similar to Ranch in Erlang ?

~~~
nathansizemore
From briefly looking across the README, it looks like Ranch is more for
maintaining a pool of connections, where this _also_ handles the marshalling
of consumer defined I/O (Non-blocking read/writes) operations, and the
workflow associated with non-blocking I/O.

------
idobai
> addr: "0.0.0.0".to_string(),

Reinventing c/cpp? ಠ_ಠ

~~~
kibwen
String literals in Rust produce references to data living in static memory.
The `String` type, which is dynamically-allocated and growable, is only
created when requested due to the cost of dynamic allocation.

~~~
idobai
So, do you need to convert the string to a string whenever you want to use a
mutable string?

~~~
zootm
If you have a string slice (&str) you need to convert it to a String you own
before you can mutate it, yes. In some other languages strings are always
immutable, and mutating them requires creating a copy. In Rust if you own the
memory (and haven't given out any immutable references to anyone), you can
mutate it.

String literals are an interesting case for the reason mentioned in the
earlier comment: they reference memory that no code "owns", as in-place
mutation of the executable would be unsafe.

Some languages have mutable strings but these are usually unsafe if used
concurrently, or require locks. In Rust this is modeled in the type/borrow
system.

~~~
idobai
> In some other languages strings are always immutable, and mutating them
> requires creating a copy.

Those languages used to introduce mutable wrappers and not a "to_string()"
method.

> they reference memory that no code "owns", as in-place mutation of the
> executable would be unsafe.

Is that similar to the dynamics of uniqueness types?

~~~
zootm
> Those languages used to introduce mutable wrappers and not a "to_string()"
> method.

Yes! This is similar to, say, StringBuilder in Java. The difference is that
ownership is intertwined with the type here. Strings are always owned, and as
such you can choose to mutate them. String slices (references) are not owned
by you (you borrow them), so you can't do anything that would be memory
unsafe. You can borrow a mutable slice of the String (like a reference to the
backing array), which you can alter in-place but you cannot do anything that
would require reallocation (Strings being growable was mentioned earlier --
mutable slices are not). Slices which come from string literals are always
immutable.

For probably-obvious reasons you can only take one mutable reference at once,
while you can have as many immutable references as you like. Taking a mutable
reference also prevents you taking any immutable references at the same time:
[https://doc.rust-lang.org/book/references-and-
borrowing.html...](https://doc.rust-lang.org/book/references-and-
borrowing.html#the-rules)

In my own experience this takes a bit of time to get used to, and in some
cases the rules are still quite frustrating. People with more experience of
the language report that they get used to it, though.

> Is that similar to the dynamics of uniqueness types?

I believe it's similar, but I'm not super-familiar with uniqueness types. This
SO answer looks good: [http://stackoverflow.com/questions/26309081/how-do-
rusts-own...](http://stackoverflow.com/questions/26309081/how-do-rusts-
ownership-semantics-relate-to-uniqueness-typing-as-found-in-clean)

------
yarrel
Loading drum samples into this will be hard.

