
Self_update: In-place updates for Rust executables - adamnemecek
https://github.com/jaemk/self_update
======
jaemk
Never thought I'd see someone link to one of my projects!

As others have pointed out, it's not a perfect solution and has a limited
scope of use. I originally made this as an experiment to make it easy to
distribute updates for cli apps that I was working on. Yes, a package manager
is the "right" way to do this, but since I was testing and building images in
my CI flow (using
[https://github.com/japaric/trust](https://github.com/japaric/trust)) I wanted
a way to take advantage of that pipeline in a seamless way. The library
exposes a couple predefined "backends" that handle common patterns like GitHub
releases and s3, but the underlying steps (downloading, extracting, replacing)
are also made available so you can build your own update pattern and add
things like verification.

------
dathinab
For a moment I thought it would do process takeover (hot reload) which is
closer to impossible to get right.

Nice, through not production ready yet.

Through I'm a bit hard pressed to find any good use-case for it. In nosy
installations the binary _should_ not have the right to override any binaries
at all, including itself. In Linux (and I think windows too by know) there is
a form of update functionality provided by the system. Even if you do your own
update for client side software it should be a separate binary (as the main
binary should not be able to so so). Maybe it would be ok in a container, but
the restarting the container would likely instead the update.

(As a side note I said should because I know that it's sometimes not the case,
at least not without putting additional work into it. Still using a update
mechanism like that would make it harder to make it the case in the future).

------
dijit
I thought this was going to be replacing the process, but it appears to be a
tool for replacing the binary on disk? Am I reading the code right?

Why would you want to do this without restarting the process? Isn't hot-reload
a much more desirable thing?

and, to that end, replacing a binary in place just ensures that what's running
cannot be re-run on stop, so you could end up in a situation where you've
overwritten a working binary with a nonworking one which could obfuscate
critical information when trying to recover service.

~~~
zamalek
Hot reload is such a difficult goal, especially if there are kernel/out-of-
process resources (such as network connections) involved.

I'm sure everyone believes that hot reload is the answer, there are probably
fewer than 5 projects where it has been done, and all those projects likely
have some seriously PHD-level code.

Beyond the most extreme manual and cognitive effort, hot reload is completely
and utterly unsolved.

~~~
cperciva
Network connections aren't the hardest problem; you can hand those across Unix
sockets.

The fundamental problem with hot reload is that data structures can change. If
the old process has a priority queue implemented as a linked list and the new
process has a priority queue implemented as a heap, how on earth is any
automated mechanism going to copy the state across?

~~~
yorwba
Changing the running code but keeping the data in memory is equivalent to
changing the code but keeping the data in the database. Changing data-
structure definitions is equivalent to changing the database schema.

If you want to change the database schema without losing data, you need to
write a migration. If you want to hot-reload code with different data-
structure definitions, you'll have to write a migration.

A hot-reloading solution for a statically typed programming language will at
minimum have to detect whether data structures have changed and either prevent
the reload or require that the new code includes a function to migrate the
data.

------
blattimwind
There doesn't seem to be support for signing in this.

------
sansnomme
Does this do live process replacement too? E.g. transition smoothly from old
web server to new web server while still handling requests without downtime.

~~~
microcolonel
I'm pretty sure it doesn't.

That would be a quite tricky thing to do. You would probably need some very
specific architecture in your application to allow for something like that.

It's not clear to me that this is a better architecture than just splitting
your application into two or more processes, one of which is of such lasting
value and high quality that it would never need replacing. Then still, you may
be better off making your gateways/endpoints not need 100% uptime in order to
maintain availability, or making your client applications not require total
availability to behave well.

~~~
sansnomme
Technically, like Erlang, Go could probably do this by redirecting new I/O
into the new process's channels and kill the old process when all channels are
empty.

~~~
euank
Go's channels aren't exposed outside a specific go process's runtime. The
runtime doesn't give you any convenient way to redirect them. They're not like
erlang's mailboxes at all in that regard.

Furthermore, channels aren't the primitive used for multiplexing IO / handling
connections on a socket in go. You typically have a goroutine (e.g.
'http.ListenAndServe' spins up goroutines), and the gorutines are managed not
by channels, but by the internal go runtime's scheduler and IO implementation
(which internally uses epoll).

Because of all those things, replacing a running go process that's listening
on sockets is no different from that same problem in C. You end up using
SO_REUSEPORT and then passing the file-descriptors to the new process and
converting them back into listeners. Channels don't end up factoring into it
meaningfully.

If you're interested in what this looks like, cloudflare wrote a library
called tableflip [0] which does this. I also forked that library [1] to handle
file-descriptor handoff in a more generic way, so I've ended up digging pretty
deeply into the details of how this works in go.

[0]:
[https://github.com/cloudflare/tableflip](https://github.com/cloudflare/tableflip)

[1]: [https://github.com/ngrok/tableroll](https://github.com/ngrok/tableroll)

------
cbolat
What about integrity and security of downloaded binary? You should implement
signature check as well to verify binary.

------
dickeytk
this won't work on windows will it?

~~~
thijsvandien
While Windows does not support replacing an open executable, it does support
moving/renaming it, so you can get it out of the way and clean it up later.

~~~
asveikau
Also: I solved this problem before by copying the binary to a different
location, then launching in the alternate location it to copy itself over top
the original. Then the newly launched one in the old location deleted the
temporary copy. [Man, this sounds clumsy to describe. But it worked!]

