
No-reboot patching comes to Linux 4.0 - tanglesome
http://www.zdnet.com/article/no-reboot-patching-comes-to-linux-4-0/
======
matteotom
More info here:
[http://lwn.net/SubscriberLink/634649/d31644d65227ead6/](http://lwn.net/SubscriberLink/634649/d31644d65227ead6/)

TD;DR: There are still issues to work out, and working live patching might not
be in for 4.0.

> "So, while it is still possible that the kernel will have an essentially
> complete live-patching feature by the end of the year, it may happen rather
> closer to the end of the year than the developers involved might have hoped
> for."

------
justincormack
Live patching has its serious critics, and it is also not nearly finished "I
think they are fundamentally misguided in both implementation and in design,
which turns them into an (unwilling) extended arm of the security theater."
[1]

[1]
[http://lwn.net/SubscriberLink/634649/f6a2aa1f39c538b7/](http://lwn.net/SubscriberLink/634649/f6a2aa1f39c538b7/)

~~~
azakai
Thanks for the link! Reading that, it looks pretty pessimistic actually. There
seems to be nowhere near a clear path forward for how to actually do this.

One concern is that stack traces are "best-effort" and not guaranteed to work.
Isn't there a way to force them to work, by disabling some stack-related
optimizations (like no-omit-frame-pointer)? I imagine there would be a little
slowdown, but maybe worth it for the ability to do quick stacktrace-based live
patching?

------
yungchin
Bizarre to think that at the time nobody stopped Oracle from picking up
Ksplice (they offered rebootless security patches for all the larger distros
back in what, 2009?). I really loved their service because it meant I could
keep my half million or so terminals in gnu screen open for months.

(Also, their blog was great - see eg this really scary proof-of-concept of
what you can do when you know how to live-patch a kernel
[https://blogs.oracle.com/ksplice/entry/hosting_backdoors_in_...](https://blogs.oracle.com/ksplice/entry/hosting_backdoors_in_hardware)
)

~~~
mmagin
Dare I ask how having half a million terminals open in screen is useful in any
way?

~~~
yungchin
Ok, not really half a million. Maybe 20 or so :) That's still a lot of state
to lose for a kernel-upgrade reboot.

I guess it's a style of working. If you need to ask, that probably means
you're not the kind of person who cares about the advanced tab-grouping
features in firefox either, and you probably also never have so many different
paper files open on your desk that you can't find space for your laptop? I'm
not sure I can explain, and certainly not defend it - you might call it
"organised chaos" :)

In any case, it's all state, and it doesn't all fit in your head. And a reboot
wipes it out!

~~~
btbuildem
Do you know of any decent way to save the layout / history of each term
window? Or more generally, the state of X desktop, to be able to reload later
/ have several "workspaces" set up for different sets of tasks?

~~~
labianchin
There is also Tmux resurrect that you might take a look:
[https://github.com/tmux-plugins/tmux-resurrect](https://github.com/tmux-
plugins/tmux-resurrect)

~~~
guiambros
+1 for Tmux Resurrect. Pretty convenient.

------
PythonicAlpha
But the Kernel is not the only reason, a system has to be restarted. What
about changed dynamic libraries? It can be a mess to fiddle out, which
processes has to be restarted, so a reboot is normally the best solution for
critical patches (eg. in the glibc).

~~~
hobarrera
I agree. Especially stuff like init, or non obvious non-trivial system
services (and, of course, in the case of these distros, systemd).

~~~
digi_owl
Supposedly systemd has a "restart in place" option. But if it fails you are
staring at a reboot anyways.

~~~
heinrich5991
Hasn't failed for me so far.

~~~
NeutronBoy
Perhaps, but the fact that it _could_ fail means that you have to be prepared
for a server outage when you attempt it anyway. In which case... the utility
of 'rebootless patching' goes down a lot, because why not just reboot it?

~~~
eru
Because there's less downtime in the vast majority of cases?

------
ams6110
_One reason to love Linux on your servers or in your data-center is that you
so seldom needed to reboot it. True, critical patches require a reboot, but
you could go months without rebooting._

Doesn't seem so lately. I haven't actually plotted it but seems like kernel
security updates drop every few weeks these days.

~~~
riffraff
I wonder if it might be an issue with recent kernels (in the sense of not
being out for a while).

I recall running some form of "stable" red hat without rebooting for very long
time years ago, while my ubuntu LTS from last year has a "System restart
required" MOTD every other week.

------
greglindahl
Is there any solution for processes using obsolete libraries? It's a pain to
figure out how to restart all of them without rebooting.

lsof | grep lib | egrep '(DEL|deleted\\))'

~~~
eudemo
If you are on Debian, you could try checkrestart

~~~
greglindahl
Ah, I'm on CentOS (RHEL), which seems to lack an equivalent.

EDIT: Note that the key feature is assistance in restarting! Which the debian
gizmo does. Just detecting can be done by the lsof pipe I posted.

~~~
benjarrell
RHEL 7 has 'needs-restarting' FWIW

------
drzaiusapelord
What does this mean on the VPS front? Will EC2, Rackspace, and Linode be able
to patch Xen or KVM without rebooting all their clients now?

~~~
rodgerd
My understanding of all the kernel splicing technologies is that it will
depend heavily on the nature of the patch.

~~~
q2
I also feel the same.

If the patch consists of executables that are in use, then it has to give
prompts to admins on the usage and to patch it later.

Another concern may be, system may start patching and meanwhile, user starts
related applications and it may make whole system unstable and unusable.

I guess, before patching, users need to be stopped till patching is completed
successfully.

------
twic
This seems like a bit of a smell. The desire for no-reboot patching stems from
the fact that reboots are painful. But since hardware and software failure is
an inevitable fact of life, it seems like it would be better to make reboots
less painful, rather than try to make them less frequent.

This is the idea underlying crash-only software:

[http://lwn.net/Articles/191059/](http://lwn.net/Articles/191059/)

------
bigbugbag
Ironic while the kernel gains the ability to patch without rebooting as the
same time, systemd makes it mandatory to reboot after each update.

~~~
the_mitsuhiko
Why does systemd make a reboot necessary?

~~~
nilved
It doesn't, GP has forgotten about `systemctl daemon-reexec` and `kill 1`.

------
seccess
This seems like it will be really useful for servers---I wonder how much it
will matter for desktops though. Logging out and logging back in seems like a
requirement for xorg, groups, etc. While it isn't a reboot, it still kills my
windows and thus my workflow.

------
atonse
I've always wondered what something like CoreOS uses to "switch" kernels in a
live system - is that also something like ksplice or a different technology?

~~~
Titanous
I believe that CoreOS can use kexec to update the kernel. kexec basically
looks the same as a reboot except instead of stopping fully, and going through
the BIOS/bootloader again (which can take a long time on server-grade
hardware), the old kernel 'execs' the new kernel and boots userspace back up.

[https://en.wikipedia.org/wiki/Kexec](https://en.wikipedia.org/wiki/Kexec)

------
jonalmeida
I remember talking to a rep at Morgan Stanley that was explaining to me that
you can perform patches on kdb as well so that you have zero downtime.
Although, I believe that was implemented a bit differently where you have a
compiled program and the patch that is read by an interpreter. I didn't get
down to learning how it would eventually take the interpreted patches to be
part of the compiled binary.

------
xroche
For those interested, here's a very nice presentation on kGraft by Jiri Slaby
(Suse): [https://kernel-recipes.org/fr/2014/patcher-le-noyau-en-
temps...](https://kernel-recipes.org/fr/2014/patcher-le-noyau-en-temps-reel/)

Note: the page is in French, but both the presentation and the slides are in
English.

------
dantillberg
This article's title really ought to include a hyphen: "No reboot patching"
reads very differently from "No-reboot patching". The former suggests that
this feature ("reboot patching") did not make it into Linux 4.0, while the
latter accurately suggests that the feature called "no-reboot patching" _is_
(edit: or may be) included in Linux 4.0.

~~~
sctb
Thanks, we updated the title.

