
Killing processes that don't want to die - ingve
https://lwn.net/Articles/754980/
======
fake-name

      Step 1: Install a kernel driver.  
      Step 2: Do blocking IO in the kernel that will never finish.  
      Step 3: Your process is now literally impossible to kill in any way. 
    

This works in windows too (for similar drivers).

This happens fairly regularly with devices with crappy drivers. If something
gets wedged in kernel IO, there is literally nothing you can do to kill it,
bar rebooting. IIRC, the kernel won't do anything until the IO finishes.

When shitty <hardware driver tool> craps out this way, it's _extremely_
annoying. I have semi-fond memories of trying all sorts of interesting tools
that attach to a process and inject faults to kill a wedged TV-Tuner
application when I was in high school. I never found a option that works.

Literally zero of the examples here for preventing unkillable processes would
work, too. If you can load kernel drivers, you're hoseable.

~~~
ahartmetz
This happened to me once in ~2006 on Linux with a scratched CD. I hope it's
been fixed.

~~~
filomeno
I think you're out of luck. I once accidentally scratched a CD and ten years
later it only got worse.

~~~
jfroma
You made me laugh a lot. Thank you

------
jacobparker
Fun article!

A modern way to do this on Linux (specifically) is to use PID-namespaces for
isolation.

You can pass CLONE_NEWPID to the flags argument of clone(2) so that the new
process is PID 1 in a new PID-namespace. One of the special properties of PID
1 is that when it dies all of other processes in the PID-namespace will
receive SIGKILL.

So, create a new "shim" process in a new PID-namespace and have that process
fork&exec the rogue program (it's probably better to fork&exec than just exec
because of the special properties of PID 1 that the rogue programming you're
running may not be built to handle.)

If you want to kill the rogue program, just kill your shim.

You need to be root (have CAP_SYS_ADMIN) to create a new PID-namespace. If
you're not (or more likely don't want to be) you can first create a new user-
namespace with the CLONE_NEWUSER flag. You'll be root in that user namespace
and can create a new PID-namespace. You'll probably also want to launch the
rogue program as a non-root user though, so make sure to figure that out.

Docker does this sort of stuff.

man pages:

* [http://man7.org/linux/man-pages/man7/pid_namespaces.7.html](http://man7.org/linux/man-pages/man7/pid_namespaces.7.html)

* [http://man7.org/linux/man-pages/man7/user_namespaces.7.html](http://man7.org/linux/man-pages/man7/user_namespaces.7.html)

* [http://man7.org/linux/man-pages/man7/namespaces.7.html](http://man7.org/linux/man-pages/man7/namespaces.7.html)

* [http://man7.org/linux/man-pages/man2/clone.2.html](http://man7.org/linux/man-pages/man2/clone.2.html)

~~~
cyphar
In a shell you can do this with `unshare -prf` which will create an
unprivileged user namespace (mapping only yourself -- if you want more
complicated stuff you'll need to use rootlesskit[1] or a shell script). Note
that most programs really don't like being in an unprivileged user namespace
(something the rootless-containers project has been working on fixing so you
can have completely unprivileged containers in every sense of the word "just
work").

[1]: [https://github.com/rootless-
containers/rootlesskit](https://github.com/rootless-containers/rootlesskit)

~~~
Liskni_si
You might also want to have a look at bwrap [1], a handy little sandboxing
tool for unprivileged users.

[1]:
[https://github.com/projectatomic/bubblewrap](https://github.com/projectatomic/bubblewrap)

------
jakeogh
I increasingly find myself looking at pledge...

[https://man.openbsd.org/pledge.2](https://man.openbsd.org/pledge.2)

It's objectively the right model, if my assumptions are wrong, the program
fails. I like that.

Maybe it could be taken further... to lie to the app and report back about
it's sandbox... or halt and log it's image.

~~~
pcwalton
pledge is objectively the _wrong_ model, because it's an ad-hoc set of fixed-
function rules. By contrast, Linux seccomp-bpf allows much more flexibility,
because you can actually write an arbitrary program to implement whatever
sandboxing policy you want.

In fact, your wish list highlights the problems of pledge precisely. You _can_
implement "pretend to succeed" and "dump core for inspection by a trusted
supervisor", but only on Linux with seccomp-bpf, not with pledge.

~~~
ori_b
It's objectively the right model, because it's simple enough to implement that
the vast majority of OpenBSD is pledged, including many ports like Firefox.
The vast majority of other systems do not have the equivalent, in spite of
having mechanisms that are in theory more powerful than pledge.

Flexible sandboxes don't help if your programs don't run in them.

~~~
kbenson
Maybe it's objectively the right model for right now, and the wrong model for
tomorrow, which would make you both right. It's hard to get programmers and
packagers to add new OS specific features, at least until most OS's support
from variant of that feature, and then they either bite the bullet or they use
from library to abstract it.

Pledge is simple, and that helps people adopt it and start using it. When most
operating systems are providing something equivalent and most programs are
making some use of it, it then becomes a selling point to do it better,
because the status quo has moved. At that point, pledge might not compare
well, unless it has evolved.

~~~
ori_b
> Maybe it's objectively the right model for right now, and the wrong model
> for tomorrow.

I'm using my computer right now. And pledge is evolving over time, as we gain
experience in what real programs need by implementing protection for them.

~~~
kbenson
Sure, but some things are hard to evolve after a while, and it may be that a
clean (or older but more complex) implementation has advantages at some point.
I have no idea if pledge will fall under this or not, but it's a common theme
in computing.

------
anon49124
Most old-school shutdown init scripts called the following at some point
before shutting down or rebooting:

    
    
       kill -TERM -1 # sending all processes the TERM signal...
       # ...
       sleep $SOME_SECS
       # ...
       kill -9 -1 # sending all processes the KILL signal...
       # ...
       sleep $SOME_MORE_SECS
    

Also old-school local DoS from root:

    
    
       # used to be very bad
       rm -rf / ; kill -9 -1
    

Another classic:

    
    
       # bash fork bomb
       :(){ :|:& };:
       # creates a function : and calls it
       # spawns a background process that pipes to itself, which creates another process... effectively two recursive calls per call.

~~~
adrianratnapala
Why are the pipes useful for the fork bomb? Or is it just a visual thing?

~~~
mongol
I think it is important to make it explode exponentially. Without it, the bomb
will be linear.

~~~
jwilk
FWIW, pipes aren't necessary to achieve explosion. Here's an example fork bomb
without pipes:

    
    
        :(){ :&:& }; :

------
VLM
During "its bad, but not that bad" situations over the last couple decades
I've had positive results with dropping the init level to single user mode "1"
and then going back to normal multiuser mode which historically could be any
number "2", "3", or "5".

Obviously this doensn't help with processes stuck in a kernel call. But for
other sorts of malfunctions, or simply testing init scripts and their
dependency tree, there's really no reason to reinit the kernel and check the
filesystem and all that, and it can be very fast compared to a complete boot
process.

This is not exactly a "gui-generation" solution for problems, but it has
occasionally helped very quickly clean things up.

------
emilfihlman
It's absolutely infuriating that you can, with a simple NFS, cause an
unkillable, unremovable, always there process to exist. There is simply no
reason to require this.

~~~
giovannibajo1
For non vital filesystems, I use "-o intr,soft,timeo=5" at NFS mount time.
This causes syscalls to return EINTR and timeout without hard locking. It can
sometimes confuse userspace, but at least you don't get a stuck process.

~~~
pjc50
I always preferred "intr,hard", because having filesystem operations fail
really stresses out a lot of userspace applications. Most developers haven't
thoroughly tested their handling of "what if this disk operation fails?",
because it's a pain to do and a rare case.

------
hayden592
Articles like this give me a better understanding of computers and how they
fundamentally work. Thank you.

------
johannkokos
I cannot get the first simple example working.

    
    
      int main()
      {
          while (1) {
              if (fork())
                  _exit(0);
          }   
          return 0;
      }
    

If I run `htop` _immediately_ after typing `./a.out`, it shows a error,

    
    
      Could not create child process - exiting
      fork: Resource temporarily unavailable
    

However, if I wait a second or more, I cannot find `a.out` in the process
list.

~~~
nytopop
You can apply a filter in htop before running your binary, it'll only show
matching processes.

I think the hotkey is F4.

------
taejo
Our product occasionally does this at the moment. We've set up asserts in our
application to launch a "phone-home" process (on the theory that launching a
new process is more likely to succeed than trying to phone home from the
borked process). However, the phone-home program uses the same assert library,
so the rare assertion failure in the phone-home will launch a new phone-home.

------
CamperBob2
How is that not a fork bomb? ("The author disagrees with this
characterization, which was added by an editor late in the publication
process.")

~~~
userbinator
I propose the term "forkworm", or perhaps "forxolotl"[1] for these processes
that continuously reproduce but don't grow exponentially.

[1]
[https://en.wikipedia.org/wiki/Axolotl#Regeneration](https://en.wikipedia.org/wiki/Axolotl#Regeneration)

~~~
caf
It already has a name - processes like this were historically called "comets".

------
RickJWagner
Hacker News paydirt!

I love an interesting programming article, this is a nice read. Kudos to the
author.

------
mjevans
A namespace freeze option for halting app process might be a helpful feature
in these cases.

However this still doesn't solve the Zombie process problem where something
has blocked on IO/another resource. Failure should absolutely always be an
option.

~~~
mnarayan01
[https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-s...](https://www.kernel.org/doc/Documentation/cgroup-v1/freezer-
subsystem.txt)

------
emmelaich
A comment in the article has it right -- use SIGSTOP not SIGKILL.

In a tight loop.

------
MaxBarraclough
Can't say I took much away from that article.

Next time my RabbitMQ broker goes nuts and refuses to die (thank you Erlang),
I'm still going to go ahead and reboot the whole machine.

------
known

       sudo dmesg -w -H --level=err 
    

displays the exact problem

------
the_ed
* 'Kill Dash Nine' by Monzy starts playing _

------
NullPrefix
Can it kill sync process?

------
daleon
Is there really no cross-platform way to do this that works o both *nix and
windows?

~~~
lozenge
There isn't even a cross platform way to start a process, never mind killing
one.

In Windows you can launch the process in a paused state then assign it a job
object which all sub processes will inherit. Then the job object has a
function to kill all processes in it.

