This makes no sense. Ctrl-C is just a way to tell your terminal to send a SIGINT signal to the current process. How that process handles the signal is up to it! It's by definition ignorable, as the author points out, but it's not rocket science to handle it in a sane fashion even in a multi-threaded application. Modern languages make this trivial. The author makes it sound like some dark art but in reality you just have to read the manpages.
SIGINT is really designed for interactive applications. Most processes should simply treat it like a SIGTERM unless they have some sort of REPL. Unless you need graceful shutdown, most processes shouldn't mask either signal. If they do, the polite thing is to unmask after receiving the first signal so subsequent signals immediately terminate.
The SIGINT (if the terminal is configured to generate one) goes to the
foreground process group, not the current process. See the termios man
page, look for INTR. This gets complicated in shell pipelines (oh look,
a process group) where one or more of the tools involved are fiddling
with the global terminal state, in which case there may be a process
group signal, or there might instead be a key for some random program of
the pipeline to to read, which it may ignore.
With an important productivity app like rogue(6) there is (probably)
only one process in the foreground process group, and curses has
(probably) set the terminal to have control+c either ignored or
delivered to the rogue process as a key event. The player probably does
not want to have their rogue process vanish because they hit control+c
by habit, like trek(6) likes to do, but someone wrote blocksig(1) as an
exec wrapper so that SIGINT can be ignored. With a complicated shell pipeline, the
player probably does want the whole thing to go away, but that may be
difficult depending on exactly how complicated the shell pipeline is and
whether any processes in that pipeline are fiddling with the global
terminal state. (Global to the process groups and their PIDs under that
particular terminal, not the whole system or universe or whatever. But
global enough to cause problems.)
Opinions also vary here, some want that LISP image to never go away
(those emacs users, probably), others may want control+c to murder the LISP
image so they can go back to hacking in vi. POSIX probably says
something about shell pipelines and control+c and such, which various
shells may or may not follow, exactly. Etc...
There is an entire world here beyond registering for a signal that comment seems unaware of. Even the simplest of preliminaries: registering for a signal is arguably non-trivial and incorrectly specified in many places since sigaction() supersedes signal().
> it's not rocket science to handle it in a sane fashion even in a multi-threaded application. Modern languages make this trivial. The author makes it sound like some dark art
Which language? I'll specify one so we can begin the process of picking each apart. Python? There is a sibling thread indicating Python issues. I don't know what the actual internal status is with Python signal handling but I am guessing the interpreter actually doesn't handle it correctly if I spent any time digging. Do you mean apps implemented in Python? They will almost certainly not be internally data-consistent. Exposing a signal handling wrapper means very little particularly when they frequently do this by ignoring all of the bad implications. I just checked Python's docs, and not surprisingly, Python guarantees you'll be stuck in tight loops: https://docs.python.org/3/library/signal.html That's just one gotcha of many that they probably aren't treating. This dialogue is going to play out the same way regardless of which language you choose.
Do you mean Postgres? I haven't used it recently but the last comment I read on HN seemed to indicate you needed to kill it in order to stop ongoing queries in at least some situations. If by a stroke of luck it does support SIGINT recovery (which would be great), what about the hundreds of other db applications that have appeared recently? You can't just call the signal handler wrapper and declare victory.
I've done plenty of signal handling in Python and it's extremely straight forward. Like other languages, the runtime takes care of safely propagating information from the signal handler to other execution contexts, which requires being careful in a language like C (it's not hard, but you can't be naive). I wouldn't be surprised if there were bugs in Python, it's a mess generally and I'm not a fan.
Postgres queries run as subprocesses. You can send them any signals you want. Postgres tries very hard to be durable, and it handles signals carefully but often to the dismay of the operator who can't force it to stop without SIGKILL.
> registering for a signal is arguably non-trivial and incorrectly specified in many places since sigaction() supersedes signal().
This isn't a good argument, no one uses signal(2), I'm not aware of that ever being recommended in recent history and even the docs on my system scream "never use this" quite clearly.
Look, if you're not going to read the docs, signal handling will be the least of your concern. Signal behavior is extremely well documented.
> it's not rocket science to handle it in a sane fashion even in a multi-threaded application
It's not, though you need to be careful if you want to exit cleanly -- you can't just exit() or _exit(). You have to get all the threads to exit, and that requires a global volatile sig_atomic_t flag and some way to signal all threads that might be sleeping in event loops or what have you.
A separate thread with sigwait() may be easier. Though I must admit, I've rarely had to do that manually, as I'm usually using a language or framework that gives an easier way to get notified about signals (e.g. Python KeyboardInterrupt or listeners in Boost ASIO or libuv). Aside from saving some boilerplate, those also emulate equivalent signals in Windows.
Threads waiting on event loops is exactly what you want on shutdown: that's what you use to notify them to exit.
I never use sigwait() or sigsuspend() -- I just have signal handlers that write a byte (e.g., the signal number) into a pipe, and maybe they set a global volatile sig_atomic_t variable. DJB calls this a "self-pipe". A self-pipe turns signals into I/O events that select/poll/epoll/event ports/kqueue/io_ring/whatever can handle like any other events.
At exit time I just make sure every thread can get an event. The main thread can pthread_cond_wait() for all the other threads to exit, waiting for the count of them to fall to zero.
Agreed -- if signal handlers are too messy, `sem_post(3)`, `sigwait(2)`, or `signalfd(2)` will get that control flow where you want it. Then the problem is reduced to "my application needs to handle a graceful shutdown event", which, though possibly complex, isn't really that novel.
I feel this. Signal handling in Python code is especially complicated. I'm not even talking about multithreading here (not like you get anything out of it anyway).
Python registers POSIX signal handlers, that upon SIGTERM/SIGINT, set a flag. Next time the interpreter loop runs, the flag is checked before the next instruction is being run and the stack is unwinded.
When you call out to some C code, that C code may run for a long time. During that time, there is no interpreter loop actually looping. Therefore, all signals are ignored in principle while C code runs.
It's possible for Python code to become uninterruptible while it is calling something like pthread_join.
Definitely - it is a total chore to get a threaded Python program to handle Ctrl-C/SIGINT properly.
Single-threaded Python handles it well - as long as you don't register a custom signal handler, Ctrl-C raises a KeyboardInterrupt exception immediately. KeyboardInterrupt is not a subclass of Exception, (it inherits from SystemExit, which inherits from BaseException directly), so any "except Exception:" clauses don't catch it. Which is the intent. This is also a primary reason to never use bare "except:" clauses (it will prevent Ctrl-C from working!).
For multithreaded Python, the easiest thing to do is just mark all your threads as daemon=True, so they die if the main thread exits. When you can't do that, the best bet is some "threading.Event" and custom SIGINT handler that triggers the event. I kind of wish SIGNINT by default would raise the KeyboardInterrupt in all threads, but I'm sure there are good reasons not to.
> We don't want our ctrl-c to leak memory. […] If you allocate a piece of memory, you need to store a pointer to that memory from the object graph, and both of those operations need to occur inside of a critical section. Otherwise, if you get interrupted right after the allocation, there won't be any way to reach your memory and it will leak.
Maybe I'm missing something here but… so what? If at the end of your Ctrl+C signal handler you exit() as expected, then the OS will clean up your process's memory anyway.
That's the point: Ctrl-C shouldn't just gracefully kill the process, it should interrupt the current computation and let you resume your work without exiting the application. The use case here is interactive applications (think a REPL, for example), not commands you run, simply expect an output from and then they just exit (like, say, curl).
You don't mention REPLs etc. until the very end of the article:
> It definitely applies to interpreters, database-style terminal interfaces, REPLs, consoles, calculators, command-lines, and other categories I've unintentionally left out.
So if your article is supposed to be exclusively about those, I'd suggest you make this clear right in the beginning.
The article’s point is basically proven by how many people here don’t even understand he’s talking about this, and not killing the program with Ctrl-C.
Rather the opposite; people have hard time understanding what he is talking about because most applications that people use already handle ctrl-c as author wants, so its not a problem many people encounter often. So its reasonable that people then think its talking about the problem that many are encountering, programs that just swallow ctrl-c without doing anything. This is not helped by author having this bit near the beginning:
> More often than not I find myself having to kill the running process from an external app, such as the shell, after first figuring out what the process ID is.
IMHO anyone launching an app via a terminal and Ctrl-C killing it either is developing the app (in which case they can manage the signal however they like) or they don’t care and just want the app to die. Any “good” repl won’t let you exit via Ctrl-C so that point is moot.
> it should interrupt the current computation and let you resume your work without exiting the application
That's not what Ctrl+C is meant for or used for. It's used to terminate the running application, not the running task within that application.
If you want to be able to "resume your work" then you should press Ctrl+Z.
If you want something else then the application should probably be listening for some other keystroke. "Catch Ctrl+C and do something else" is a pretty awful idea for the very reason mentioned at the top of TFA (when you press Ctrl+C, it's to get out of whatever you're stuck in, so that you don't have to go open another terminal and type in killall ...)
> That's not what Ctrl+C is meant for or used for. It's used to terminate the running application, not the running task within that application.
I spend a lot of time running computations in REPL, and sometimes I realise that I made a mistake and I don't want to wait for the current operation to complete, or the mistake itself is such that the operation will complete only after I become old and die. In this case, I expect Ctrl+C to abort the current computation and return to the REPL, with the previous state (all the variable assignments) intact (modulo assignments made inside the loop I killed). I think a lot of people have the same expectation, and it's usually satisfied in modern REPLs.
> That's not what Ctrl+C is meant for or used for. It's used to terminate the running application, not the running task within that application.
If that's not how it should behave, how come any REPL I have handy handles Ctrl-C the exact same way? i.e. it doesn't exit the interpreter, it gets me back to the REPL. You can try yourself by getting stuck in a while loop and pressing Ctrl-C
Because they either fork their processes so the running task is it’s own process (which is how classic shells, like Bash, work) or they capture ^c and interpret it to behave like the classic shells do because that’s the behaviour people expect from shells.
You have to remember that Bash isn’t a language like Python in the sense that it’s core libraries are built into the Python runtime. in classic shells like bash literally every command is an executable. Granted they’ll ship some “builtins” but they’re still invoked via fork() to behave like external commands. So literally every ‘if’, ‘echo’ and ‘for’ (etc) has its own process ID in Linux/UNIX. Thus you can ‘kill’ an ‘echo’.
> or they capture ^c and interpret it to behave like the classic shells do because that’s the behaviour people expect from shells.
Which is kinda the point.
> So literally every ‘if’, ‘echo’ and ‘for’ (etc) has its own process ID in Linux/UNIX. Thus you can ‘kill’ an ‘echo’.
Do they? Because if I try like this `while :; do echo; done` and in another terminal I do `ps -x --forest` I can see the original bash running but it doesn't have any child process.
Besides, is it relevant to the discussion at hand?
I thought I’d get picked up on that part. My point was that shells are just a UI for invoking other applications (like a desktop shell but CLI). That’s the precedence and anything that’s shell-like but doesn’t follow POSIX is still inclined to emulate the same behaviour of killing applications because that’s the behaviour that people expect after decades of POSIX.
So it really is more about killing applications than killing tasks.
> Do they?
That was the original design (there’s even standalone executables for those commands included in coreutils for historic reasons). However Bash might have since optimised out a few forks.
The shell I’ve written certainly doesn’t fork() every built in either. However that doesn’t change how ^c’s behaviour was intended.
> Besides, is it relevant to the discussion at hand?
I’m talking about the behaviour for ^c and how it is handed in the shell, as a direct response to your comment about it. So yes. It’s exactly relevant to the discussion.
I'm having trouble judging what exactly the author wants here. My best reading is that he wants interactive programs to respond to SIGINT not by bailing out but by terminating the current task and returning to user input.
I'm having trouble, however, thinking of programs to which this applies. I just scrolled through my shell history, and the most common interactive program I've used in my history file is a debugger, which handles killing the active program correctly with no issues, followed by resource monitoring applications, shells, etc.
Can somebody tell me an example command-line application where there's a high degree of interactivity but is also multithreaded, has DB consistency guarantees, network requests in-flight, etc? I'm genuinely having trouble thinking of anything that's not a REPL or vim/emacs.
I'm not sure that the author is exclusively talking about command-line applications. The expected behaviors would make sense inside IDEs, particularly if they are blocked by a modal window.
it's %2. But it's also %+, %%, and just %, which is what the "+" after the [2] means. In my case %1 is evince zhegalkin-sm7433.pdf (Running, not Stopped), which I definitely do not want to kill.
Plenty of people open a new terminal window for every new program they want to run, but I commonly have several "jobs" in the same window, stopped or even running. Less often now that monitors are bigger, but still.
It is true this improves the bad path. It ignores desired happy path cases: downstream processes, custom debugging, graceful shutdown, preserved workspaces, and so on.
Excellent advice, thanks for sharing. Would in turn recommend using CopyQ to store this tips (and other like it) as a pinned items in folder with explanations for use two years later, that's how I personally stay on top of terminal kung-fu without overloading the consciousness-in-meat*
I wouldn't call this excellent advice - kill -9 will rob the process of the opportunity to clean up after itself and leave everything in a good state (e.g. any binary files being manipulated by the application). So I would use this as a last resort - start with Ctrl+C and then "kill INT %1" and then "kill TERM %1" before "kill KILL %1".
(For those who don't know "kill KILL" is equivalent to "kill -9". And despite the name "kill" is a tool for sending signals to processes.)
Anything can keep ports locked for a bit (if either side doesn't properly close the connection). That's how TCP works. Set reuseaddr on your daemon's sockets.
Laugh all you want, but this is is precisely why I like "old-fashioned" asynchronous exceptions (the ones which unwind the stack), and ensure most programs are ready to handle a clean stack unwind at practically any point inside the program (e.g. asynchronous-unwind-tables).
The way exceptions are handled as a result of siglongjmp'ing out of a signal handler is currently platform-inconsistent and one of the many dark areas I alluded to. It isn't even consistent on Linux between compilers.
Even though it makes sense from the name, SIGINT, to interrupt, I've rarely seen console software "return control" to the user when the signal is received.
What I've mostly seen in programs is a clean exit from the running application, if live user input is not intended to be used. Clearing a line or something similar like redrawing the terminal (that's mostly Ctrl-L though) is what interactive programs do, let's say shells or ncurses UI programs.
Whenever I made some hobby scripts that exit cleanly when receiving a SIGINT, I've made a global counter of interrupts. When SIGINT is received, the counter is incremented, which tells the main loop to stop as soon as possible. But if this counter exceeds three signals, the application would exit immediatley. This may not be ideal, but CTRL-C CTRL-C CTRL-C is easier than kill -9 `pgrep a.out`.
Like the top comment says, expecting a concrete and general behaviour on different types of software for such a broad signal doesn't gain wide approval.
What "return of control" did the author mean, on what kinds software?
A lot of multi threaded server software handles ctr+c just fine. A lot of Java based server software have a shutdown hook, which is something that you can easily add to any jvm based program because it is part of the standard library. If you use Spring Boot, for example, it has one and it will start shutting down your Spring Context and call any destroy functions on DestroyingBean implementations, which is how you can add your own shut down logic in Spring.
As far as I can tell, this appears to be confusing Ctrl+C (SIGINT, which terminates a process, and is usually not restartable), with Ctrl+Z (SIGTSTP, which pauses a process, and is thus restartable).
The only software I can think of that could "restart" after a Ctrl+C is usually daemons or other long-lived processes (which already need to be able to "restart" after any kind of shutdown and thus have significant amounts of code dedicated to serializing and unserializing their internal state).
TFA even goes so far as to talk about memory leaks - which are completely irrelevant when your process is about to exit anyway!
This article is quite a roller coaster to get what it is about. As far as I understand it, the author wants SIGINT to become some sort of a universal “cancel” button which may or may not exit a process, because the idea is to stop and rollback to a nearest sensible restart point. E.g. an interactive disk formatting tool may stop lenghty formatting on SIGINT but wouldn’t just exit. It would clean up the mess and return to its menu where e.g. batch configuration happens, so a user doesn’t lose next steps. The author basically wants modern gui features in console via signals.
Yeah, and as others have pointed out already, many existing interactive terminal programs handle SIGINT in this way. E.g. programming language repls interrupt running code and return to the top level prompt. E.g. mutt (SIGINT will cause mutt to politely asks if you want to exit before doing so).
I think of it this way: we have both SIGINT and SIGTERM for a reason. One "interrupts" and the other "terminates" and there are often good reasons to handle "interrupt" differently from "terminate" -- at least in interactive programs.
> this appears to be confusing Ctrl+C (SIGINT, which terminates a process, and is usually not restartable),
Respectfully, you seem to be confusing SIGINT, which is an interrupt signal and SIGTERM, which is a terminate signal. Many processes interpret SIGINT in a way which is indistinguishable from SIGTERM, but others do not (e.g. most REPLs).
I use Firebase emulator and man that likes to carry on running (or limping?) after Ctrl-C alot, hogging the port so you need to hunt it down and kill it before you can start it again. Both on Linux and Windows.
I think it is a Java (or more to the point JVM) program, not sure if that has anything to do with it. In addition I believe it is a lot of parallel programs running at once, or they could be different threads. As there are lots of Firebase services it needs to emulate.
Surprisingly, it is possible to do exactly what the author wants. I know because I've done it. However, it is as complicated as the author says it is.
The project in question is my `bc` [1].
Until version 3.0.0 [2], it used a "yield" architecture: every loop it could enter had a check for a signal. This got tedious, so I decided to make the jump to instant-ish reset.
I was lucky in several ways. First, `bc` is a really good program to reset; you just stop it executing, wipe all data away, and ask for more input with a blank slate. Second, it is single-threaded.
Nevertheless, it was still really difficult, especially to have no memory leaks.
First, I had to learn how to use `sigsetjmp()` and `siglongjmp()`. Yep, that was how I was going to do this. Once I learned, I implemented a stack of `sigjmp_buf`'s. Then, when a signal happens, each individual `sigjmp_buf` is used. This allowed me to properly free memory on the way.
In essence, if a function had allocated memory, then it would push a `sigjmp_buf` on the stack, and then when a `siglongjmp()` happened, execution would go to a label where that memory would be freed before continuing the jump series.
Then I implemented signal locks. It is safe to `siglongjmp()` out of signal handler, as long as it didn't interrupt code that was non-async-signal-safe. So I used signal locks for that, and when "unlocking" the lock, it would check for a signal and jump. And if the signal handler sees a lock, it just sets a flag and returns.
Then I had to go through my codebase and protect every bit of non-async-signal-safe code with locks. It was tedious, but the result is fantastic.
Edit: I forgot to add that there is more information at [3] and [4].
Nowadays, I'm working on a threaded build system, and when it gets SIGINT, it sends a message to threads to stop as soon as their children are done. If it receives a second, it just exits.
So yeah, every application is different, but it is possible.
Maybe I’m missing the point, but this (essentially random example) is multithreaded (asynchronous) and gracefully handles ctrl-c. Yes, it’s a high level language that makes it easy, I guess.
Tight loops in higher-level languages generally always have yield points/opportunities to break provided by the runtime, even if the loop itself doesn't appear to have any.
"More often than not I find myself having to kill the running process from an external app, such as the shell, after first figuring out what the process ID is."
Short cut here, ctrl-z to background the process, then kill -9 %1 to kill the first job (type jobs for the numbers)
Just wonder if author is aware of SIGSTOP/SIGCONT that allows to pause/resume any process gracefully ? Both signals can be caught and handled.
Crtl-C (SIGINT), as far as I know, was used to "gracefully terminate" interactive process from day zero of Unix. I cannot find any use in that of what author proposes: suspend execution by sending SIGINT, but then what ? Get to some process built-in debugging shell ? Isn't that what GDB was made for ?
SIGSTOP cannot be caught and handled; you're thinking of SIGTSTP.
Non-interactive programs do not need any special handling for SIGINT, and that seems to be what you're talking about. The author was talking about interactive programs like irb, bc, bash, gdb, and python, all of which behave as they desire, returning you to their REPL prompt upon receipt of SIGINT. One example of an interactive program that does not have the desired behavior is GNU Units.
SIGINT is really designed for interactive applications. Most processes should simply treat it like a SIGTERM unless they have some sort of REPL. Unless you need graceful shutdown, most processes shouldn't mask either signal. If they do, the polite thing is to unmask after receiving the first signal so subsequent signals immediately terminate.