
Investigating the Performance Overhead of C++ Exceptions - ingve
https://pspdfkit.com/blog/2020/performance-overhead-of-exceptions-in-cpp/
======
lallysingh
(quoting desc) _sigh_ (end quote)

Exceptions _make your code faster_ in the non-exceptional case.

See all those if-branches for error checks that don't exist in your code? That
branch-free code is going to run faster than your branchy code. And the code's
shorter, because it doesn't have the checks and handling intermixed, which
makes it fit better into cache.

This isn't rocket science, I'm amazed that this isn't obvious.

~~~
zozbot234
Branchy code where the branches are statically known to be _very_ unlikely
ought to be practically as fast as the branch-free equivalent. You're probably
forgoing a few optimization opportunities since you do have to build a proper
exception record (and perhaps a stack trace) if the code errors out but other
than that, it just doesn't matter that much.

~~~
mhh__
Static branch prediction isn't found on any major architectures, right?

You can optimize the code generation around that knowledge but not passing it
directly to the CPU

~~~
hrydgard
It is absolutely done, it's just implicit rather than explicit. The simplest
heuristic is to predict backwards branches as taken (loops) and forward
branches as not taken, and it works quite well (though I think it might be
more sophisticated than that).

~~~
ncmncm
Of course the error-code checking branches will most usually be forward
branches, thus not predicted correctly the first time through, or after
spilling.

------
flqn
I can't say I'm a fan of these sorts of articles. "Which strategy is most
performant?!" is rarely the appropriate measure for an error handling
strategy, it's more nuanced than that. Concerns like what makes a recoverable
error vs an unrecoverable one are more important. Ensuring correctness is more
important. The usability of an API is more important. Readability and
maintainability are more important. Knowing what errors mean in terms of your
domain model is more important.

Use what fits for your situation, don't take the "when all I have's a
hammer..." attitude to anything. Sometimes return codes make sense (eg in hot-
loop code). Sometimes exceptions make sense (in constructors, for example. And
no, two-step initialisation is not a good alternative.). Sometimes neither
make sense and you actually should just terminate the whole program (out-of-
memory). Sometimes contracts make more sense (precondition checks, validating
function arguments).

Anyway, regardless of your strategy if you can provide some method of querying
if a thing _can_ succeed to your API, do so; it's always better to handle
errors by just making sure you don't cause them!

~~~
rumanator
I would add that a error-handling strategy should never be a concern wrt
performance because obviously the purpose of this sort of code is to serve as
guard rails and ideally never be called at all ever.

If your error-handling code is triggered so often that you feel it might be
causing a performance hit, you have far more serious problems than the
performance overhead of how you handle errors.

~~~
sixplusone
My web server gets about 1000 to 1 hits of hacking attempts to valid requests
(malformed headers, etc.) so I hope nginx and fcgi handle errors efficiently,
because I pay for those cycles.

~~~
rumanator
You're not describing exceptional events. You're describing pretty much the
happy path of any reverse proxy. In fact, in some applications these sort of
events outnumber HTTP Status 200-worthy requests.

~~~
sixplusone
That's a strange idea of what "happy path" means. An invalid request is, by
definition, an error or exception.

~~~
rumanator
That assertion is obviously wrong. Think about it: you have a web server
listening to the world. Do you really expect all requests to be neatly
pointing to sanitized URLs? Of course not. Why on Earth should you assume that
handling invalid requests is not one of the primary use cases?

~~~
sixplusone
You and flqn are trying to convince me that expected errors are not errors
[https://en.wikipedia.org/wiki/Happy_path](https://en.wikipedia.org/wiki/Happy_path)

When I want a parameter is an integer, I don't check all the ways in which
it's NOT an integer; the happy path is when it _is_ an int and the program can
continue, and anything else is an _Error_. If it overflows, that's an error
too.

i.e., happy means it's basically the longest path (aside from diagnosing the
error).

The question is: should the subroutine set an error number or throw an
exception? Knowing the cost for each might help me decide for various
situations.

------
thedance
It should be well-known that actually throwing an exception in C++ is
extremely costly. I think it is also fairly widespread knowledge that even
allowing the compiler to support exceptions has runtime costs. Exceptions do
not conform to C++'s "zero-overhead" principle. That's why numerous large
organizations from NASA to the Dept. of Defense to Google decree that C++
exceptions must not be used.

~~~
cmrdporcupine
I believe the exception ban at Google these days is justified more on the
grounds that integrating with existing code would be problematic, rather than
on performance grounds. I seem to remember reading something to that effect.

~~~
billti
Their coding guidelines on the topic lay out the pros/cons and ultimate
reasoning pretty well:
[https://google.github.io/styleguide/cppguide.html#Exceptions](https://google.github.io/styleguide/cppguide.html#Exceptions)

------
desc
_sigh_ If your application is slow, odds are it's not because you used
exceptions.

If you're throwing enough exceptions for this to matter it'll show up on a
profiler, and then you can change that specific chunk of code to avoid
treating that particular case as 'exceptional'.

~~~
gameswithgo
Tim Sweeney, creator of Unreal Engine, Epic Megagames, etc, recently had a
code base where exceptions were causing problems even when not actively being
thrown:

[https://twitter.com/TimSweeneyEpic/status/122307740466037145...](https://twitter.com/TimSweeneyEpic/status/1223077404660371456?s=20)

Quoting him in a follow up tweet:

>They weren’t throwing and a disassembly showed no visible artifacts of
exceptions. But turning off the possibility of throwing exceptions gained 15%
and just made the assembly code tighter with no clear pattern to the
improvements.

~~~
abainbridge
How does he know that the performance improvement wasn't just a side effect of
altering the alignment of his code? Here's a guy getting a 48% performance
improvement just by randomizing the alignment of his functions:
[https://youtu.be/Ho3bCIJcMcc?t=351](https://youtu.be/Ho3bCIJcMcc?t=351) or
slides if you prefer
[https://github.com/dendibakh/dendibakh.github.io/blob/master...](https://github.com/dendibakh/dendibakh.github.io/blob/master/_posts/presentations/PerfAnalysisOnModernCPU.pdf)

~~~
gameswithgo
I am very familiar with this effect. We could of course use effects like this
to cast doubt on any benchmark someone has ever run, unless they specifically
mention that they tested for this, and 100 other bench marking gotchas. We
can, if we like, assert that all things are unknowable, while at the same time
asserting that the thing we want to be true is for sure true.

However, it appears to be a rather commonplace occurrence that having
exceptions on, even if you aren't using them, can cause performance problems,
it isn't just a one of in Tim's case. Also Tim has quite a bit of experience
working on bleeding edge C and C++ performance code so there is a good chance
he did account for this. You can ask him.

~~~
int_19h
How common is it, actually? And on what platforms/architectures? It was a
common problem back in the day when most code was compiled for x86, since
exceptions weren't designed to be zero-cost in that ABI.

For what it's worth, the article itself has this bit:

"Thanks to the zero-cost exception model used in most C++ implementations (see
section 5.4 of TR18015), the code in a try block runs without any overhead."

~~~
abainbridge
It has been at least a 10% effect in the last two things I profiled, which
were a simple software rasteriser and a distributed key value store. The other
significant benchmarking gotchas I see are: 1) the CPU being in a sleep mode
when the test starts and takes a while to get running at full speed, and 2)
other stuff running on the machine causing interference. But these two are
easy to work around compared to the alignment-sensitivity problem.

------
fefe23
Exceptions are not meant to be fast. In fact, exceptions are meant to be
exceptional. Hence the name.

Exceptions are meant to be fast if and only if you do not throw any. And even
this is only truly true on Unix.

The original implementation of exceptions on Windows put extra unrolling info
on the stack and then cleaned it up at the end of the function, adding a
runtime cost even when no exceptions were thrown.

The Unix way of exception handling does not put anything extra on the stack,
so exceptions are free if you don't throw any. The compiler puts some metadata
into your binary and generates cleanup code. If an exception is thrown, the
run-time (libstdc++ for gcc on Linux) will grab the return address from the
stack and then do a binary search through the metadata to find the right
unwinding code generated by the compiler. This is obviously a comparatively
enormous overhead. In particular because the metadata are separated from the
code, so the OS never has to read it from disk unless you actually throw an
exception.

The reason for that is that you are not supposed to use exceptions for
signaling. They are for error handling. Exceptions are for occasions like "you
asked for memory but there was none left" or "you asked for the 15th element
in this vector but it only has 10". They are not meant for non-exceptional
errors like "you asked for data from the network but there is no data yet" or
"you wanted me to check if the file exists and it does not".

The thinking behind all this is that the error handling path is not time
critical. If it happens, something exceptional has happened, and the OS will
probably have to swap in the error handling code from disk to begin with,
which is horrendously slow. In fact, this is not just a possibility. The
feature was designed specifically so that the linker would move the exception
handling code to a different page and the OS would only read that page from
disk if an exception actually occurs. This is not an accident. It is
deliberate planning and took engineering effort to achieve.

Running a benchmark of throw vs return is an attempt to answer the wrong
question.

Important side note: operator[] on a vector does not do range checking. at()
does. That's why you should always use at() instead of operator[].

~~~
quietbritishjim
You seem to be arguing that, because exception tables end up on separate
pages, we should consider exceptions as being as expensive as disk accesses.
(Apologies if I've just built a straw man, that's how I read it.) I don't buy
that. If you use exceptions more than totally exceptionally, according to your
definition, then those pages will be loaded to memory and kept there – while
the optimisation sounds very clever and sensible, I have a hard time imagining
that exception metadata is huge in comparison with the rest of the program
code.

Once that data is in memory, you're back to the numbers in the article, where
throwing an exception can take barely more than a microsecond. That's huge
compared to a function call, but is fast compared to a lot of things that an
overly cautious programmers wouldn't dare follow up with an exception. To
steal your own example, checking whether a file exists certainly falls into
this category – that's likely to take _milliseconds_.

(Even your other example – no data available to read (I'm imagining a select /
poll situation) – would probably be fine with exceptions, though I agree it's
not the right tool for that situation. Just making a call to the OS is likely
to cost a comparible amount to throwing an exception, even though you're
describing a non blocking call. And by definition this won't end to being
called in a critical inner loop - if data becomes available then the exception
will do being thrown.)

~~~
edoo
He makes the point that exceptions should be truly exceptional conditions for
your program that would prevent it from running, generally due to system
malfunction. Some people use it for basic business logic and it was never
meant for that. It is meant to help you gracefully abort things when
everything is about to go to hell.

~~~
quietbritishjim
> He makes the point that exceptions should be truly exceptional conditions

The comment did say that, but the only explanation of _why_ exceptions should
only be used that way seemed to be performance based. If that is not the
issue, then what is? Saying "that's what they're for" without any other detail
is a circular argument.

~~~
fefe23
I feel slightly misrepresented.

I explained how exceptions are implemented, and the actionable intelligence
you can derive from that understanding (if perf matters to your code and/or
scenario, then use exceptions only for hard errors, not for generic
signaling).

If perf does not matter in your scenario, than obviously the guidance does not
apply to you. To me that does not mean it's a waste of time to explain how
exceptions work and what that means to performance critical code.

------
ratboy666
Converting an exception into a return. Which loses the "have to deal with it;
cannot ignore it" property that made the exception valuable.

"The key takeaway here is that an exception cannot be ignored or forgotten; if
we have an exception, it must be dealt with."

If converted to simple "C style" status returns -- it is only 5 times slower
(11.5ns vs 57.5ns) -- and can be ignored and forgotten.

In the associated slide deck, boolean valid() must be used before value. In
this case, this is just as unsafe as C return codes. A C++ exception is always
considered an lvalue, meaning that union is initialized, but I am not sure
what reading the other side results in (I don't think that this is allowed).

My take on this? Please use an exception if you mean an exception (for C++).
If it is too slow (which I consider unlikely), please deal with it some other
way (I would suggest a status return).

~~~
patrec
> Converting an exception into a return. Which loses the "have to deal with
> it; cannot ignore it" property that made the exception valuable.

[[nodiscard]] -Werror

------
not2b
If exceptions aren't used, the error case has to be handled somehow. The usual
approach is to return two results, and callers have to check the error flag
somehow. If exceptions are very rarely thrown, then even with their very high
overhead the total cost could well be lower than the cost of returning a
separate error result and having callers check that result, perhaps by pattern
matching.

------
arunc
Exceptions are expensive and I hope no body uses it in the hot path (80%). The
compiler has to inject a lot of code to unwind the stack that affects the
cache. It is however not expensive for common operations (20%).

There are couple of threads in dforum that discusses this and to have throw
attribute [1] and eventually making nothrow as default [2].

[1]
[https://forum.dlang.org/thread/sbdrybtyfkxfhxxjgqco@forum.dl...](https://forum.dlang.org/thread/sbdrybtyfkxfhxxjgqco@forum.dlang.org)

[2]
[https://forum.dlang.org/post/qur0l0$2h8s$1@digitalmars.com](https://forum.dlang.org/post/qur0l0$2h8s$1@digitalmars.com)

~~~
josefx
Why do you link to forum entries from dlang on a c++ exception topic? Is the
implementation identical or at least somewhat similar?

~~~
mhh__
D is evolved from a C++ compiler, and can - in principle, it's a question of
priorities I believe, catch C++ exceptions

------
fwsgonzo
I made a website where you can measure these things in both time and
instructions. It's not perfect because it's a VM, but at least you can see for
yourself the horror show that is the first exception thrown. And also the
bloat that exceptions add. I still use exceptions simply because they are part
of the language, although sparingly. But I have a gut feeling that the
implementation of C++ exceptions in the current compilers, and the ABI, is
just subpar. It has to be, because nothing that isn't a context switch should
be measured in hundreds of thousands of instructions.

See: [https://droplet.fwsnet.net](https://droplet.fwsnet.net)

Just experiment with adding and removing exceptions, and compare it. It's
shocking.

Hello world using printf:

    
    
      Instruction count: 14988
      Binary size: 345 KB
    

Hello world try->throw->catch (and printf):

    
    
      Instruction count: 286926
      Binary size: 397 KB
    

Hello world using std::cout:

    
    
      Instruction count: 63502
      Binary size: 888 KB

~~~
ncmncm
You are just measuring initialization time. The global "C" locale used by
printf is statically initialized. With constexpr that could be done in a C++
library if anybody cared. Runtime spent initializing might matter for startup
in an embedded system, but they tend not to use cout much.

You could submit a PR constexpr-ing std::cout initialization.

------
dyingkneepad
I would love to see the results for increasingly values of randomRange. This
is just benchmarking the case when you throw exceptions 50% of the time.

------
Felk
If you want to see Herb Sutter's take on exceptions (and RTTI), I can really
recommend this talk where he explores his Result-esque (like in Rust) solution
that he wrote a paper about and proposes for standardisation:
[https://youtu.be/ARYP83yNAWk](https://youtu.be/ARYP83yNAWk)

------
m0zg
Performance overhead, I'm not that worried about - if you're throwing
exceptions routinely in the critical path, there's likely a lot else wrong
with your code. The crappy part about C++ exceptions is that they also
introduce a substantial binary size overhead.

------
ncmncm
Spot the Howler: " _it’s always best to catch an exception as close to the
throw point as possible_ ".

No. It's fastest to do that, but if you cared about fast, you wouldn't be
throwing at all.

Exceptions, by definition, are _off of the critical path_. If there's a
generalization to be made, it's that it's always best to catch them _as far
from the event as possible_ , because then you will have few catch clauses. If
you have more than a few, you are Doing It Wrong.

It shows a fundamental misunderstanding to worry about how fast throwing an
exception is. You legitimately worry about how fast it is not to throw them,
and that is generally quite fast indeed.

------
lanevorockz
It's popular because it gives to people what they want to hear. There has been
lots of research that gave different results and people just love to ignore.

------
benibela
The overhead also depends on how the compiler implements them

Some compilers use setjump/longjump (sjlj), and then it needs to save all
registers somewhere on every try{}. Probably also in every function that calls
a destructor. That is very slow, even if no exception is thrown at runtime

------
CodeMage
I was disappointed they didn't do a test case with std::error_code.

EDIT: Is there something wrong with std::error_code? I don't care about my
points, but I'm worried I might have missed something I should know.

~~~
Thorrez
I personally have a hard time understanding std::error_code. Specifically I
can't figure out when std::generic_category vs std::system_category should be
used, and I worry that confusion between the 2 of them will lead to confusing
and inconsistent APIs where it's hard to predict what error_code will be
returned in what situation.

~~~
vlovich123
std::system_category and std::generic_category are generally the same on non-
Windows platforms. std::system_category on Windows is any Windows error
(HRESULT, GetLastError, etc) and std::generic_category always represents
errno.

So for cross-platform APIs that exist on both you'll generally use
std::generic_category as the API is usually defined to modify errno in the
case of an error. For Windows-only APIs use std::system_category.

------
ec109685
Any idea of the cost of the final approach they used for the “success” case?
E.g. if the rand errored 1/100, would the cost of returning that union object
outweigh the exception?

------
Thorrez
Instead of using Expected you could also use std::optional or std::variant.
Expected might have a slightly clearer API for this use case though.

------
LessDmesg
1\. Exceptions are necessary for correctness. This alone solves the question
of whether they should be used. Error codes just don't cut it because they
only handle _expected_ errors (which is never enough in the real world).

2\. Arguing about C++ exception performance is pretty pointless anyway since
C++ performance is its (only) strong side. Until Java/C# matches C++ in e.g.
cutting edge 3D engines, this question won't even become interesting.

~~~
sika_grr
I disagree with both points, in fact with every statement you made.
Correctness can be achieved without use of exceptions (whether this is easier
or harder to do is up for debate). Next, in some cases correctness may be
sacrificed for performance reasons (e.g. solving a traveling salesman
problem). I find it easier to properly handle unexpected error codes than
unexpected exceptions. C++ has other strong sides beside performance. Java/C#
often throw exceptions in very non-exceptional cases (e.g. .Net's File.Delete
method).

~~~
LessDmesg
Nope, there's no practical way to cover all possible cases. There is always a
possibility of a totally exceptional situation arising (we're only human). And
then your beautiful ole error-cods returnin' function either takes down your
whole application (which might result in someone dying while on life support
in a hospital) or plods on with a successfull error code (leaving your program
in an incorrect state). All because you didn't have a catch-all-exceptions
clause.

This is why I find C++'s "nothrow" a joke, by the way. It's not within human
power to guarantee that any piece of code never throws exceptions. A gamma ray
might flip a bit. You never know. The only practical way to reliability is the
Erlang way: be prepared for errors arising anywhere anytime, not rely on
fallible notions of what can and cannot cause errors.

~~~
sika_grr
There is a difference between preventing a program crash and handling errors.
Please don't use catch-all-exceptions clause for either, especially not in
hospital machines.

The difference between writing if (status != SUCCESS) and catch (Exception) is
that the former will allow the program to actually crash when it should (see
panic in Rust). You should not be able to recover from OutOfMemory or
GammaRayFlippedABit errors (use ECC memory to prevent that). You should
restart the process (or the entire machine) because you can't hope the
developer was wise enough to destroy and recreate appropriate amount of state.

The additional problem with exceptions in Java and C# is that many times you
don't care about failure of some operations, but the language practically
forces you to write catch-all statement to prevent a program crash, so you end
up accidentally catching unrecoverable errors.

------
krapht
If you wanted this today, why would you implement the Expected template
instead of using Boost Outcome?

~~~
detaro
For the blogpost it makes more sense showing an example implementation IMHO.

And in some places, it's easier just writing it out than deal with the process
to add a dependency.

------
jimbob45
TL;DR: We rediscovered monads.

~~~
gpderetta
Well yes. And that's why there's a group that has in the committee that has
been trying to steer await towards a proper do-notation.

