
Python exceptions considered an anti-pattern - sobolevn
https://sobolevn.me/2019/02/python-exceptions-considered-an-antipattern
======
mindslight
It amazes me how enduring formulaic it is to single out some particular design
tradeoff of a language, draw up some examples of expressing something where
that tradeoff creates worse code, and then act like it's some mortal flaw in
the language.

Python chose untyped exceptions, period. How is this surprising, given that
its basis is untyped parameters?

If you don't like that, use Java with its checked exceptions. Or remove
exceptions from the implicit monad altogether and use C or Rust. Just don't
then go on to write some lengthy post about how any one of those is too
explicit.

~~~
philipov
Do you mean unchecked exceptions? Python exceptions are strictly typed, that's
core to how they work. An except clause will catch subclasses of the target,
so you need to have a custom type for each exception you want to raise in your
code.

The mistake I see people make is to use built-in exceptions without
subclassing, making it impossible to explicitly catch specific errors. Or the
opposite, catching Exception, which will catch everything indiscriminately.

~~~
mindslight
No. I mean checked exceptions. At the _language_ level, Java checked
exceptions are typed, and unchecked exceptions are untyped.

The Python runtime is _dynamically typed_. The Python language is _untyped_.

~~~
CodeMage
Both checked and unchecked exceptions are typed in Java. Unchecked exceptions
are not included in the method signature, but that doesn't mean that the
exceptions are untyped.

~~~
mindslight
The _exceptions_ , meaning the runtime objects themselves, are indeed typed.

But at the language level, unchecked exceptions are not part of method
signatures (as you said), therefore not type- checked/declared/inferred.
Therefore appropriately described as _untyped_ -

When calling a method, you have no (formal) list of unchecked exceptions that
might be raised.

If you think of exceptions as being an implicit union type around every
function return (ie monad), analogous to how you have to explicitly check for
errors in C/Rust, you'll see what I mean. Java's unchecked exceptions are akin
to calling a function in Python that you expect to return objects of only one
type, but not being "sure" that it can't return something else.

------
detaro
The interesting bit starts at [https://sobolevn.me/2019/02/python-exceptions-
considered-an-...](https://sobolevn.me/2019/02/python-exceptions-considered-
an-antipattern#how-to-be-safe), where it introduces a library for wrapping
errors in return types, even if it got little to do with the headline. IMHO
the article would be stronger with a title about what it presents and without
the first half.

~~~
sobolevn
Thanks, my idea was to state the problem and propose a solution. Hope, that
you will find `returns` helpful.

~~~
detaro
One risk with the title is that it doesn't convey that it's actually proposing
a solution instead of being one of many "$languagefeature is bad" articles. I
personally also found the problem statement presentation not all that
convincing and almost closed the tab before reaching the interesting section,
which is why I made the comment above.

------
chillacy
Counterargument: Exceptions are Pythonic
[https://jeffknupp.com/blog/2013/02/06/write-cleaner-
python-u...](https://jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-
exceptions/)

~~~
kstrauser
I'm soundly in that camp. For instance, we write API endpoints that look a lot
like this:

    
    
      def update_password_view(session_cookie):
          values = request_params(['old_password', 'new_password'])
          user = get_user(session_cookie)
          verify_password(user, values['old_password'])
          update_password(user, values['new_password'])
          return 200
      
      def request_params(param_names):
          values = {}
          for key in param in param_names:
              try:
                  values[key] = request.params[key]
              except KeyError:
                  raise BadRequestError('Missing parameter', key)
          return values
      
      def get_user(session_cookie):
          users = db.get_user_with_session(session_cookie)
          if len(users) != 1:
              raise NotFoundError('No user with that session')
      
          return users[0]
      
      def verify_password(user, old_password):
          if hash(user.old_password) != hash(old_password):
              raise BadRequestError('Bad password')
      
      def update_password(user, new_password):
          user.password = hash(new_password)
          db.update_user(user)
    

Notice that each function raises an HTTP-ready exception, so
update_password_view has no explicit error handling of its own. You can look
at that function and read the intent of how it actually works, as each line is
only reachable if the one before it 100% succeeded. After `user =
get_user(...)`, you know that `user` will have valid data and not some
sentinel value you have to check for.

Our actual implementations are more subtle. We have our own exception
hierarchy with classes like `UserNotFoundError` or `BadPasswordError` that
subclass the corresponding HTTP error classes, so you can still write code
like:

    
    
      def upsert_user(data):
          try:
              user = get_user_by_email(data.email_address)
          except UserNotFoundError:
              user = User(data)
              db.save(user)
          do_something_with(user)
    

in the cases where that exception isn't fatal.

In practice, we've found this coding style to be much easier maintain than
idioms like `if not_found(user): return None` where you spend half your lines
of code explicitly checking return values for error sentinels. Life's too
short to live like that.

------
kissgyorgy
I never understood, that people who don't understand Python at all, or don't
like essential Python constructs, why they bother using it? Why not use a
different language in the first place? If you are using this library, you are
not writing Python anymore and you lost the biggest advantage of the language:
simplicity.

~~~
rpedela
Often the language is chosen because it is the only option like JS for web, or
it is the best option for the project. For example, you are doing physics and
you need a general purpose language that is easy to write and handles gigantic
numbers correctly so you pick Python.

~~~
sametmax
>Often the language is chosen because it is the only option like JS for web,
or it is the best option for the project. For example, you are doing physics
and you need a general purpose language that is easy to write and handles
gigantic numbers correctly so you pick Python.

You could pick Julia, or R. A community could create their own tools. It's not
like JS, which has an absolute monopoly and people were forced to use it.

Python __became__ popular for that purpose.

Not Java. Not Haskell.

So don't try to implement Java or Haskell error handling in Python if you are
using Python in a niche where it shines: it shines here because people working
in that field decided it fits well.

~~~
AstralStorm
JS interpreter, thanks to asm.js, can be used to run any language. Knock
yourself out.

Of course you get to deal with a bit of FFI when calling built-in
functionality of the browser.

~~~
sametmax
Thanks to adapters, you can plug an american fridge in a french socket. I'll
still buy the one made for the local electrical system.

~~~
AstralStorm
So, you'd write in an inferior language when even web framework authors don't
want to?

(See CoffeeScript, TypeScript, Elm, etc. Or Transcrypt if you want pythonic
semantics.)

I understand this being a business requirement made by a business droid. Other
than that, there is really no reason to be married to JavaScript.

------
ltbarcly3
"X considered an anti-pattern" article titles considered an anti-pattern

~~~
posix_compliant
I'm bothered by how frequently people paraphrase this quote Dijkstra to suit
their own purposes. Borrowing a famous quote makes me think that they need the
help sounding credible.

------
1337shadow
Title should be "another person on internet considers it an antipattern".

I'm part of the people who advocate automated testing and insanely high
coverage. Does that mean I want to write all tests manually ? No, I use
fuzzers, light test code against many autogenerated fixtures and the like, and
get no runtime exceptions.

Duck typing means "bad language" for some people, for me it means "freedom".

------
eemax
The library described in this blog post looks kind of interesting as an
implementation of a Maybe monad in python, but the example case is pretty
silly.

It re-implements a 4 line function as a 13 line class, but the logic at the
caller doesn't get any simpler:

    
    
      try:
         result = get_user_profile(id)
      except:
         # handle any exceptions...
    

vs. with the library:

    
    
      result = FetchUserProfile(id)
      if (result is a Failure):
        # handle the failure
      # do something with the result

------
Glench
I'm glad this issue is getting some attention. I found the "Exceptions are not
exceptional" section to capture something I've noticed several times — that
code in Python can fail at a huge number of points even in a small function,
nevermind the many exceptions that might happen in nested functions, or code
we're using from libraries.

We should have language-level mechanisms for being explicit about what's
supposed to happen when unexpected situations happen.

~~~
gnahckire
> that code in Python can fail at a huge number of points even in a small
> function

This is why unit-tests are really important in Python.

> We should have language-level mechanisms for being explicit about what's
> supposed to happen when unexpected situations happen.

That's implemented as a try-except block. You're not supposed to catch errors
at the low level unless you're explicitly handling them. If there's an error,
bubble it up to the main().

Python's try-except can catch classes, sub-classes, and/or groups (tuples) of
classes. It's important to categorize your errors.

------
skrebbel
Any sufficiently typed programming language will grow a vocal subcommunity
that will try writing Haskell in it.

------
epage
Since using Rust, I've longed for this in Python but

\- imo it effectively requires using type checking (mypy)

\- Too out of place with the rest of the python ecosystem (even if I like
something, I'd rather not force non-standard practices on others dealing with
my code)

~~~
StavrosK
I don't know, for things where you can get team buy-in, this looks like a
great way to have more solid contracts. I've always disliked how you can
specify accept/return types in Python but then there's this whole side-channel
of exceptions that you can't check or document well.

~~~
futureastronaut
How familiar are you with Java?

~~~
X-Istence
I was about to post something similar to this.

In java you need to specify what your function can throw and if it tries to
throw something that it has not specified as being able to throw, then your
program won't compile.

~~~
weberc2
Agreed, checked exceptions solve the same problem, but having separate control
flow mechanism for certain kinds of data still seems subpar. Return values can
solve the same problems and are more general.

~~~
AstralStorm
Not if you cannot enforce them to be checked.

------
sslnx
Here is the greatest inconvenience of Python exceptions for me. Say you have
to try 10 different methods, and you only need one to work. Then you have to
write 10 try...except blocks so that each next block is indented relative to
previous. This creates unreadable code and does not scale for say 100 methods.
The solution that came to mind is labeling try blocks and referring them in
except blocks. For example:

    
    
      try as method1:
          method1()
      except@method1 try as method2:
          method2()
      except@method2 try as method3:
          method3()
      except@method3:
          raise NoMethodWorked()

~~~
akubera
I can't think of a single time I've needed a large collection of "backup"
methods in case of a chain of failures; I can't even think of an example which
could scale to 100. Anyways, this solution does scale:

    
    
        methods = [method1, method2, method3, method4]
        for method in methods:
            try:
                method()
            except Exception:
                pass
            else:
                break
        else:
            raise NoMethodWorked()
    

You can even pair specific exceptions to each method:

    
    
        methods = [(method1, TypeError), (method2, KeyError)]
        for m, e in methods:
           try: m()
           except e: ...
    

But the whole thing really sounds like you're trying to do too much with one
function and you really should rethink the whole structure of your code.

~~~
sslnx
This is really helpful. Thank you.

------
bvrmn
> There are so maybe potential problems with these three lines of code, that
> it is easier to say that it only accidentally works.

A very strong and emotional point. Accidentally working code _will_ have
exceptional handling because of problems during development.

The whole article is depreciated by naive library implementation because
unwrap() hides source of original exception. Sadly it's not even a POC.

Edit: Failure doesn't capture trace information at all. Library users will get
unusable error in a wrong place.

------
yarrel
> So, the sad conclusion is: all problems must be resolved individually
> depending on a specific usage context.

...and that's a good thing.

Recognising that specific usage contexts require specific recovery strategies
is a key part of effective program design.

Division by zero is an exception, yes. That's basic math.

Getting to the point where one of your inputs is "bad" (zero in this case)
shows that you have a problem. You should either catch zero up front with a
conditional or catch it when it blows up, with an exception. Python favors the
latter.

But without context you don't know what that bad input _means_ and so you
cannot "fix" division by zero in the general case because it isn't the
division by zero that is the _cause_ of the problem.

Asking "what should a division by zero actually return?" is the wrong question
asked at the wrong point with the wrong information. Does the zero indicate
lack of initialization? Does it indicate an empty container or volume? Does it
indicate absence? How much of a problem to the logic of the program is this
particular zero? How much of a problem is it for the person running it?

So while I personally dislike exceptions and prefer return codes, exceptions
are just the messenger here and they are an effective messenger. Don't shoot
them.

~~~
AstralStorm
Interesting fact of life: floating point returns NaN or Infinity for certain
operations and it propagates. Real pain in the rear to debug. You cannot even
reliably trap on it most of the time because there are no such options in the
compilers.

------
X-Istence
It all feels very Go-ish/Java-ish to me vs Pythonic.

~~~
cpburns2009
I was thinking Rust-ish. Java's error handling is similar to Python's.

------
sarah180
If you're not Dijkstra, Hoare, Knuth or a principal contributor to the
language's design, I would refrain from "considered (harmful|anti-pattern|…)."

It's hyperbolic and frames your position in a way that's likely to create
poison. The people without enough experience & knowledge to evaluate your
argument critically are likely to end up parroting it and looking like fools.
The people who _can_ evaluate your argument critically are going to scrutinize
it much more carefully because you've taken an absolutist position.

Unless you have the knowledge & authority to really make such an absolute
statement you do yourself and your readers a disservice.

------
rs23296008n1
Putting the exception handling inside the divide is foolish. Put the handler
outside in the calling function where decisions about why it happened can be
implemented.

Python could use a "raises" keyword though. Whether that has real benefits is
debatable.

------
mistrial9
news flash - there is no perfect system for this.. Its a feature of Python IMO
that a coder can just write a few lines and run.. some architectures may not
need good error handling, except for development e.g. ephemeral data flows

Lazy is good (sometimes)

------
bvrmn
I have a question. How proposed solution can solve problem with exceptional
control flow? Monadic approach skips calculations under the cover and one need
to track branching in his head.

------
rffn
Calling a personal opinion an anti-pattern is an anti-pattern.

~~~
futureastronaut
It reminds me of the Principal Skinner meme. "Maybe I'm out of touch... No,
it's the Python programmers who are wrong."

------
proc0
Yet another Monad tutorial in disguise. Good review of exceptions, etc., and
how to handle them. tl;dr: Use Maybe and Either instances (or equivalent
GADTs) for better control flow.

------
giancarlostoro
And here I am using exceptions to do HTTP redirects with CherryPy. I haven't
had major issues with exception handling in Python yet.

------
ltbarcly3
Lets all agree that it is impossible to write code that will never have an
unexpected outcome. Imagine that we somehow write a function that is totally
bullet-proof. It can't fail, it will always do precisely what it was intended
to do. Further, lets say whenever we run this function we run it on N
computers and take the consensus result if any of the computers disagree. No
matter how large N is, if we run the function enough times eventually we will
get a majority of the computers to agree, return the same result, and that
result will be wrong. Whether it's from cosmic rays resetting bits in memory,
or multiple cosmic ray strikes resetting multiple bits and thus defeating ECC,
sooner or later things will break no matter what you do. Even if you shield
all the computers with 5 meters of pre-nuclear age lead, eventually it will
break.

The point is that it is just not possible to get to perfect reliability. Your
actual reliability is always going to be less than 100%. You can invest money
and effort to get closer to 100%, but obviously you are going to get
diminishing returns.

The correct analysis is to decide where the optimal trade-off is between
investing in reliability and the return on that investment.

Example 1: You are calling a web service that checks the weather. The service
might be down. If it is down you wait a few seconds and try it again. The
'cost' of it being down is that a user doesn't see the current weather. Is it
worthwhile to carefully try to determine whether the error when calling the
service is due to a server returning a 500 status code versus invalid json?

No, it's not worth it. Either way the client can't use the response. In fact,
it doesn't matter what causes the exception, since anything that goes wrong
can't be corrected by the client. Whether it's bad json, a network failure,
dns failure, the server is being rebooted, the webserver is misconfigured, or
the device is in airplane mode, the resolution is always the same, wait and
try again in a few seconds. Exceptions work pretty much ideally in this case,
you only have to code the 'happy' path and handle all exceptions the same way
generically.

Example 2: You are writing code to update a database containing financial
transactions. If something goes wrong in an unexpected way you need to make
sure the financial data isn't updated or left partially updated.

Again, you don't care about unexpected exceptions. For failures you expect and
are coding to work around them, possibly by catching the generic exception
where it happens deep in the call stack, and then raising your own exception
class which properly identifies the error and contains the context necessary
to perform the recovery. For example, if you need to send an email via receipt
for the transaction, you call some function which formats and sends the email.
That function fails due to the email server being unreachable. The network
exception is caught and an EmailCantBeSent exception is raised with the
relevant details in it (the user_id you were emailing, the transaction_id the
email is for). The resolution is to log a critical error and insert a record
to the database with the relevant details of the email so that someone can
make sure it is sent later. Then it continues with the transaction. If
something unexpected happens the database transaction is never committed.

My point is that there are two kinds of exceptions you run into, the ones you
are being careful to trap and resolve as part of your applications design, and
the ones that you aren't trying to resolve and so result in just a generic
'this failed' situation.

So finally getting back to finding the optimal tradeoff between investment to
improve reliability and payback on that investment, you just need to make sure
your generic failures are rare enough that you aren't pushed far from that
optimal point, which is almost always going to be the case, even if you
basically don't ever handle any exceptions and only code for the happy path.
Obviously there are tons of counter-examples and sometimes you need to make
sure things work even when something goes wrong (if you are working on an
autopilot you will require much higher reliability and so much more careful
planning to reach it compared to a twitter client, where you just need to not
lose what the person typed).

Ok that's a lot longer than I intended.

TLDR; If you do any kind of analysis on why code fails and what you should do
about that, you quickly realize that this library doesn't help at all. This
library isn't even bad, the problem it is meant to solve is not well posed.

------
PaulHoule
"X considered harmful"

"X" implies "X considered harmful"

Thus

"X considered harmful" considered harmful

"'X considered harmful' considered harmful" considered harmful

...

QED

~~~
1337shadow
argumentum ab auctoritate

