

Pitfalls of Callback-Based APIs - rian
http://thelig.ht/callbacks/

======
drawkbox
I think not just callbacks, which are currently a focus due to promise love,
have many balancing ranges that programmers have to operate in to prevent
technical debt and spaghetti code.

That is why programming is hard. Programming takes thought, design, planning,
and why programmers aren't a factory line where you can just add a bunch in.
Programming is solving tough problems and making them easier using good
balance for time, budget, maintenance, integration and more.

Careful nuances, balances, structures, flow and many other things can be
disrupted if programming is treated incorrectly or with lack of care from too
many cooks, too much pressure, or not enough prototyping. This problem will
always exist, good individuals and teams will always know there is no all
encompassing fix from some new tech, there will always be the balance that
determines success using all advancements at your disposal that help you and
consistently improving that. The design has to continually keep up with the
evolution of the project.

------
califield
This article reads like a mind dump lacking a cohesive narrative... I mean
look at how the author closes the article:

> This article isn't really about callbacks and it isn't even really about
> APIs.

Well then, perhaps you should have named the article something else...?

~~~
dalke
I thought it was a good narrative, stepping through the problem, the ending
with the realization that there was no solution.

The essential problem is that any program which can incorporate arbitrary
recursion is subject to the halting problem. The seemingly simple initial
fixes, like a lock or asynchronous callback queue, can improve things. The
author proposes third-party code analysis tools to help; I'm less sure of
that. The Gödel dragon is not easily pushed back, and we often accidentally
create Turing machines.

I think the title's fine. It's better than "Callbacks and the Halting
Problem", since most of the essay covers, after all, "Pitfalls of Callback-
Based APIs".

~~~
peter-fogg
In this case, the dragon is slain without much trouble. In LVish
([https://github.com/iu-parfunc/lvars](https://github.com/iu-parfunc/lvars))
we've got an effect-tracking system that allows us to ensure determinism (read
more here: [http://www.cs.indiana.edu/~lkuper/papers/dissertation-
draft-...](http://www.cs.indiana.edu/~lkuper/papers/dissertation-draft-
latest.pdf)). This would be pretty easily extended to only allow computations
that are safe to log. And this is all in GHC Haskell! No external static
analysis is necessary; the compiler can do it for us.

~~~
dalke
The dragon cannot be slain, only pushed back. It is the halting problem.

Yes, further automated analysis can find some of these problem. That's what
the author of the essay proposes, and it's good to know that there are some
solutions.

Let's go down the road further and expand the system so there are multiple
loggers, and where subsystem logger configuration via an external
configuration file occurs at run-time. Some configurations are acyclic, others
are not.

This could be prohibited by edict - if a program cannot be analyzed then it's
not valid. As the essay points out, there are also formal verification
techniques - and that very few people use them. As I pointed out, it's
surprisingly easy to make a Turing machine by accident. (See
[http://beza1e1.tuxen.de/articles/accidentally_turing_complet...](http://beza1e1.tuxen.de/articles/accidentally_turing_complete.html)
and its HN comments
[https://news.ycombinator.com/item?id=6577671](https://news.ycombinator.com/item?id=6577671)
.)

In practice then, it's very unlikely that most complex software will be able
to use these tools, because it's hard to restrict the solution space to what
those tools can analyze.

Here there be dragons.

------
mkozlows
I find it impossible to take seriously any argument that claims to be true in
general, but where the only example given is logging.

Logging is weird and special, running at unusual times in unusual states, and
therefore has unusual requirements.

So, yeah, having a pile of arbitrary functions that your logger can call is
not really going to work. But if this were a "notify when a comment is posted"
example, the author would really struggle to find problems with that same
approach, because that's a much safer operation that happens in a more
predictable way than logging is.

The only real takeaway from this article is "be careful with your logging
implementation," but hopefully you already knew that.

------
bakhy
Not a very convincing example. A component provides a callback, and then the
callback ends up using the same DB access mechanism that the component is
using? Or the same lock? I mean, is it providing a callback to itself?
Layering the code, or isolating the component from those who subscribe these
unpredictable callbacks seem kind of obvious to me, and would prevent all of
this. The author is right, though - there is something to take away from this,
but I think it's simply the importance of clean separation of concerns. Or,
the importance of good factoring, if I'm using the word correctly? Other than
that, we're entering the realm of the eternal programmers' dream: programs
that write themselves...

------
kragen
Mark S. Miller wrote his dissertation
[http://www.erights.org/talks/thesis/](http://www.erights.org/talks/thesis/)
in large part on expanding the "deferred callback" pattern this article
recommends, including explaining how to make it applicable when it doesn't
seem to be applicable. His solution is based on a sort of ultra-fundamentalist
object-orientation approach, even though (as I pointed out in another thread)
normal object-orientation makes these problems worse rather than better.

I don't think the "Pure Solution" mentioned in the article works. The only
effect of a pure function is to compute its return value, so not only is it
safe to invoke pure functions as callbacks, it's also perfectly useless — you
can't tell whether it even got invoked or not. (Rian in another thread argues
that qsort's comparison function is a "callback", but I don't think that's
what people usually mean by "callbacks".)

~~~
rian
Just curious, how would you describe what most people usually mean by
"callbacks"?

I've always considered function arguments to functions like qsort() as
callbacks, and from my interpretation of Wikipedia it seems to agree. Now I
wonder if my interpretation has been overly general or if a more specific
interpretation has become more commonplace.

~~~
dllthomas
While my impulse was to agree with you, I think there might be a worthwhile
distinction to be made here. If so, we all need to be a bit clearer on it.

If qsort's comparison function is a callback, is that true of any function
argument to any higher-order function? If so, why the additional term? If not,
what makes a function passed as argument a "callback"?

~~~
kragen
When I read "callback", I interpret it as "function argument to be invoked to
deliver a notification to a user-specified place". For example, to tell you
that your transaction has completed, that your network connection was broken,
that your write failed, that a log message is available, etc. Typically a
callback returns no useful result and takes one argument, but sometimes takes
no arguments, and in systems that don't have closures (like C) it typically
takes an additional argument supplied along with the function pointer.

~~~
dllthomas
I think that's pretty reasonable.

------
groby_b
Except that this is not a callback issue. It's an issue of circular
dependencies - callbacks are merely the chosen dependency injection vehicle
for this particular item.

The message pumping example the author gives has the exact same issue. The
core here is that two items (database & logger) are mutually dependant. That
will always be an issue. It doesn't matter if your promises are circularly
dependant, your context objects, your callbacks, or your globals.

Static analysis won't help - fully resolving circular dependencies is
equivalent to solving the halting problem, IIRC.

And so the only way to prevent those dependencies is to constrain the design
in a way that prevents circular dependencies. That's where the ad-hoc rules
mentioned in the article stem from. They artificially restrain the problem
space to prevent (classes of) circular dependencies.

~~~
rian
Yes, the core issue in the database logger example is that it's a circular
dependency.

The point of the article is that callback-based APIs like this obscure the
actual problems (like circular dependencies).

You're right that constrained designs prevents issues like these but callback-
based APIs aren't inherently constrained. They allow anything to happen, which
is why they conversely _encourage_ errors like these (and are subject to
pitfalls).

~~~
groby_b
What would be an inherently constrained API, then? I've successfully managed
to create circular dependencies in quite a few different styles, so I'd love
to find one that saves me from myself ;)

~~~
rian
Maybe inherently was the wrong word. Though you can imagine a logger API that
was like this:

    
    
        typedef enum {
            CONSOLE_LOGGER,
            DATABASE_LOGGER,
            /* etc. */
        } LoggerType;
    
        void add_logger(LoggerType);
    

In this API you're constrained by what loggers you can add. It's not the
totally unchecked free-for-all that a callback-based API provides. The user is
strictly unable to shoot themselves in the foot.

But this limits the expressive power that callbacks provide. Sorry I don't
have any other API recommendations. The only way I can think of to stay
expressive while safeguarding against unintended abuse is to include code
analysis.

------
sbov
Note that the queue doesn't fix the original database logging problem. The
original form is still an infinite loop, the queue just makes it slightly less
noticeable.

~~~
dalke
To make it noticeable, have the database's save_document() log a "saved
element to database" message.

------
deckar01
I find that promises are an elegant dependency resolution mechanism when
requesting a resource asynchronously from disparate parts of an application.
Promises still rely on callbacks, so the initial pitfall of calling `log()` in
`get_database_handle()` could still happen. The main failure I see is that
`log()` is called before the function has a chance to change the state of the
application to reflect the fact that the resource is already being requested.
Instead of creating a new variable to store the state of the request before
logging, store a new promise, log, then resolve the promise.

------
dmethvin
> Time passes and your application becomes multi-threaded.

Is it realistic to think that any app starting with a bunch of assumptions
about being single-threaded can ever just "become multi-threaded" without
bloodshed?

~~~
valleyer
Sure. E.g., AppKit on OS X is in general not thread-safe (still), and for many
years apps were almost exclusively single-threaded. But apps and frameworks
could occasionally spin up threads as long as they were guaranteed to do work
in some bounded set of functions (i.e., as long as they were sure not to call
into AppKit to update the UI...). So while the author's example is a bit
contrived, I can see where it came from.

------
al2o3cr
"Pure functions have no impact on any runtime state and that makes them always
safe to invoke as a callback."

It also makes them UNIVERSALLY USELESS. Think about it: a function that
definitionally has no side-effects is being called _solely_ for its side-
effects.

The article using that to wrap up makes it the equivalent of a complicated
category theory proof that successfully proves that the empty set has TONS of
amazing properties...

~~~
rian
Pure callbacks aren't universally useless. For instance, the qsort() API can
use pure synchronous callbacks effectively:
[http://linux.die.net/man/3/qsort](http://linux.die.net/man/3/qsort)

The article doesn't advocate always requiring pure callbacks. It's just
offered as one possible way to make it easy to reason about the correctness of
using an callback API.

------
SCHiM
I don't see how you can ever pin problems with mutexes to callbacks. The
author seems to confuse multi threading with callbacks. His example seems more
like a mistake in recursion and improper structuring than something that went
wrong because of callbacks. It's not the callbacks fault that you try to
acquire the same lock twice.

~~~
rian
Yeah problems with mutexes are ultimately the fault of the programmer.

The point that the article is trying to make is that because callback APIs
don't expose their correctness requirements and because their correctness
requirements are externally defined, they _encourage_ programming error like
this.

------
syntern
This article feels like it is talking about the pitfalls of not using OO
principles (e.g. types, encapsulation).

I won't shed a tear if callback-hells are replaced by proper async APIs (like
in Stream and Future-based dart:async, or the Thenable in the new JS
standards), but this article seems to be misalinged.

Downvoters: care to elaborate?

~~~
dalke
I'll take a stab at elaborating.

The essay is about callback-based APIs, so types and encapsulation are out of
scope from the start. It then expands the scope and observes that OO
principles don't address the points covered. For example, it says that one of
the "magic" requirements would be a way to state: "Do not add a callback that
calls log() or acquires any locks held while log() is called." There is no
type for that.

Other than pure functional code, there's no way to do that.

The essential equivalent for a multi-actor system is deadlock prevention. OO
principles don't help there either.

You can get the problem with a stream. Consider a logger stream, where the
listener opens a database connection to save the value then closes it, and the
database adds 'open' and 'close' events to the logger stream. This will lead
to an geometric explosion of events on the stream, because nothing at the API
level says you shouldn't put those pieces together that way, other than the
documentation.

I could go into the OO details of how this stream might be implemented in
Dart, but really the OO nature of the stream API obscures the essential self-
referential nature of the problem.

You can propose an equivalent counter-example if you want to demonstrate that
those APIs really do solve the problem. I happen to agree with the well-
written essay, and OO principles or "proper async APIs" solve nothing.

~~~
lmeyerov
Promises are single assignment, so they solve the problem. As soon as they
resolve, they're safe. Calls to them don't happen until they resolve. If you
want to change the logger, you use streams, which will handle safe replacement
for you (flatmap).

We do some crazy async HPC code, and during on-boarding, new members
invariably get some initialization / error handling wrong with raw callbacks,
and convert pretty quick to FRP after that. Unstructured async is crazy.

~~~
dalke
Yes, the example problem doesn't match to promises.

I don't understand the "change the logger" comment. The example was the logger
sends a message to the listener, which opens a database handle, which sends a
message to the logger, and repeat. There's nothing to changing. I don't see
how FRP helps eliminate that cycle.

~~~
lmeyerov
This might be what's happening, which would surface as no sensible declaration
ordering:

//try making the logger first

var logger = require('myLogger')(backends);

var listener1 = require('mySync')(logger);

var backends = Rx.Observable.fromArray([listener1, listener2]);

//try making the backends first

var listener1 = require('mySync')(logger);

var backends = Rx.Observable.fromArray([listener1, listener2]);

var logger = require('myLogger')(backends);

From there, you'd switch to unsafe FRP methods (imperatively injecting into
event streams, e.g., Subjects in Rx), which is the warning sign of weird
cyclic behavior.

