
A Programming Idiom You've Never Heard Of  - pavel_lishin
http://prog21.dadgum.com/121.html
======
psykotic
This idiom is even more useful in functional programming where the last step
is an inverse of the first step in the applicative sense rather than in the
side-effects sense.

Mathematicians call it conjugation, and it can be usefully thought of as a
temporary change of coordinates or change in vantage point. Problems often
become simpler, even trivial, when working in the right coordinates.

In practical programming, it is commonplace that the "inverse" that occurs in
conjugation idioms is a genuine right-inverse but only partially a left-
inverse. This is true for his example of squares and square roots. For another
instance, consider

    
    
        reverseWords = unwords . reverse . words
    

The composition words . unwords :: [String] -> [String] is the identity, but
unwords . words :: String -> String is the identity only for strings whose
words are separated by a single space rather than multiple spaces, tabs, etc.
This property can actually be put to good use:

    
    
        normalizeWhitespace = unwords . words
    

If you like, you can think of unwords . words as an orthogonal projection onto
the subspace of strings on which unwords is a genuine left-inverse of words
(cf. Moore-Penrose pseudoinverses).

That's cool and all, but the effect on reverseWords is that it normalizes
interword whitespace in the process of reversing words, whether you like it or
not.

Incidentally, it's not too hard to abstract out this pattern with invertible
functions:

    
    
        data Invertible a b = Invertible (a -> b) (b -> a)
    
        invertible = Invertible
        apply (Invertible f f') = f
        unapply (Invertible f f') = f'
        inverse (Invertible f f') = Invertible f' f
        compose (Invertible f f') (Invertible g g') = Invertible (f . g) (g' . f')
        conjugate i f = unapply i . f . apply i
        on = flip conjugate
    

Now you'd have to set up invertibles for all the usual foo/unfoo pairs of
functions. Assume that has been done. Our example only needs words/unwords:

    
    
        import Data.String as S
        words = invertible S.words S.unwords
    

With those preliminaries out of the way, you can then reimplement reverseWords
very neatly:

    
    
        reverseWords = reverse `on` words
    

In fact, it's now so concise and self-explanatory that you hardly need the
named definition anymore and can just refer to reverse `on` words directly.

~~~
sfvisser
One of the interesting properties of the Invertible datatype is that it can be
made an instance of the Haskell Category type class, which makes invertible
functions fully composable using the (.) operator.

~~~
dons
Yes, composable data structure editors (lenses) with an overloaded (.) make my
day to day work much more pleasant.

------
sgentle
A similar pattern that blew my mind when I first saw it was ruby's scoped
open().

instead of

    
    
      f = open "test.txt"
      print f.read
      f.close
    

You can write

    
    
      open "text.txt" do |f|
        print f.read
      end
    

The file is automatically closed at the end of the block, and it also handles
exceptions and so on. I find it also lines up better semantically with how I
think about opening handles to things.

I believe Python also added something similar via the 'with' syntax. Not sure
about other languages.

~~~
jlarocco
Yeah, Python has "context managers" that let you do the same thing. Here's the
Python equivalent:

    
    
        with open("text.txt") as inf:
            print(inf.read())
    

User defined classes that implement special methods __enter__() and __exit__()
can be used as context managers. More info:
<http://www.python.org/dev/peps/pep-0343/>

For just file output/input Scheme has the "with-output-to-file" and "with-
input-from-file" macros which redirect output and input to a given file. I
vaguely remember Common Lisp having a similar thing, but I can't remember the
name off hand.

~~~
Luyt
I often use:

    
    
      for line in open("name.txt"):
          print line # Or do something else with it.

~~~
reinhardt
Me too when I'm lazy but we shouldn't; it happens to work fine on CPython but
on implementations use garbage collection instead of reference counting (e.g.
Jython) it leaks file handlers. The "with" statement is the right cross-
platform way to handle resources.

~~~
rbanffy
I'd imagine it's an implementation bug. Once the iterator finishes, the file
should be closed and the file handler freed.

~~~
yangyang
Perhaps in this case - but how about a case where you partially iterate over
the file, but then the file object goes out of scope? CPython would close (and
free) that immediately, but other implementations probably wouldn't.

~~~
rbanffy
While editing the answer to masklinn comment, this case came to me. You both
are, of course, right. There is no reliable way to close the file from the for
iterator.

:-/

------
thristian
Interestingly, Python has the "with" statement for this basic genre of
operation (executing a block of code while some particular resource is held),
although it doesn't require that the third, "undo" step be automatically
derived from the first step - it's just a protocol where a 'start' and 'end'
method are called on the object representing the resource.

It sure makes it easy to be sure you're closing your file handles/database
connections/mutexes, though.

~~~
heyitsnick
I find it very handy with GUI programming under python too.

with wait_cursor(wxwindow):

    
    
       ... change cursor to hourglass whilst process runs...
    

I have similar context processors for locking controls or other temporary GUI
changes

------
chrislipa
This idiom is common in mathematics, where it's called a conjugate.

Any time you have an element, y, of a group, the action of pre-composing with
any other element, x, and post-composing with x's inverse to get xyx^{-1} is
called conjugating, and the result is a called a conjugate of y.

<http://en.wikipedia.org/wiki/Conjugation_(group_theory)>

------
robotresearcher
Constructor / Destructor pairs are how you do this in C++. This is so
fundamental to the object system that it's a bit steep to claim that "you've
never heard of it". You have. The author's example of "get the rake - use rake
- put away rake" maps exactly to [construct - call methods - destruct].

Given the jmp-iness of exception handling in C++, this is the recommended
(i.e. the only safe) way to clean up after yourself.

~~~
dextorious
This is just for resource allocation.

The pattern the article talks about is FAR more general -- it's about all
reversible processes, even things that have nothing to do with allocation.

~~~
hythloday
Sentry classes, which are a generalization of RAII, are used idiomatically in
C++ for non-allocation tasks, such as whitespace skipping, logging, and
profiling.

~~~
JoeAltmaier
Also mapping state to scope e.g. mutex objects that set/clear on ctor/dtor

------
sigil
_Is there a programming languages where functions have inverses?_

How about a computer architecture where every instruction has an inverse, and
therefore all functions and even programs are invertible? (Quantum computing,
for example.)

<http://en.wikipedia.org/wiki/Reversible_computing>

~~~
andrewflnr
Interesting. I'm sure you could do a VM where all the opcodes are reversible.
Definining inverses of functions as inverses of opcodes and other functions
would then be feasible. Might make a cool project.

------
jbarham
Go handles this type of situation nicely using deferred function calls:

"Go's defer statement schedules a function call (the deferred function) to be
run immediately before the function executing the defer returns. It's an
unusual but effective way to deal with situations such as resources that must
be released regardless of which path a function takes to return. The canonical
examples are unlocking a mutex or closing a file."
<http://golang.org/doc/effective_go.html#defer>

------
v-yadli
Seems that everybody is talking about files. :-) Open, operate, close. I think
the author tries to bring us more, mathematically. Like a matrix operation:
Move to origin, rotate, move back. But this is a lot harder than just
"resource guard", imo. Any ideas?

~~~
morsch
Exactly. So far I'm kind of disappointed by the HN debate, it seems like many
people only read half the article and are then quick to point out that their
own favorite language has a way to release resources when done.

I liked the _sum under square_ better than the _read under open_ example
(although the latter may be more relatable). _Sum under square_ squares items,
sums them up, and then performs the "inverse-square" operation on the sum. I
don't think this is similar to _using X_ or RAII in C++.

------
scscsc
Bidirectional programming is somewhat related.

[http://www.cis.upenn.edu/~bcpierce/papers/lenses-
etapsslides...](http://www.cis.upenn.edu/~bcpierce/papers/lenses-
etapsslides.pdf)

e.g. read a file, parse it into an AST, modify AST (i.e. rename a variable),
unparse the new AST, close the file.

~~~
andrewflnr
Awesome slides. Would you happen to have a transcript for it as well. The
slides themselves are pretty clear, but there's probably stuff missing.

------
lispm
That's in many cases what a LET with a dynamic binding does in Lisp.

Set origin draw Reset Origin

is

    
    
        (let ((origin (move 50 50)))
          (draw something))
    

Similar in the Common Lisp object system we use :before, :after and :around
methods.

    
    
        (defmethod draw :before () (move 50 50))
    
        (defmethod draw () (draw-something))
    
        (defmethod draw :after () (move -50 -50))

~~~
draegtun
> _Similar in the Common Lisp object system we use :before, :after and :around
> methods_

Ditto for Moose (in Perl)...

    
    
      before draw => sub { shift->move( 50, 50 )};
    
      sub draw { shift->draw_something }
    
      after draw => sub { shift->move( -50, -50 )};
    

Also you can package up these method modifiers into a role which not only can
be composed into a class but can also be applied just to an object...

    
    
      {
        package DrawRole;
        use Moose::Role;
        
        before draw => sub { shift->move( 50, 50   )};
        after  draw => sub { shift->move( -50, -50 )};
      }
    
      my $d = Draw->new;
      DrawRole->meta->apply( $d );

------
sdevlin
People have posted a lot of examples in a lot of different programming
languages. What they show is that all you really need to build this idiom is
first class functions (or some approximation thereof). (It also shows that the
title is not correct; many, many people are familiar with this idiom.)

When you need to wrap up a common block of code, you use a function. This lets
you call the same block in a number of different contexts. When you need to
fix the context in place and let the block in the middle vary, you write a
function that takes another function as a parameter. An example in JavaScript
(might be a little off):

    
    
        var withFile = function (path, action) {
            var f;
            try {
                f = File.open(path);
                action(f);
            } finally {
                File.close(f);
            }
        };
        
        withFile('foo.txt', function (f) { ... });
    

This can be generalized (to some extent and depending on language) so you
don't need to write the same structures every time:

    
    
        var context = function (prolog, epilog) {
            return function (action, ...args) {
                var o;
                try {
                    o = prolog.apply(args);
                    action(o);
                } finally {
                    epilog(o);
                }
            };
        };
        
        var withFile = context(File.open, File.close);
    

This example uses JavaScript' hypothetical future "rest" parameters, but there
are other ways to manage the same goal.

(Other comments have mentioned the author conflates context management with
applying reversible functions. I've only dealt with one of those things here,
but similar techniques apply in solving both problems.)

~~~
dwc
I think that few people are actually aware of this as an idiom. Rather, many
people are aware of just a few special cases, but have not considered this in
the more general form.

All the examples of various languages providing this for open files also kind
of misses it. The point is not whether a particular language supports this for
open files, but whether people recognize the idiom and the language supports
doing this yourself wherever it fits. For a pattern this incredibly common
it's a point well worth making.

~~~
sdevlin
Yeah, you may be right that the full generality of the pattern is not
appreciated.

For what it's worth, I think most commonly used languages can implement some
form of the context function in my example.

------
jey
C++'s variant is RAII:
[http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initial...](http://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization)

~~~
angelbob
What RAII and Ruby's block statement lack here are automatically doing the
reversing of functionality.

Few languages have any procedural way to _do_ that.

If Ruby had some way to state "the opposite of opening a file is closing a
file" and then use it everywhere, we could do that. The block-syntax open is
_nearly_ that, but not quite. You'd need to have no non-block-scope way to use
the resource for it to work.

RAII would also give the same benefit, _if_ there were no non-RAII way to
declare the resource.

~~~
strictfp
Hmm. I don't agree with this conclusion. Take for instance a file class in C++
which opens the file descriptor in the constructor and closes it in the
destructor. Now, for any instance of this class, it is impossible for the open
fd to outlive the instance!. That is an as strong guarantee as I need! Now, if
I leak the instance the fd will leak too, but that is more an issue of manual
lifecycle management if you ask me.

This very significant property is what makes me love RAII. I don't count C#'s
using-statement and/or rubys block statement as RAII since it relies on useage
patterns. C++'s automatic destruction of graphs of objects also makes it
outshine all other methodologies, and the guaranteed destruction is the key
here.

~~~
vidarh
RAII _is_ a usage pattern: You must stack-allocate an object or wrap it in a
suitable smart pointer. It's just as easy to do things differently as it is
with Ruby's blocks or C#'s using statement. The reason it has a name is
because it is a pattern that's talked about so much exactly because following
the pattern is so intrinsic to modern C++ programming.

Once you're passing a block to a method like this in Ruby, the execution of
the cleanup code is just as guaranteed as it is in C++.

~~~
strictfp
I don't agree completely. As I said, you cannot leak the fd without leaking
the memory of the object. This means that in C++ you get the correct behavior
as long as you take care of the lifecycle of your objects correctly. You don't
need to do stack allocation or smart pointers as long as you de-allocate the
object when your are done with it. So the lifetime of the fd is tightly bound
to the lifetime of the wrapper instance. If you stack allocate without
deallocation you have a bug _anyway_ in your C++ program, thus I would like to
argue that it is a problem of manual lifecycle management, not a flaw in the
RAII implementation.

You do however have a point when you say that you can miss deallocation and
that is just as easy as forgetting to put a variable in a block. In an ideal
world I would like to be able to decouple de-allocation time from destruction
time, as someone mentioned above they are completely orthagonal issues. If
this would be possible, one could guarantee the destructor callback, but defer
deallocation until later. This would probably give leeway to more effective
memory allocation patterns while keeping RAII alive.

And another thing about blocks: how do you deal with trees and graphs of
objects? That is not trivial when using blocks. You can get around it, but
continuing that way would most likely lead you to something like Haskells
monads (the "programmable semicolon" aspect of them).

------
d0mine
It is a common idiom in C++ RAII
[https://en.wikipedia.org/wiki/Resource_acquisition_is_initia...](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization)

Context managers in Python:

    
    
      with fly('Seattle'):
          see_sights()
    

Real-world example from Fabric <http://fabfile.org>:

    
    
      with cd('/to/path'):
          do_stuff()

------
rmc
Python has this with the "with" statement. It's often used for files, but you
can write your own with classes and an __enter__ and __exit__ method.

However you can use the ``contextlib`` module to make really simple
``contextmanger``'s with the yield statement

e.g.:

    
    
        from contextlib import contextmanager
        
        @contextmanager
        def glcontext():
          glBegin()
          yield
          glEnd()
      

(This is an example where someone wanted to do OpenGL in python)

------
sbmassey
The OP seems to be conflating two different things: undoing an action (closing
a file, etc), and performing the inverse of a reversible function.

The first is amply covered by the RAII/unwind-protect etc idioms than most
languages do have.

The second is not, as most programming languages don't have the concept. The
closest I can think of is the ability to use a predicate in either direction
in Prolog (or logical programming in general)

------
tikhonj
This reminds me of my friend's project for a recent hackathon where he
implemented a language that guaranteed every function in it was reversible. It
would let you do something like this for _any_ function.

Unfortunately, I don't think he found it particularly useful and gave up on it
after the hackathon. Still, it was a cool project.

~~~
edsrzf
This sounds fun. Is there a link anywhere?

~~~
snprbob86
There are a pair of research languages called Harmony and Boomerang [1] that
does this with what they call "lenses". There are also some easy-reading
introductory slides [2,3].

[1] <http://www.seas.upenn.edu/~harmony/>

[2] [http://www.cis.upenn.edu/~bcpierce/papers/lenses-
etapsslides...](http://www.cis.upenn.edu/~bcpierce/papers/lenses-
etapsslides.pdf)

[3] <http://www.cis.upenn.edu/~bcpierce/papers/harmful-mfps.pdf>

------
sdevlin
C#'s 'using' statements allow this kind of pattern for types implementing
IDisposable. It looks something like this (might be a little off):

    
    
        using (var file = File.Open("foo.txt"))
        {
            ...
        }

~~~
dubajj
This is what I thought of immediately except to implement the end with
behavior you have to override the IDisposable shut down sequence? (does that
make sense?)

------
cpt1138
LISP has had this for years:

(with-open-file (output "foo.txt" :direction :output :if-exists :supersede)
(print-foo output))

~~~
andrewflnr
That's not a general op/inverse thing like J apparently has, though. Also,
which lisp?

~~~
prodigal_erik
That looks like Common Lisp, which also has a general version named unwind-
protect:
[http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...](http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec/Body/speope_unwind-
protect.html)

~~~
SeanLuke
Not really. unwind-protect is just Lisp's version of Java's "finally"
statement.

~~~
prodigal_erik
Oh you're right, it'd need to take a param with the setup form(s) and somehow
look up the corresponding cleanup form(s), sort of like getter and setter code
registered with setf.

------
ssp
The language Nickle (<http://nickle.org>) has a "twixt" control structure:

    
    
        twixt (expression1, expression2)
               statement1
    

If expression1 is true, then statement1 will be executed followed by
expression2 in that order. The twixt statement guarantees that expression2 is
always executed if expression1 is true, similarly to how "finally" works in
java.

~~~
groovy2shoes
> If expression1 is true, then statement1 will be executed followed by
> expression2 in that order. The twixt statement guarantees that expression2
> is always executed if expression1 is true, similarly to how "finally" works
> in java.

Am I missing something here? How is that different from, say:

    
    
        if (expression1) {
            statement1
            expression2
        }
    
    ?

~~~
ssp
It's different in that expression1 is always evaluated whenever control enters
the twixt and expression2 is always evaluated whenever it leaves. A simple
example is that if statement1 throws an exception, then expression2 is
evaluated before the exception is handled. This is how finally works in Java.

More interestingly, Nickle has support for continuations, so if statement1
generates a continuation and makes a non-local exit, then expression2 is
evaluated before that exit, and expression1 is evaluated before the
continuation is resumed.

In other words, statement1 is _always_ bracketed by expression1/expression2,
even in the presence of non-local control flow.

~~~
groovy2shoes
Aha! Thanks for clearing that up.

------
ajuc
If there was a way to mark functions and their inverses consistently in
language, then functions with inverses could be used in (semanticaly) pure
functional code, even if they were destructive.

For (stupid) example:

    
    
        def reverse(x) as inversible with(reverse): #special case - reverse is itself inverse function
            x = x.reversed() #I'm too lazy to code this by hand
    
        def lastFive(x) as pure: #let's say language checks if pure function only uses pure and inversable functions
            return x[:5] under reverse
    

Here lastFive is pure function even thought reverse is destructive.

This assumes everybody sticks to one definition of reversing something, in
this case functions f and g are inverse of each other, when after invoking
f(x), doing only pure or reversable things to x, and invoking g(x), x is
unchanged.

And I don't know if this will be useful for anything - the only point seems to
be avoiding making copies and still making pure functions. But resulting code,
while semanticaly pure, is not threadsafe. So I don't know if it's worth
anything.

------
drx
You could dispense with the 'you've never heard of'.

------
phzbOx
IIRC, we use this Idiom in SICP while doing symbolic algebra.

For instance, when we have x+y=z and know only x and z, we need to reverse the
equation to have y=z-x. The system was built on nodes which would propagate
change (a-la events).

IMO, python's with statement with __enter__ and __exit__ looks mostly like
this.

------
niklas_a
I refuse out of principle to click on links with headlines that are so
obviously trying to trick me into going to a site.

Make the headline informative instead, if you have great content people will
read and share it anyway.

Better version of this headline:

XXXX: A programming idiom that helps you do YYYY

------
Zak
Other posts have already mentioned that specific macros of this form exist in
several Lisps. I've seen more than one macro-writing guide that talks about
the with-something macro pattern for managing resources, and I've seen it
applied in libraries that provide macros like with-database and with-query-
results.

CL and Clojure could easily support a generic with macro requiring that there
be an open and close method for the resource in question.

------
dromidas
In C# this is "using". using(new something as something) { Do something with
it. } //it goes away now.

Typically this idiom is the function call itself. The act of sipping coffee is
the point of the function, and picking up the cup and putting the cup back
down are the whole point of the function call.

------
mvanveen
This reminds me of Suzuki's idiom, "Leave No Trace," found in the book _Zen
Mind, Beginner's Mind_.

------
singingwolfboy
Python has this idiom for a while now: it's known as a "context manager", also
known as the "with" statement. Here's the PEP:
<http://www.python.org/dev/peps/pep-0343/>

------
kabdib
The BETA language does this with methods (you can't override a method, but you
can add pre-and-post operations that wrap it).

I was never convinced this was a great idiom. BETA didn't become very popular,
in any event.

~~~
jon6
Beta lives on in the Racket class system as (inner).

[http://docs.racket-
lang.org/reference/createclass.html?q=inn...](http://docs.racket-
lang.org/reference/createclass.html?q=inner#\(form._\(\(lib._racket/private/class-
internal..rkt\)._inner\)))

------
thurn
An interesting approach to this is the one used in Go, where the "defer"
keyword schedules code to run on function exit. It's simple and doesn't
require adding more nesting to your code.

~~~
vidarh
Ruby has "ensure" if you want something to be guaranteed to run on exit - it
can be used at the end of methods, or by explicitly wrapping code in begin ..
end.

~~~
radiowave
(Assuming it works the same way as Smalltalk's ensure...) a really nice aspect
of this is that the resource is held open while you're in the debugger, but
cleaned up when you finish debugging.

------
rbanffy
"Never heard of" is quite a stretch.

During my dark past as a VB programmer (early-mid 90's) I even indented the
code for pairs like this.

------
lelf
Oh, c'mon! All people with _with_ and _RAII_ and files, please reread the
article. At least the last example.

~~~
jodrellblank
I was going to argue that point, but glance at the J docs here:

<http://www.jsoftware.com/help/dictionary/d202n.htm>

Obverse is defined in a table and some special cases, and there's a built in
"this is the obverse of that" instruction.

You could easily tie square/sort together as a pair in Python and it wouldn't
be significantly different.

Under(square, sum, mylist) or whatever.

------
tkahn6
Pretty sure Ruby has this.

File.open(name, mode) receives a block, executes it, and then closes the file.

~~~
silentbicycle
Ruby has a specific instance of that pattern, but to my knowledge it lacks a
way to say, "do some arbitrary operation _that has a side-effect_ , do this,
then undo the operation". The function taking the block still has special-case
code to open and close files.

~~~
skrebbel
You still have to implement the inverse of the arbitrary operation, at some
point, somehow.

A programming language can't figure out what the inverse of open() is without
someone somewhere telling it to close(). The point of the idiom is to have a
library figure that out so that users don't have to. Therefore, Ruby supports
it fully (there where API developers are sane, that is).

~~~
silentbicycle
Having that built in for _every side-effect in the language_ is pretty
amazing, though.

Case by case, the block scoping idiom is a good way to handle it.

