

Code blocks in Python - mtomassoli
http://mtomassoli.wordpress.com/2012/04/20/code-blocks-in-python/

======
francoisdevlin
This is a terrible idea, and ruby used the wrong implementation. If you want
this behaviour, you should just define a higher order function/decorator, and
put your "block" in its own function. This is a case where there should be
only one way to do it.

~~~
stcredzero
Blocks in other languages can help maintain encapsulation, if they can access
variables from their local scope. They can also enable programmers to write
their own control structures which look like natural first class control
structures in the host language. I don't see this as a natural way of doing
either.

~~~
sirclueless
Nested functions can access variables from local scope in Python, so this
isn't a huge issue. It's a bit wonky in python (see discussion of the nonlocal
keyword, Python did this wrong from the start) but it solves your
encapsulation problem.

    
    
        def sorted_by(xs, attr):
            def cmp_attr(a, b):
                return cmp(getattr(a, attr), getattr(b, attr))
            return sorted(xs, cmp_attr)
    

It takes a little more vertical space and requires you to give a name to
something that might otherwise be anonymous, but in the long term it's
actually beneficial in my opinion. I've actually taken to using the same
pattern in my JavaScript development, giving all of my callbacks names,
because it makes maintenance so much easier later.

    
    
        var load_into = function (url, elem) {
            var handle_response = function (response) {
                $(elem).html(response);
            };
    
            $.get(url, handle_response);
        };

~~~
stcredzero
That's not bad. I agree named functions are fine for many solutions for
maintaining encapsulation. They're not that great for writing your own control
structures. How would you implement my ifTrue:ifFalse:ifMaybe: example?

------
bcl
Seems like a good way to make your code harder to read and harder to debug.

~~~
stcredzero
Anonymous functions can generally be abused in ways that make code harder to
read. Then again, so can recursion. It's a double edged sword.

------
lee
Can someone give me an example where using a code block is better than using a
simple function? And is the added complexity and loss of readability worth it?

~~~
rbanffy
Some people may consider:

    
    
        def is_prime(n):
            if n % 2 == 0:
                return False
    
            sqrt_n = int(math.floor(math.sqrt(n)))
            for i in range(3, sqrt_n + 1, 2):
                if n % i == 0:
                    return False
            return True
    
        def main():
            with concurrent.futures.ProcessPoolExecutor() as executor:
                for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
                    print('%d is prime: %s' % (number, prime))
    
    

uglier than (borrowing some JavaScript syntax):

    
    
        def main():
            with concurrent.futures.ProcessPoolExecutor() as executor:
                for number, prime in zip(PRIMES, executor.map(
                        function(n) {
                            if n % 2 == 0:
                                return False
    
                            sqrt_n = int(math.floor(math.sqrt(n)))
                            for i in range(3, sqrt_n + 1, 2):
                                if n % i == 0:
                                    return False
                            return True
                        }
                        , PRIMES)):
    
                    print('%d is prime: %s' % (number, prime))
    

I don't, mostly because in the second example, I cannot unit test the anon
version of is_prime in isolation. This is a rather trivial case, but I've seen
much more complex and enigmatic anon functions in Node applications.

Note: the original comes from
[http://docs.python.org/dev/library/concurrent.futures.html#p...](http://docs.python.org/dev/library/concurrent.futures.html#processpoolexecutor-
example)

~~~
aiscott
"Some people may consider: [example A] uglier than: [example B]."

Do people really think that? I ask in all seriousness. I am a hardware
developer by profession and don't often dive very deep in software, though I
love exploring software space.

I find example A to be very clear and readable. I like to look at things in
bite sized chunks.

I also like to create things in bite sized chunks. In hardware design,
specifically digital chip design, I will know what my overall problem is to
solve (aka the requirements), but I don't always have a clear view of what
pieces I need to get there.

In the cases where I'm not sure about the smaller pieces, I'll write the
larger overall structure and when I get to a spot where I'll need a block of
smaller functionality, I'll just make up a function name on the spot and
pretend it's already written and does the right thing.

Then I take a second pass, implementing those smaller blocks, and keep going
down. I guess that's a typical top-down approach? I don't know, but it is what
I do.

But I also do bottom up. Even without knowing all the steps between, I'll
often have some idea that there are some basic building blocks I'm going to
need, and I'll implement those.

So I iterate with some steps top-down, others bottom-up, until I meet in the
middle.

All that is a long winded way for me to say that your example A fits both
approaches for me. I can start with the overall structure and implement "main"
first, or know what my basis will need and start with "is_prime" first.

Whereas example B seems like an all at once approach that is both more
difficult to write and read. At least, that is how it seems to me.

edit: wording

~~~
rbanffy
> Do people really think that?

People have been investing a lot of time in this. That certainly indicates
someone considers it a worthwhile endeavor.

------
kibwen
I'd like to see some before-and-after examples of Python code that's been
rewritten to take advantage of blocks using this module.

------
Goladus
Interesting, but I'm having a hard time seeing how this syntax saves me much
in terms of code size and complexity compared to just wrapping the code blocks
into named functions and passing those.

~~~
Peaker
The problem with that is that it forces moving the code above where it is
passed, which can greatly clutter things.

For example, consider the function "forever" in Haskell:

    
    
      forever action = do
        action
        forever action
    

Now you can use it as a new control structure, e.g:

    
    
      forever $ do
        (sock, addr) <- accept listener
        forkIO $ handleClient sock
    

The $ simply means "apply" and is low-precedence, so it removes the need to
put () around the entire argument to be applied.

In Python, you could define:

    
    
      def forever(action):
        action()
        forever(action)
    

but then, to use it, you have to give a name to your function, so:

    
    
      def accept_once():
        (sock, addr) = listener.accept()
        fork(partial(handleClient, sock))
      forever(accept_once)
    

This makes "forever" much less useful as a new control structure/looping
primitive.

In this sense, Python makes DSLs less usable. The built-in primitives are
first-class, and can have code directly within their use. Library functions
are second-class, and can only have code passed by name which must then fully
appear before the use.

Another example is callbacks. The reason "twisted" is probably called
"twisted", and that people hate callbacks so much, is that it forces writing
the code in backwards order, precisely because of this problem. For example:

    
    
      def handle_result(result):
        print "Done:", result
      def connection_started(conn):
        conn.request(SomeRequest(), handle_result)
      def start():
        start_connecting(connection_started)
    

Compare this with Haskell, as an example:

    
    
      start = startConnecting $ \conn -> do
        request conn SomeRequest $ \result -> do
          putStrLn $ "Done: " ++ show result
    

Note, in Haskell, this would actually be worked out to be (by overloading the
semicolon):

    
    
      start = do
        conn <- startConnecting
        result <- request conn SomeRequest
        putStrLn $ "Done: " ++ show result
    

But even the former nested representation is better than the backwards
("twisted") representation that makes people hate callbacks so much.

~~~
hythloday
But you _don't_ have to write your callbacks in reverse order in python:

    
    
      >>> def fight(batman_wins):
      ...     def t(): return "pow"
      ...     def f(): return "oof"
      ...     def b(msg): print msg
      ...     # if_but not defined at this point
      ...     if_but(lambda: batman_wins, t, f, b)
      ...
      >>> def if_but(cond, true, false, but):
      ...     if cond():
      ...       v = true()
      ...     else:
      ...       v = false()
      ...     but(v)
      ...
      >>> fight(True)
      pow
      >>> fight(False)
      oof
    

Sure, if you want to call if_but outside a function, it has to be defined, but
inside a function (as in your example, and in 99% of python development that
doesn't occur in a REPL) _x()_ is effectively _globals()['x']()_. Have I
totally misunderstood your point?

~~~
Peaker
This problem is indeed less severe when the names are defined in a non-local
namespace (inside a class, or as globals).

But callback-style often appeals to defining very-local callback code. In
fact, it would be anonymous in languages that support it, but it is forced to
not only be given a name, but also potentially clutter some namespace (or be
written in reverse order).

Consider having every for/while loop, or every "if" require giving a name to
the code block[s] within it. It sounds insane. The same doesn't sound insane
for library functions, but in my opinion that's really just because
everybody's used to this limitation.

When I moved from Python to Haskell, one of the many joys was that I could
stick an anonymous code block anywhere so easily.

~~~
hythloday
Thanks for the explanation, that makes sense.

------
northisup
This seem like a whole series of posts on 'how to twist python syntax to save
a line of code or two and make it unreadable'.

------
lucian1900
Adding function literals to Python would be nice. Something to turn "inc =
lambda x: x + 1" into "inc = def(x): return x + 1".

But blocks? They're a misfeature of Ruby, badly copied from Smalltalk.

[edit: added "return"]

~~~
recursive
It looks like you changed "lambda" to "def" and added parentheses. It seems
like that feature is already there...

~~~
lucian1900
Perhaps my example was crap. They would also allow anything inside them.

~~~
sirclueless
The problem with multiline lambdas it that no one knows a good way to indent
them. If they weren't butt-ugly python would probably already have them.

~~~
stcredzero
"The problem with multiline lambdas _in Python_ is that no one knows a good
way to indent them."

How about new keyword(s)? "begin" and "end"? I imagine this has thoroughly
been discussed ad-nauseum. Is there a good synopsis?

~~~
recursive
Basically, there is no scope or block ending token anywhere else in python.
And Guido thinks inline multi-line anonymous functions make programs harder to
understand.

------
bicknergseng
I also question the utility and reason behind this. I spend a lot of time
fixing memory leaks from lousy developers who use closures and anonymous
functions incorrectly in JavaScript. I'm just learning Ruby as well, and see a
lot of room for similar black-boxing pitfalls in procs and blocks. It seems to
me like you're reducing code size at the expense of readability and
debugability. That might be the ruby way, but it seems to me like a bad way to
program.

Like I said, I'm just learning... so I'd love counterexamples and reasons why
I'm wrong.

------
fendrak
This is all well and good, but it doesn't appear to add support for the one
thing I've often wanted in Python: proper closure support for functions within
functions.

~~~
d0mine
Do you mean 'nonlocal' keyword?

------
optymizer
"code blocks" is why they invented functions and procedures half a century ago
(and even those differ only in how they return). I think people forget (or
don't know) that these are all labels to jump or branch to. So your named
functions, functions, lambdas, code blocks, etc, become.. well.. equivalent.
Hipsters, don't confuse the young ones with your 30 ways of jumping into a
block of code. Cheers!

~~~
stcredzero
Not the whole story. Code blocks can be used to protect encapsulation and
create DSLs which are first class citizens of their host language.

 _So your named functions, functions, lambdas, code blocks, etc, become..
well.. equivalent._

They can also become so lexically awkward as to be unusable. For example, one
code a toy "fuzzy logic" system in Smalltalk with a handful of methods. The
result looks like a 1st class member of the language, just like the control
structures that are already there. (Same goes for the loops)

    
    
        "Here is the standard if-else"
        (condition)
            ifTrue: [ ... ]
            ifFalse: [ ... ]
    
        "Here is what my DSL can look like"
        (condition)
            ifTrue: [ ... ]
            ifFalse: [ ... ]
            ifMaybe: [ ... ]
    

Doing this with named functions is going to scatter code between different
functions. It's semantically equivalent, but harder to read. Write anything
involved in such a way, and it becomes untenable. Blocks can make such DSLs an
order of magnitude more readable.

 _Hipsters, don't confuse the young ones with your 30 ways of jumping into a
block of code. Cheers!_

A good heaping fraction of hipsters don't understand what's good about code
blocks.

~~~
optymizer
"Code blocks can be used to protect encapsulation". So can functions, which
are already in the language.

"Doing this with named functions is going to scatter code between different
functions". Please show an example.

"A good heaping fraction of hipsters don't understand what's good about code
blocks." That's one way of phrasing it.

------
scott_s
The code examples are confusing. First, I think there is a typo in the first
two examples; I think _<_ should be _< <_. But he never actually explains what
the semantics of _< <_ are until several examples in, and it took me a long
time to figure out that the string value 'x, y=3' gets evaled into code. Which
is strange, to mix strings as code, when the whole point is to have code be
code.

At the very least, the semantics of this need to be explained better. When you
give a new code example, you always need to say "And this is the result."
Otherwise, I can't close the loop; I have new code with new and unknown
semantics, and an unknown result. I need to have a known result to figure out
the new semantics. (Much like you can't solve a single equation with two
unknowns; you either need to have one unknown, or two equations.)

~~~
mtomassoli
Just read the docstring of the module. It's very detailed.

As for mixing strings with code, I still need to adhere to Python's syntax in
order to avoid syntax errors. Anyway, syntax errors in strings are caught at
"rewriting time". Using words instead of operators would also cause some
problems.

~~~
scott_s
You're putting the carriage before the horse. People want to evaluate what it
is they're getting into _before_ they download it. If they can't understand
the code examples, they won't get as far as downloading the module and looking
at the docstring.

I understand why you had to use strings as code. I had two concerns about
that. One, you didn't explain it - I had to figure it out on my own. And two,
if a solution is supposed to make things cleaner, but ends up introducing
warts like that... maybe it's not worth it. What you implemented is
interesting, no doubt, but I wouldn't want to use it for that reason.

However, I think you would benefit greatly from having improved examples. Even
if people don't use your module, they can still build on your ideas if it's
well explained.

~~~
mtomassoli
People want...? codeblocks is free and I don't get paid for it. If someone is
so lazy that he won't even have a look at the doc in the code (as suggested in
my article), I don't want to have anything to do with him or her.

1 click --> bitbucket

1 click --> source

1 click --> codeblocks.py

It's that too much to ask?

~~~
scott_s
_shrug_

I'm assuming that you want people to use your work. People need convincing.
There are thousands and thousands of ideas and projects out there. The ones
that are well explained are the ones that will get attention. I do research
for a living. The research itself is only half the job. _Presenting_ the
research - convincing people that what I did is important and useful - is the
other half.

Basically, what is your objective: do you want to be _right_ , or do you want
to be _heard_? You said you welcome constructive criticism. That is what I'm
trying to provide - except it's not about your work itself, but about better
ways to present it.

~~~
mtomassoli
I do research for the heck of it and if other people think that what I'm doing
is useless, I'm ok with that. That's the mathematician way. The exploration is
the end, not the mean.

I'm not trying to convince anyone that my "project" is useful because that's
not my project anymore. Anyone can look at it, dissect it, propose new
feature, etc... Now that the thrill of the exploration is almost over, I'm
losing interest... but I'll keep an eye on my repository on bitbucket.

By the way, if someone could provide better examples, documentation, etc...
I'd be grateful. Let's just say I'm not much into the "convincing thing". If
you need convincing, don't look at me. I gave you an object. Now it's your job
to figure out what to do with it (or just toss it away) ;)

------
rbanffy
Adding blocks to Python feels like adding classes to JavaScript... Do we
really need them?

~~~
qznc
No. You can always use "named blocks", but they are called functions.

------
llimllib
I did something very similar 3 years ago:
<http://billmill.org/multi_line_lambdas.html>

~~~
mtomassoli
Oh, I know. My module is your fault! :)

~~~
llimllib
I really like the << x syntax, it's devious :)

