
Click – Python library for command-line interfaces - grn
http://click.pocoo.org/
======
the_mitsuhiko
I did not expect this to be on hackernews this early. I want to point out that
I have not made a release yet and it's not yet feature complete. Mainly I want
to ask for feedback on the general design.

~~~
timtadh
Armin, my feedback is in the form of my own version of this library.[1] I have
been working on this on and off for some time and like you have not made a
release. That said, I have been using it quite a bit both in my research and
at work and I think it helps.

The main idea behind it is make it easy to write a "getopt" style program with
arbitrary command nesting. I have found that although argparse and sisters are
nice libraries they don't allow me to do many of things I like to do in my
interfaces. For instance sometimes like options such as

    
    
        foo -x a -x y -x q ...
    

where I would process that into like so:

    
    
        extras = list()
        for opt, arg in opts:
            if opt in ('-h', '--help',):
                util.usage()
            ...
            elif opt in ('-x', '--extra'):
                extras.append(validate_or_die(arg))
    

I also believe that you should have "fast fail" validators. So I have several
in `optutils` which are like:

    
    
        util.assert_dir_exists(path)
    

which if a directory doesn't exist on the path it creates it. If there is
already a file there and it isn't a directory it dies with an error. When it
dies, I try and have unique exit codes for various errors (for testability)
and provide usage information immediately. This style is nice because it
provides immediate feedback to the user with no fuss. I think a lot of "option
parser frameworks" miss the point in having lots of things for parsing ints
and things. Most of the time I deal with files, directories, and "string"
parameters which these libraries don't help with.

In general, the standard libraries make it way to hard to write really nice
command line tools. I like some things about your library, but I think that
you need to increase the flexibitly for how options are processsed to you can
do whatever you want with them. I also think that option parsing and
configuration should be integrated. I am working to support that but I am not
there yet. (see optutils/conf.py for my current ideas)

[1] [https://github.com/timtadh/optutils](https://github.com/timtadh/optutils)

~~~
masklinn
> I have found that although argparse and sisters are nice libraries they
> don't allow me to do many of things I like to do in my interfaces. For
> instance sometimes like options such as

?

This is trivial to do with argparse (or optparse for that matter):

    
    
        import argparse
    
        def a_prefixed(string):
            if not string.startswith('a'):
                raise argparse.ArgumentTypeError("%r does not start with 'a'" % string)
            return string
    
        parser = argparse.ArgumentParser()
        parser.add_argument('-x', '--extra', action='append', type=a_prefixed)
    
        print parser.parse_args()
    

resulting in:

    
    
        > python test.py -x afoo -x abar
        Namespace(extra=['afoo', 'abar'])
        > python test.py -x afoo -x baz
        usage: test.py [-h] [-x EXTRA]
        test.py: error: argument -x/--extra: 'baz' does not start with 'a'
    

> I also believe that you should have "fast fail" validators.

Isn't that what the callback parameter is for, especially with is_eager=True?
Or ParamType if you need either something more reusable or something more
extensive.

> Most of the time I deal with files, directories, and "string" parameters
> which these libraries don't help with.

[http://click.pocoo.org/api/#click.File](http://click.pocoo.org/api/#click.File),
argparse has something similar.

~~~
timtadh
@masklinn good point. I haven't used argparse (mostly out of compatibility
requirements with 2.6). So I may have mis-characterized the state of the art.
It does do many things well. One note, is that the `prefix parsing`
functionality could cause weird behavior when doing partial parsing (which I
do a lot of).

> Isn't that what the callback parameter is for, especially with
> is_eager=True? Or ParamType if you need either something more reusable or
> something more extensive.

yes. I think almost everything should work like this. I think the getopt style
makes this a bit easier to understand.

>
> [http://click.pocoo.org/api/#click.File](http://click.pocoo.org/api/#click.File),
> argparse has something similar.

Not at all the same. I never said my programs were going to open the files
themselves. I often have to write automation scripts around other things. In
these cases I need to make sure files and directories are sane but I don't
open them. I just canonicalize them and pass them on.

The big thing is the lack of integration with configuration files which is
something I am still working on myself.

------
hoodoof
Gorgeous. This guy has a full time job, a wife, and wrote Flask and Werkzeug,
ItsDangerous, Sphinx, Markupsafe, Jinja2 (and Jinja) and that's just some of
the well known stuff. Handsome and young too. Bastard. He probably saves the
world in a dinner jacket in his spare time.

~~~
lftl
To be fair he only got married this month, so the jury is still out on where
his productivity goes from here ;)

~~~
the_mitsuhiko
Given that I wrote parts of this on my honeymoon I think it helped. Fingers
crossed :)

~~~
Cyph0n
You never cease to amaze :D

~~~
mixmastamyk
That's actually not a good sign for future marital bliss.

~~~
megabulldog
o rly? ;)

------
tudborg
It looks really well done.

I still prefer docopt
([https://github.com/docopt/docopt](https://github.com/docopt/docopt)) . It is
so much simpler to use. Click seems to be a bit overengineered.

~~~
ramblerman
As someone who has never used either, the documentation and simple example of
Click won hands down.

I could use it for simple cases within 15 seconds of reading. Docopt not so
much.

~~~
aidos
You're right - looking on the [http://docopt.org/](http://docopt.org/) page
it's not immediately obvious how it works.

The general idea is that you don't specify any code, you just give the help
message as text. Docopt parses that to figure out all the options etc and
convert those into the parameters coming in to your system.

It's a really clever idea. You specify the human interface, docopt converts
that into the code version (so long as you adhere to a few common
conventions). I haven't seen a cleaner system anywhere.

From their example:

    
    
        """Naval Fate.
        
        Usage:
          naval_fate.py ship new <name>...
          naval_fate.py ship <name> move <x> <y> [--speed=<kn>]
          naval_fate.py ship shoot <x> <y>
          naval_fate.py mine (set|remove) <x> <y> [--moored | --drifting]
          naval_fate.py (-h | --help)
          naval_fate.py --version
        
        Options:
          -h --help     Show this screen.
          --version     Show version.
          --speed=<kn>  Speed in knots [default: 10].
          --moored      Moored (anchored) mine.
          --drifting    Drifting mine.
        
        """
        from docopt import docopt
        
        
        if __name__ == '__main__':
            arguments = docopt(__doc__, version='Naval Fate 2.0')
            print(arguments)

~~~
voltagex_
That's brilliant. I wonder if there's a way to use docstrings like that to
create REST services in Flask.

------
sotte
Does anybody know docopt? I really like it because you simply write the
help/usage as text and docopt automatically generates the parser for it.

Take a look at the example in the README:
[https://github.com/docopt/docopt](https://github.com/docopt/docopt)

~~~
denibertovic
I'm using docopt....It's really nice with it's automagicness. But for some
stuff it's just not enough, too unflexible. I'm really excited about click.

------
clarkevans
It seems to me that there are a few projects similar this. Here is another,
[https://pypi.python.org/pypi/Cogs/](https://pypi.python.org/pypi/Cogs/)
(conceptualized as a "Makefile" replacement).

I'm wondering if there could be a breakout at the next PyCon to see if we
could discuss approaches and come up with a unified way to do convert Python
libraries into command line scripts?

~~~
toyg
Yeah, plenty:

\- Cliff
[http://cliff.readthedocs.org/en/latest/](http://cliff.readthedocs.org/en/latest/)

\- docopt [http://docopt.org](http://docopt.org)

\- argparse

\- optparse

etc etc etc...

~~~
windexh8er
And the author acknowledges that:

"There are many alternatives to click and you can have a look at them if you
enjoy them better. The obvious ones are optparse and argparse from the
standard library.

click is actually implemented as a wrapper around optparse and does not
implement any parsing itself. The reason it’s not based on argparse is that
argparse‘s design does not allow proper nesting of commands by design and has
some deficiencies when it comes to POSIX compliant argument handling."

What I love about Pocoo is they always have stellar documentation and give
clear rationale - from the beginning.

~~~
vonmoltke
Does that mean optparse will be maintained along with click? optparse has been
deprecated in favor of argparse since 2.7/3.2.

~~~
the_mitsuhiko
Given that argparse has known problems and that not all optparse code can be
ported to argparse I doubt optparse will ever go away. It's also a very small
module and if optparse will ever disappear only a subset of optparse is needed
to make click work.

------
pandatigox
Kudos again to the pocoo team for creating such a simple and useful library.
It sure beats optparse :)

On a side note, however, does anyone know why the team prefers to wrap
functions in decorators? Flask also uses them, but what's the design decision
behind them?

~~~
hoodoof
Decorators are so simple and minimalistic and elegant and easy to understand.
They can surface an arbitrary amount of extra functionality with a single
statement. For example, ensure a function is only accessible to authenticated
users by simply saying something like @requires_authenticated_user. Another
example would be to create a logging decorator to wrap a function or a class
to create a log of every time the function is accessed @log_function_access.
Or to wrap a function to ensure that every access to the properties of that
function are pushed through some sort of validation or transformation
@validate_set or @transform_get. Decorators allow encapsulation of functions
within other functional concepts with almost no code, that's the design
decision for using them.

~~~
HeyImAlex
Yep, it's best us case is basically acting as pythons anonymous function
(since lambdas suck for everything but the most trivial things).

Where in js flask might read app.route('/', function(){//do something});
python uses a decorator right above a regular function definition. The
alternative is to manually add it after it's defined (in flask that's
add_url_rule), but that kind of hides the intent of the function and is more
cumbersome as you need to manually pass in the fn name.

------
peterjs
On a related note. Which python library would you recommend for text-based
interfaces? I have never used ncurses, so I don't know how complex it is. What
I would like to achieve is having a user launch my script from the command
line, use the text based interface to select a source and destination folder,
set a few parameters and show a progress bar.

~~~
phaer
I'd say [http://urwid.org/](http://urwid.org/) should be the best choice for
something like that. The main alternative would be to hack it together by
yourself using
[https://docs.python.org/3.4/library/curses.html](https://docs.python.org/3.4/library/curses.html)

~~~
SEJeff
Urwid is definitely the nicest of the curses widget frameworks I've used for
python.

------
bru
>You can get the library directly from PyPI:

>pip install click

Well... no you cannot. See the page:
[https://pypi.python.org/pypi/click](https://pypi.python.org/pypi/click)

The package hasn't been uploaded yet. However one can install it straight from
the git repo:

> pip install git+ssh://git@github.com:mitsuhiko/click.git

~~~
philtar
> pip install git+ssh://git@github.com:mitsuhiko/click.git

Can someone comment regarding using pip like that? Is it fine to put this on
req.txt? Any best practices somewhere?

~~~
toyg
That syntax is usually meant for packages that you want to edit after
installation (with -e).

If you plan to release your stuff, the dependencies in your req.txt should be
as pinned as possible. The classic example is Requests: when it changed the
API fairly significantly, umpteen installers broke... just because people did
not bother with specifying a version for that lib.

------
xiaq
A bit off-topic, but I like how pocoo.org uses different fonts for different
projects:

Flask: Georgia for text, Garamond for titles

Werkzeug: Lucida Grande for text, Ubuntu for titles

And now click: Ubuntu Mono for text, Open Sans for titles

~~~
gojomo
I had the same thought, and will almost certainly be cribbing this page's
body-text font stack...

 _font-family: 'Ubuntu Mono','Consolas','Menlo','Deja Vu Sans Mono','Bitstream
Vera Sans Mono';_

...(fulfilled by Menlo on my Mac) for a future project.

------
erlkonig
Why is it that (nearly) every description I read about some random new Python
command wrapper fails to get the "python" and ".py" out of the command
examples, even in Linux?

I don't blame this particular offering, since I don't think release was
actually planned just yet and any number of other projects have made the same
subtle mistake.

Command Name Extensions are Harmful. Don't expose such an implementation
detail in every example, lest everyone actually follow them. Use the
"#!/usr/bin/env python" or whatever at the top of your scripts. And yes, you
can keep the .py if what you have is a library, not just a command (but it's
nice to then make a wrapper the doesn't expose the implementation language).
And obviously in other OSes where the command extension can be omitted and
still work this isn't such a big deal.

But in Unix/Linux, commands should be reimplementable in a different language
without making some .(extension) a like, retained to keep from breaking other
things that depend on it. Just say no :-)

~~~
Tyr42
Did you see the last part on setuptools?

It actually shows you how to set it up so you don't even use a #!, but rely on
setuptools to make an executable for you, that'll work in a vitualenv or on
windows. And the script name doesn't have .py at the end in his example,
though he doesn't call that out specifically.

------
mahmoudimus
I should write up a comparison between:

\- Cement ([http://builtoncement.com](http://builtoncement.com))

\- Cliff
([http://cliff.readthedocs.org/en/latest/](http://cliff.readthedocs.org/en/latest/))

\- Plumbum ([http://plumbum.readthedocs.org](http://plumbum.readthedocs.org))

\- Argh
([https://pypi.python.org/pypi/argh/0.24.1](https://pypi.python.org/pypi/argh/0.24.1))

\- Aaargh
([https://github.com/wbolster/aaargh](https://github.com/wbolster/aaargh))

\- Baker
([https://pypi.python.org/pypi/Baker/](https://pypi.python.org/pypi/Baker/))

So many more to choose from. Now we get to evaluate Click. Seems like the
reason Armin wrote Click was to load options dynamically, but that's what
Cliff does via stevedore
([https://github.com/dreamhost/stevedore](https://github.com/dreamhost/stevedore)).

My favorite feature about Cliff though is:
[http://cliff.readthedocs.org/en/latest/complete.html](http://cliff.readthedocs.org/en/latest/complete.html)
which comes out of the box, but then again, there's Argcomplete
([https://github.com/kislyuk/argcomplete](https://github.com/kislyuk/argcomplete)).

EDIT: Updating from previous posters

\- Naked ([http://naked-py.com](http://naked-py.com))

\- Docopt ([http://docopt.org](http://docopt.org))

\- Clint
([https://github.com/kennethreitz/clint](https://github.com/kennethreitz/clint))

\- Argvard
([https://github.com/DasIch/argvard](https://github.com/DasIch/argvard))

\- Commandr
([https://github.com/tellapart/commandr](https://github.com/tellapart/commandr))

\- Argtools
([https://pypi.python.org/pypi/argtools/0.1.2](https://pypi.python.org/pypi/argtools/0.1.2))

\- Plac
([https://pypi.python.org/pypi/plac](https://pypi.python.org/pypi/plac))

~~~
lookACamel
Clime ([https://github.com/moskytw/clime](https://github.com/moskytw/clime))

------
lorenzfx
This looks pretty similar to aargvard [1] (which is, as stated in the README,
inspired by flask).

[1] [https://github.com/DasIch/argvard](https://github.com/DasIch/argvard)

------
eddd
[https://readthedocs.org/projects/argh/](https://readthedocs.org/projects/argh/)

~~~
dankilman
I also really like argh, though it seems the provided link documents an api
the mandates decorators and in fact what I like about argh is that functions
can remain clean.
[http://argh.readthedocs.org/en/latest/](http://argh.readthedocs.org/en/latest/)

~~~
icebraining
You're never forced to use decorators, they're just syntactic sugar.

Their example:

    
    
      @click.command()
      @click.option('--count', default=1, help='number of greetings')
      @click.option('--name', prompt='Your name',
                  help='the person to greet', required=True)
      def hello(count, name):
          for x in range(count):
              print('Hello %s!' % name)
    

Could be written as:

    
    
      def hello(count, name):
          for x in range(count):
              print('Hello %s!' % name
    
      hello = click.option('--name', prompt='Your name', 
                 help='the person to greet', required=True)(hello)
      hello = click.option('--count', default=1, help='number of greetings')(hello)
      hello = click.command(hello)

~~~
u124556
The command decorator is nice, but the group decorator feels wrong, why create
a function that does nothing just to decorate it? I'd rather use `cli =
click.Group()`.

------
thu
The biggest gripe I have with using Python for rich command-line tool is the
startup time. One of the first reason I like to write a nice command-line tool
when I start a project, say foo, is to be able to do `foo --help` to quickly
see and remember what the project can do (I have a very bad memory and doing
this makes it possible for me to jump back faster to a project, even well
documented. I can forget what I was doing in just a few days and so I add a
lot of small commands).

In short running `foo --help` should be instant and if it loads all its
modules to list the different sub-commands and their respective description it
is really too slow.

A possibility is to cache some information (e.g. generate a text file or a
small Python script).

~~~
Serow225
I'm completely clueless about this, but would it be possible to have a
persistent python interpreter always running in the background somehow, and
submit the commands to it?

~~~
zokier
You can keep ipython kernel running in background if you want. Quick
unscientific test showed that it didn't reduce ipython console startup time
significantly. But it should be possible to make a leaner console to connect
to the kernel.

edit: my test was flawed, with more accurate test there is almost a full
second time difference:

    
    
        $ echo -n | time ipython console --existing kernel-29793.json
        1.06user 0.06system 0:01.52elapsed 74%CPU (0avgtext+0avgdata 29968maxresident)k
        0inputs+64outputs (0major+19245minor)pagefaults 0swaps
        $ echo -n | time ipython console
        1.07user 0.08system 0:02.40elapsed 48%CPU (0avgtext+0avgdata 30264maxresident)k
        0inputs+72outputs (0major+20538minor)pagefaults 0swaps
    

of course two seconds is ridiculously slow either way

------
nubela
Ahh.. I really like pocoo's products. But I've found manage.py
([https://github.com/Birdback/manage.py](https://github.com/Birdback/manage.py))
to be far leaner and simple. What do you guys think?

~~~
anentropic
don't like that, clashes with Django

~~~
mborch
I think this is intentional.

Django's manage command is already extensible, so if you're using Django then
perhaps it's best to use its tool chain.

------
jobeirne
This doesn't look much different than argh, which has been around for ages:
[http://argh.readthedocs.org/en/latest/tutorial.html](http://argh.readthedocs.org/en/latest/tutorial.html)

------
buster
Looks nice but the example looks almost exactly like
[https://pypi.python.org/pypi/argtools/0.1.2](https://pypi.python.org/pypi/argtools/0.1.2)

------
gdw2
Reminds me of Commandr. Both use decorators.
[https://github.com/tellapart/commandr](https://github.com/tellapart/commandr)

------
moondowner
Little unfortunate naming. Isn't Click a trademark of ASF?
[http://click.apache.org/](http://click.apache.org/)

------
eeadc
Looks like a nice wrapper around argparse:
[https://docs.python.org/3/library/argparse.html](https://docs.python.org/3/library/argparse.html)

~~~
rjgray
It's a wrapper around optparse, as per
[http://click.pocoo.org/why/](http://click.pocoo.org/why/).

I'm curious about that decision given that optparse is deprecated. It says
it's because argparse doesn't allow nested commands, but I don't really see
why that feature is so desirable as to warrant using a deprecated module. I
probably need to have a play with it to find out.

Looks like a nice module.

~~~
the_mitsuhiko
Click can be extended lazily which helps execution time a lot when working
with many, many plugins. It also can be extended at runtime as per
configuraton.

argparse requires the parser to have full knowledge of everything which makes
it slow if you add many commands. Biggest problem though is that it's parsing
system is a bit broken when it comes to escaping. Options with arguments
cannot have values starting with dashes which is problematic for delegating
subcommands to other things.

For what I wrote Click for I could not find any alternatives that worked that
way besides optparse itself but that is hard to use.

~~~
sophacles
Can you provide an example of what you're saying? I've found that
argparse.ArgumentParser.parse_known_args() and sub-parsers can do everything I
can see click doing. Including lazy loading of plugins and the like.

~~~
the_mitsuhiko
Argparse has too magic of a design that you could fully nest it. I tried it
many times but you always run into it's limitations.

Links to some open bugs on it:

    
    
      * http://bugs.python.org/issue13966
      * http://bugs.python.org/issue14191

------
kylemaxwell
Trying to figure out how this compares to Naked ([http://naked-
py.com](http://naked-py.com)) in terms of goals.

------
waitingkuo
Time to rewrite some of my dirty command-line programs

------
Spiritus
[https://github.com/mitsuhiko/click](https://github.com/mitsuhiko/click)

------
frodopwns
user@uweb1:/$ pip install click Downloading/unpacking click Could not find any
downloads that satisfy the requirement click No distributions at all found for
click

------
_ZeD_
do you know plac[0]?

[0] [https://pypi.python.org/pypi/plac](https://pypi.python.org/pypi/plac)

------
pekk
There are like 20 libraries for this already and this looks very similar to
those. But Armin Ronacher, therefore it will be popular

------
gr3yh47
the name is a bit oxymoronic

------
tiziano88
is there anything like this for Go?

~~~
jebus989
Can't say I've tried it but the above-mentioned docopt has a Go
implementation:
[https://github.com/docopt/docopt.go](https://github.com/docopt/docopt.go)

