
Show HN: Pythonpy – the swiss army knife of the command line - Russell91
https://github.com/Russell91/pythonpy
======
ctrijueque
Nice. Similar to PyP from Sony Imageworks:

[http://opensource.imageworks.com/?p=pyp](http://opensource.imageworks.com/?p=pyp)

The Pyed Piper Tutorial:
[http://www.youtube.com/watch?v=eWtVWF0JSJA](http://www.youtube.com/watch?v=eWtVWF0JSJA)

~~~
gfxmonk
I wrote a similar tool a while back:
[http://gfxmonk.net/dist/doc/piep/](http://gfxmonk.net/dist/doc/piep/)

Mostly out of frustration for PyP not being lazy (on large inputs it reads the
entire file up-front, or at least used to).

But it was quite interesting to implement all the standard python idioms (like
slicing) in a lazy way, and it's not that complicated a tool. I still use it a
lot whenever I have a nontrivial pipeline to write.

~~~
Russell91
Yea, fortunately pythonpy supports lazy iteration over sys.stdin when you
really need it. Just like in python, the syntax won't be as nice as using a
list. But it works:

    
    
      py 'itertools.count(1)' | py 'itertools.islice(stdin, 0, 10, 2)'
    

However, the number of times that you need this are surprisingly rare. Most
lazy operations don't require that each row be aware of the surrounding row
context, and using the much simpler:

    
    
      py -x 'new_row_from_old_row(x)'
    

will get the job done in a lazy fashion. Usually, when you need rows to be
context aware, as in:

    
    
      py -l 'sorted(l)'
    

or

    
    
      py -l 'set(l)'
    

it's just not possible to accomplish your task without reading in all of
stdin.

~~~
gfxmonk
Cool :), glad it's supported, at least for the simple case of line-wise
transforms.

Some things can't be done without reading everything. But there are still a
number of operations on "all of stdin" that can safely be done lazily. I'm
particularly fond of "divide stdin into chunks of lines separated by
<predicate>" [0]. Which does need context, but only enough to determine where
the current chunk ends (typically a few lines).

`py` seems to be aimed at a single expression per invocation (nice and
simple), while `piep` recreates pipelines internally (more complex but also
means pipelines can produce arbitrary objects rather than single-line
strings). So I'm not really sure how you'd do the above in `py` anyway.

[0]
[http://gfxmonk.net/dist/doc/piep/#piep.list.BaseList.divide](http://gfxmonk.net/dist/doc/piep/#piep.list.BaseList.divide)

------
k_os
You should put up a donation link. If more people do that and it becomes
common practice to donate to useful open source tools / projects then imagine
what more awesome stuff could be created. Many people desire to build tools
but don't have the financial support to do so.

~~~
thejosh
This is what gittip is for.

And Kickstarter/Indiegogo/Crowdfunding has shown genuine interest to help fund
useful tools and libraries lately which is great.

~~~
canistr
I think if Git Tip had a button on a GitHub repo (a la Heroku button), this
would be amazing.

~~~
whit537
How about the Gittip badges from [http://shields.io/](http://shields.io/)?

------
zokier
These sorts of things have been one of the strongholds of Perl, glad to see
pythonic alternative.

------
Qantourisc
ls | py -x '"mv %s %s.txt" % (x,x)' | sh has me worried, I can't find the
escape code directly. Also you might wish to offer the following api ls | py
-X '["mv","%s","%s.txt"] and run those using processes or sub-process. This
would ensure the right argument in the right place. Just an idea.

~~~
zokier
Of course for this case, using shutil (or something similar) would be
preferable. Eg ls | py -x "shutil.move(x, "%s.txt" % x)"

------
d0m
Looks so useful, can't believe I never thought of hacking something to do
that. Will experiment with it!

------
S4M
I like it! Now I can count the number of files I have in a directory:

    
    
        ls |py -l 'sum(1 for x in l)'

And also I can count the number of files whose names matche a certain
extension:

    
    
        ls |grep .py |py -l 'sum(1 for x in l)'
    

I am not saying it was not possible before, but I don't know how to do it in
bash.

Looking forward to use that for some basic stats on file. The following should
sum up the first column of a csv.

    
    
        cat data.csv| py -x 'x[0]' |py -l 'sum(x for x in l)'

~~~
mitchty
You'll probably want to learn the regular unix way as well. Tools like this
are nice, but you're already 99% there as it is for most things.

    
    
        ls | wc -l
    
        ls | grep -c .py
    

(note the cat and pipe are not needed)

    
    
        awk -F',' '{sum+=$1}END{print sum}' data.csv

~~~
S4M
Ah! I haven't thought of using wc, although I frequently use it to count the
number of lines in my code. For awk, yes, I have to learn it. Thanks to you
and the others who gave the example in bash.

~~~
mitchty
Technically this will work on any shell not just bash but good luck! Shell
scripting is rather fun once you get the hang of it. Though if I'm honest, if
I get a shell script about 30+ lines I tend to back up and break out
ruby/perl/python/anything but shell. Think of it as a succinct glue language
and you'll do fine.

In the meantime feel free to use this python thing, its rather fun to be
honest. Just wanted to demonstrate the unix-y way of doing it.

------
kseistrup
A simpler version of py is e ⌘
[https://pypi.python.org/pypi/e/1.4.5](https://pypi.python.org/pypi/e/1.4.5)

------
cohomologo
I attempted to install on Windows - it doesn't work due to the lack of
os.geteuid(), which is called in the setup.py file to check for root. With a
little bit of finagling I could get around that and make it install, but not
work due to undefined SIGPIPE in the signal module. It makes sense that this
could be UNIX only, but that wasn't stated anywhere on the webpage. Perhaps
that should be mentioned somewhere.

~~~
Russell91
Yea, sorry about that. I had an initiative to make a windows version, wpy, a
little after launch, but couldn't really get any windows developers interested
in making it happen. From my time working on windows, I know this tool would
be even more valuable there than on unix. I'll try to find a place to indicate
this doesn't work. Perhaps there's a good way to check if the system is
windows in the setup.py and warn users.

------
arno_v
For those too lazy to learn the right shell commands ;)

~~~
n0body
Or perl.

~~~
Demiurge
Or learned Perl and are too afraid...

~~~
n0body
obviously you learned bad perl, not modern perl

------
Lambdanaut
I usually use Python(or another lang's interpreter) as my calculator, so this
will save me a few keystrokes in the future. I think I'll still use short
scripts rather than the command line to do more involved things (like find
long palindromes in a txt file), but this will save me a bit of time when
doing quick calculations. Thanks!

------
Grue3
Would've been more useful with a language that doesn't limit one-liners (e.g.
you can't have "for x in [1,2,3]: dothing1(x); dothing2(x)").

------
leichtgewicht
Reminds me of 'js':
[https://www.npmjs.org/package/js](https://www.npmjs.org/package/js)

~~~
__debug__
I just started something similar for javascript a couple days ago:
[https://github.com/danielstjules/pjs](https://github.com/danielstjules/pjs)

------
mgr86
Honest question. Why would I want to use this? I am already quite happy with
awk,sed,grep, etc.

------
fsiefken
is there a ruby equivalent?

~~~
judofyr

        $ ruby -e 'puts (0...3).to_a' | ruby -ne 'p $_.to_i*7'
        $ ruby -e 'puts (0...8).to_a' | ruby -ne 'puts $_ if $_.to_i % 2 == 0'

