
Sh.py - daenz
http://amoffat.github.com/sh/index.html
======
russelldavis
For a similar library with a slightly different take, check out plumbum:

<http://plumbum.readthedocs.org/en/latest/index.html>

Here's the explanation on the differences:

"The project has been inspired by PBS of Andrew Moffat, and has borrowed some
of his ideas (namely treating programs like functions and the nice trick for
importing commands). However, I felt there was too much magic going on in PBS,
and that the syntax wasn’t what I had in mind when I came to write shell-like
programs. I contacted Andrew about these issues, but he wanted to keep PBS
this way. Other than that, the two libraries go in different directions, where
Plumbum attempts to provide a more wholesome approach."

~~~
mixmastamyk
Ooh, I like the piping syntax better. The module name is a bit long and hard
to remember though.

~~~
jdnier
Plumbum is the latin name for lead, periodic table symbol "Pb", perhaps
memorable enough considering Plumbum was inspired by an earlier project called
PBS.

~~~
mixmastamyk
Still doesn't roll of the keyboard, does it?

------
sartakdotorg
Perl has this too: <https://metacpan.org/module/Shell>

Written in 1994, by Larry himself!

~~~
mleonhard
The module documentation doesn't mention error handling. Perl has no
consistent exception system and this makes it very difficult to build reliable
programs in the language. The standard library has inconsistent error
handling. And CPAN modules like this usually have none at all.

------
Sidnicious
So, this feature:

[http://amoffat.github.com/sh/index.html#interactive-
callback...](http://amoffat.github.com/sh/index.html#interactive-callbacks)

Lets you replace `expect` with Python code pretty darn easily.

~~~
bryogenic
why not use pexpect, it has worked great for me in the past.

<http://www.noah.org/wiki/pexpect>

~~~
playhard
+1 for pexpect. it is very handy to automate user inputs

------
SoftwareMaven
I like PBS (now sh.py) for certain use cases. If I'm writing an actual shell
script, I think it is brilliant. It keeps the script focused on the task at
hand instead of Python's somewhat painful process communication.

On the other hand, if I have an application that needs to communicate with a
subprocess as a small piece of the whole, I'll use other methods that are less
"magical". It's not that I'm inherently against magic, but rather that, in
that use case, I generally want very explicit control over what is happening.

~~~
okal
Not sure I agree that this is magic. Syntactic sugar? Definitely. But it
doesn't obscure any of the underlying logic.

~~~
lmm
It causes "import" to behave in ways you wouldn't expect.

~~~
im3w1l
Could you elaborate on this point? What happens and what would you expect?

~~~
apl
This tool hijacks the import mechanism by directly writing to Python's look-up
tables. After all, when you do

    
    
      from sh import git
    

there's no module 'sh' invoked in the normal sense. Instead, the library
generates a wrapper for the shell command 'git' on the fly. While that kind of
monkey patching may be neat, it's also a bit brittle and a potential security
issue.

~~~
SilasX
You must consider Ruby on Rails to be pure hellspawn then. (I agree, but for
different reasons ...)

~~~
MBlume
Well, the Python and Ruby communities have different engineering cultures and
this is part of that. "Explicit is better than implicit" is one of the mantras
on the Python side.

<http://www.python.org/dev/peps/pep-0020/>

------
jon6
I spent a few minutes trying to figure out how to install the thing so for
anyone that is equally as lost

    
    
        $ pip install sh
    

or goto the github page <https://github.com/amoffat/sh>

~~~
pattern
To the parent and anyone else who hasn't experienced the wonders of `pip`,
check out the article "Tools of the Modern Python Hacker: Virtualenv, Fabric
and Pip" [1]. All three of these tools are invaluable for even the most
trivial of Python projects, and make developing in Python much more enjoyable.

[1]: [http://www.clemesha.org/blog/modern-python-hacker-tools-
virt...](http://www.clemesha.org/blog/modern-python-hacker-tools-virtualenv-
fabric-pip/)

~~~
bduerst
I just learned Python a week ago, and PIP was an important part of that.

For those of you who are just learning, PIP will automatically download,
install, and then compile any python modules you point it at. Just make sure
you have the module's recommended compilers installed first.

------
subhobroto
I have found this extremely useful - used it to write many things - from a set
of scripts that bootstrap chef server onto a node from scratch to a file
chunking program that optimizes log files to align with hadoop block sizes
using multiprocessing and this. It made a lot of things very easy.

This version introduces many positive changes: specially 'Iterating over
output' that I have been waiting for a long time.

Andrew wants to increase his support for MacOS and would like to have test
results from "python setup.py test" (to run the whole test suite). One
identified bug is: <http://bugs.python.org/issue15898>

I would love to see more people use this to simplify their work!

If anyone is interested in looking into the scripts I wrote to see what's
possible, let me know.

~~~
europa
Yes. Please publish it.

------
philp
Could somebody explain to me how this is different from envoy?
<https://github.com/kennethreitz/envoy>

Not meant as a snide remark; genuinely curious.

~~~
Aissen
Just look at how it's used. It's not the same. sh imports shell commands as
functions. envoy allows you to run shell commands _very_ easily, like perl's
``.

------
saikat
For anyone looking for a nice subprocess library for Ruby, my friend Greg
released one earlier this week - <https://github.com/gdb/rubysh>

~~~
b6fan
I write another one for ruby, check out samples here:
<https://github.com/quark-zju/easysh>

Although its functions are a subset of sh for python. It has many synatactic
sugars, makes full use of ruby Enumerable, and does lazy-execute.

------
arturadib
Definitely neat, but of course platform-dependent.

Due to the cross-platform needs of Mozilla's PDF.js build scripts, we've been
writing a Node.js lib on top of Node's APIs that enables you to write shell-
like scripts that run seamlessly on multiple platforms:

<http://shelljs.org>

Like Sh.py, you can (if you must) also run external commands, either
synchronously or asynchronously.

~~~
sciurus
This seems only tangentially related. The point of something like sh.py is to
ease the use of external commands you need to use. The functions that shelljs
partially implements (cd, pwd, ls, find, cp, rm, mv, mkdir, test, cat, sed,
grep, which, echo, exit, env) are already trivially accomplished in Python.

------
AntiRush
I wrote something similar on top of nodejs to simplify some problems at Game
Closure.

<http://www.github.com/gameclosure/jash>

It's probably not ready for prime time - past it's initial use cases it hasn't
been tested much. Things like sh.py and jash are a really neat solution for
some problems.

~~~
tiles
I was hoping for a Node.js alternative. One question, have you thought about
using <https://github.com/samshull/node-proxy> instead of searching through
the environment to find binaries? Using .get() you would only need to identify
programs that are requested, rather than knowing them all up front.

------
jlgreco
Rather tempting to use this with aa Python REPL to replace a more traditional
shell.

------
nvmc
I foresee a lot of frustrated users trying to google things about this
project.

------
nodesocket
Founder of Commando.io (<http://commando.io>) here. The tutorial on SSH was
particularly interesting, since we are doing some of the same sort of things
to help with orchestration of servers. Currently we are using `libssh2` via a
PHP module, but switching to a sparkling new node.js interface for the SSH and
SCP connections and executions shortly.

------
notatoad
This is beautiful. Thank you.

------
nimrody
Throwing exceptions when a command returns non-zero exit status is very useful
indeed. However, this isn't very different from using the shell's own &&
operator.

I still believe that wrapping shell commands with functions is the way to go.
Functions can intelligently check their arguments and prevent propagation of
dangerous (or otherwise obviously incorrect) arguments.

~~~
justincormack
Surely implementing the shell commands natively is the way to go?

This way you then have to parse the output of ifconfig, say.

eg I have been doing this for Lua [1] you can do

> i = nl.interfaces()

> print(i.lo)

lo Link encap:Local Loopback inet addr: 127.0.0.1/8 inet6 addr: ::1/128 UP
LOOPBACK RUNNING LOWER_UP MTU: 16436 RX packets:261454 errors:0 dropped:0 TX
packets:261454 errors:0 dropped:0

> print(i.eth0.macaddr)

f0:da:f0:38:36:39

The functionality is now reasonably comprehensive, so you can rename
interfaces, add addresses, although it is still a work in progress, as there
is a fair amount of work involved as there is a lot of shell to implement, but
it can be incrementally useful.

[1] <https://github.com/justincormack/ljsyscall>

------
bthomas
Do the python3 print statements imply anything about whether it's compatible
with python2?

And what's the best way to quickly look and see which versions a package is
compatible with?

~~~
daenz
It's confirmed compatible with python 2.6-3.2

------
ragmondo
I would rather have py.sh ... a unix shell running python.

------
ompemi
It saved my day, I expected partials with cwd parameter and they were there. I
used this instead of GitPython + manual popen for some git management tasks.

------
riffraff
for those wondering how "import madeupname" works, basically the incantation
is:

    
    
        # unless __name__ == "__main__" 
        self = sys.modules[__name__]
        # SelfWrapper has a custom __getattr__
        sys.modules[__name__] = SelfWrapper(self) 
    

which seems somewhat unpythonesque (aren't import hooks supposed to be used
for this?) but it's cool and I hadn't seen it before.

~~~
fijal
import hooks leave crap in your global state. this only modifies sys.modules,
which is probably a much better option.

------
arnarbi
What happens to the order of keyword arguments?

~~~
davidism
Good point. I guess if order matters you can do everything as positional args,
then order will be preserved.

------
OrdojanAndrius
Ohh this looks awesome, I wish it would work for windows thought.

------
scdoshi
This is cool. Could have used it yesterday, literally.

~~~
andybak
In that case you'll have to hurry.

------
SIULHT
Does this include scp/rsync?

~~~
andybak
from sh import anythingyoudamnwellplease

------
maskedinvader
this is awesome, thank you for posting

------
esschul
Hey, just like groovy's "".execute()

~~~
freehunter
Lua has a function called os.execute() where you can call programs and
utilities from the command line. Cross platform as well; I've got more than
one Lua script where it's working in the command line and can handle being on
Unix or on Windows.

A common example of this that I use quite often is "clear" or "cls". It makes
the determination what OS it's using, then issues the appropriate command to
clear the terminal window.

