

Show HN: Vex, a new way to run things in Python's virtualenvs - sashahart
https://pypi.python.org/pypi/vex

======
sashahart
This is based on my personal experience with tools for using virtualenv.

virtualenvwrapper is very powerful with all kinds of hooks but I just never
used all that power and needed things like speed more. virtualenvwrapper
provided a lazy loader but that would have various problems from time to time
as you'll see on the issue tracker.

So then for a long time I just ran 'virtualenv' and source
wherever/bin/activate and found that I really didn't miss virtualenvwrapper
all that much, which was a surprise because everyone finds 'workon'
invaluable, a position I do understand.

Later my wife taught herself virtualenv and then wrote like 5 lines of shell
to do the same thing as virtualenvwrapper (the classic environment-modifying
thing) which I found hilarious, and I wrote a version of that in zsh with a
little more error checking to make me happy and just jammed it in my dotfiles.
Then after some months of using that I decided this is nice enough to publish
so I worked on rewriting it as portable shell functions with more error
checking and was going to put it on PyPI.

Toward the end of writing that I was feeling pretty finished, when some last
detail that I forget made me hacked off about modifying the current shell
environment, so on a lark I wrote a prototype of the new idea of only
modifying a spawned process's environment in about an hour (most of which was
learning further about portable shell) and once it was working I looked at
implementing more error checking for all the weird conditions that may arise
in shell and I said, you know what, I have no reason not to do this in Python
and totally cut the dependency on which shell is being used, who is going to
use this without Python anyway?

Then I ended up figuring out how to do prompts and completion and pretty much
had a publishable project, which was surprising.

It's still faster to directly run the virtualenv's Python or to use your own
small shell function but while YMMV, for my uses virtualenvwrapper is a lot of
code I will never use and I use vex every day.

Highlights of my todos: pydoc still doesn't work (the reasons for this are
lame) and probably it will be nice to implement --make/\--remove to substitute
mkvirtualenv/rmvirtualenv (which would also let me dump a pydoc script in
virtualenv's bin/ or Script\ dir to fix that problem...)

~~~
runiq
> everyone finds 'workon' invaluable, a position I do understand.

At first I wanted to praise virtualfish [1] and its ability to automatically
activate a virtualenv when I cd into it, but then I realized that vex's way
does indeed make more sense. What do we do in a virtualenv? We basically just
run pip to do various things, or run the program (in case it's an application
and not a library) or test suite.

So yeah, I think I'm gonna switch. The added portability is a nice bonus.

[1]
[https://github.com/adambrenecki/virtualfish](https://github.com/adambrenecki/virtualfish)

~~~
StavrosK
Am I the only one who activates the virtualenv by hand? I don't understand
what workon and other things provide. These days, I just have a single
virtualenv that gets sourced when my shell starts up, all my projects run in
virtual machines (I mostly work with Django). Projects that conflict with
others get their own virtualenv, but sourcing it once at the start of the day
is still not that much hassle.

~~~
sashahart
You aren't, I do it all the time. But it does get slightly tedious to type out
the whole path. (when switching virtualenvs frequently, unlike your use case)

What I use that is analogous to your single virtualenv sourced at shell
startup, is pip install --user. I tend to use that for command line tools like
style checkers.

------
schwuk
I've never bothered with virtualenvwrapper - I create virtualenvs alongside my
projects, and call `$virtualenv/bin/python` or `$virtualenv/bin/pip` etc.
directly. Sometimes I activate the virtualenv, sometimes I don't.

~~~
StavrosK
That's what I do too. I mostly activate them by hand nowadays, just to avoid
calling python. Fish is smart enough that it knows that, if I type a period in
a directory that contains a virtualenv, I want to activate it.

------
Deejahll
I'm very happy more people are coming around to this idea, despite multiple
independent discoveries. :)

Here's another project that attempts something similar:
[https://github.com/berdario/invewrapper](https://github.com/berdario/invewrapper)

Here's a sortof-blog-post and some more discussion about the superiority of
modifying a subshell rather than sourcing a bunch of janky shell-functions
that modify your current shell:
[https://gist.github.com/datagrok/2199506](https://gist.github.com/datagrok/2199506)

~~~
sashahart
I did not know about invewrapper, thanks. pew is another decent name.

edit: after looking at the doc, my impression is that invewrapper seems to
follow virtualenvwrapper's design more closely with a large number of
subcommands with names similar to virtualenvwrapper's, most of
virtualenvwrapper's options and features (except hooks and maybe some of the
project stuff). I would describe the options as "comprehensive." I
specifically wanted something with a dramatically simpler interface and
feature set.

invewrapper also seems primarily or only intended to run a shell. I get
personal use out of running arbitrary commands under vex as if it were sudo or
something.

It is an improvement not to modify the current environment and couple tightly
to specific shells either way, though.

One thing I do envy is the PowerShell prompt ;)

~~~
berdario
(invewrapper/pew author here)

I just discovered your project, so I haven't tried it yet, but since the
approach seems similar, it should work just fine on Windows (maybe with some
tweaks)... and you can just add the Powershell prompt to your docs, no need to
envy it :)

I agree that pew is intended as a virtualenvwrapper replacement, obviously you
can get something similar to `vex env cmd` with `pew in env cmd`...

Unfortunately I've neglected a little bit the project lately: the most serious
thing I want to do is rewrite the test suite, to be able to run it on windows.

Since you mention PowerShell maybe you use it quite often? I'd like to get
some feedback on Windows (I added Windows support out of completeness, but I
seldom use that OS), since I don't know anyone who uses pew on windows :)

BTW, I discovered some new tools to manage environments in the last year:

pyenv (I actually already knew rbenv), modules[1] and I even started to use
Nix

These have not the same scope, and in fact I appreciate the latter 2 idea of
using the same system to manage dependencies and versions for all different
kind of tools. I don't have an opinion yet on what's the correct way to do
these things so, in the meanwhile, even tools that are python-specific (like
ours) have a niche to fill

[1]
[https://archive.fosdem.org/2014/schedule/event/hpc_devroom_e...](https://archive.fosdem.org/2014/schedule/event/hpc_devroom_environment/)

------
acjohnson55
Ha. I always thought something was unnecessarily awkward about virtualenv, but
I couldn't quite put my finger on it. Virtualenvwrapper was an improvement in
many ways, but its lack of portability could be a pain. Reading the docs, Vex
makes perfect sense. We should use virtualenv basically the same way we sudo.
It's kind of amazing it took this long for someone to do it. Thanks!

~~~
ProblemFactory
Vex seems very convenient, but it is also possible to avoid the "magic" with
just standard virtualenv.

If you provide the full path to the python binary in the virtualenv, it
executes with the virtualenv loaded:

    
    
      /home/acjohnson/virtualenvs/foo/bin/python myscript.py
    

This is useful for crontabs, system service config, etc. which don't load a
regular user's bash setup files.

~~~
shuzchen
Similarly, anything you've installed in that virtualenv that installs itself
in a bin folder (pip, mercurial, django-admin.py, etc) will be in that bin
folder, and calling them from that directory will execute them in the context
of that virtualenv as well.

------
crdoconnor
I usually do this just by cd'ing to the project directory, which has the
virtualenv inside it and running './venv/bin/python'.

That eliminates the possibility that I'll accidentally run the wrong version
of python - e.g. a system python or a python from another environment that
didn't properly deactivate.

Most other issues are easy to deal with (e.g. non-existent virtualenv, being
in the wrong folder), but the issue of accidentally using the wrong
environment is _awful_ because it can manifest itself with subtle bugs or
weird exceptions rather than an obvious error.

~~~
e12e
I mostly do this, along with a little hack for certain commands I need, such
as mercurial: I have a virtualenv under ~/opt/venv/py-2.7-misc, and make a
symbolic link from ~/opt/venv/py-2.7-misc/bin to ~/opt/pybin -- and then my
.bashrc checks for a ~/pybin and adds it to the PATH if it exists ( [[ -d
"${HOME}/pybin" ]] ...).

So I can just go "pip install -u mercurial", or "pip install <some useful
utility>" without worrying too much. And (my) "hg" is in my PATH -- and I can
still run stuff in custom virtualenvs on a project basis, just as you say, by
specifying the full path (./proj/venv/bin/python ...).

I've never understand why people use anything more complicated.

I do see occasions where activate is useful -- certain packages, I think scipy
and pygame are among them -- are not only rather difficult to install via pip,
but also tend to be very picky about their environment. Normally I get by by
just installing those (and other packages with strong c/c++ library
dependencies) via the package manager though.

------
selectnull
This is cool, I like tools that focus on one thing.

Just like the author, I never used all virtualenvwrapper's features and was
always happy to create and delete VEs on my own. The only thing missing from
standard virtualenv workflow is easy way to activate the VE; so I wrote that
on my own and been using it since ever. It's trivially small shell script but
it does the work well, you can check it out at pypi:

[https://pypi.python.org/pypi/ave/](https://pypi.python.org/pypi/ave/)

~~~
sashahart
What is truly horrible is that I wrote my shell function, named it ave,
thought it was a great name and then months later (edit: after I released vex)
realized you had written a package named ave. Oh well.. :)

This is also worth checking out if you just want 'workon' type functionality
without the fuss.

~~~
selectnull
I guess I was faster picking ave :) but vex is good too.

------
isbadawi
I like this. I've used both virtualenvwrapper and pew before, but they both
have lots of features I don't use. Using this plus some small shell functions
like the below to create, list and delete virtualenvs is enough for me.

    
    
      function mkvirtualenv {
        virtualenv "$HOME/.virtualenvs/$1"
        if [ $# -eq 2 ]; then
          vex "$1" pip install -r "$2"
        fi
      }
      
      function lsvirtualenv {
        ls "$HOME/.virtualenvs"
      }
      
      function rmvirtualenv {
        rm -rf "$HOME/.virtualenvs/$1"
      }

~~~
sashahart
It looks like there is a lot of demand for --make/\--remove options to vex so
I will do those (and that should nearly approach the end of the new features I
will do on vex).

Probably good to do some error checking on the arguments to mkvirtualenv and
rmvirtualenv (sleepily hitting enter too early, etc.)

------
tudborg
Looks good.

I currently use
[http://tudb.org/articles/2014/03/31/vpython/](http://tudb.org/articles/2014/03/31/vpython/)
for pretty much all of my python projects (also, i am the author of vpython).

~~~
deathanatos
Well huh.

> Vpython is a tiny(-ish) bash script to help with your day to day virtualenv
> needs.

> * You don't have to worry about sourcing the activate script.

> * You don't have to point to your virtualenv path.

> just use vpython instead of python to invoke your scripts.

That's basically why I wrote vpython: I'm the author of _this_ vpython[1],
which isn't the same as yours, nor what the article is. (I suspect mine is a
little more immature, as I've not worked on it for as long.) I guess similar
needs beget similar utilities.

    
    
      [1]: https://github.com/thanatos/vpython

~~~
tudborg
Ha. How fun :)

We take different aproaches though.

Repo link for reference:
[https://github.com/tbug/vpython](https://github.com/tbug/vpython)

Mine expects you to know virtualenv and how it works, and also that you
explicitly install it. Same goes for dependencies.

My needs where to avoid sourcing the virtualenv, so i wrote the tool to fix
just that. Also, i built mine to resolve symlinks and be usable as the shebang
in your python scripts, which is pretty neat when writing command line tools.

------
neotrinity
one of the things I do in virtualenvwrapper is cdsitepackages to do some
patch/workaround on the installed libraries. Not a lot though but its a nifty
command to have.

------
mmelin
This looks great. Any way to have a system-wide .vexrc?

~~~
sashahart
Yes, there CAN be a way :) but would you do me a favor and write a spec for
how you want that to work and jam that in a github issue?
[https://github.com/sashahart/vex/issues](https://github.com/sashahart/vex/issues)

------
waitingkuo
Cool! That's the tool I always need!

