

Incremental Plans to Improve Python Packaging - pmoriarty
http://python-notes.curiousefficiency.org/en/latest/pep_ideas/core_packaging_api.html

======
tedivm
I'm going to say it, even though I'll probably get slapped by all of HN- PHP
beats the crap out of python for this exact problem. The php package manager
Composer ([https://getcomposer.org/](https://getcomposer.org/)) is an amazing
tool that I wish python would emulate. It uses a very simple json file for
metadata, an indexing site to point package names to actual git repositories,
and then git tags to handle versioning. The workflow is very simple and tends
to meld with actually development nicely. In the end all you have to do to
make a new release is tag it.

Okay, commence telling me I'm an idiot for thinking PHP has some ideas worth
looking at.

~~~
toyg
The problem is that Python libraries will often depend on code compiled in C /
C++ or something else, and inevitably something as simple as "just have a git
repo" just doesn't cut it (for someone somewhere with some specific workflow).

Package management is not rocket science, but it's very _political_ in terms
of deciding who should be heard and who should be let down by The One True
Blessed Way Of Packaging.

~~~
weaksauce
rubygems and bundler made me jump ship from python/django to the ruby and
rails ecosystem. the package management and isolation in that is top notch.
virtual env and pip/easy_install/whatever just don't match it for
repeatability.

I like python too but the packaging ecosystem is nowhere near where it should
be.

~~~
est
virtualenv beats the crap out of rvm though. If you ever tried ton use rvm in
crontabs, etc

~~~
tbrock
rbenv is much better and doesn't overwrite "cd" (yes the builtin command). I
thought everyone moved away from rvm years ago?

------
chhantyal
Looks like this is quite old post, head over to
[https://packaging.python.org/en/latest/](https://packaging.python.org/en/latest/)
for updated docs about Python packaging, especially 'Tools Recommendations'
section.

~~~
jokamoto
Link needs a trailing slash:
[https://packaging.python.org/en/latest/](https://packaging.python.org/en/latest/)

~~~
chhantyal
Opps. Thanks corrected now.

------
tbrock
Yup, python's packaging is even awful for the users. Half of the time pip
cannot determine what version of a package to install for a given version of
python on its own. "pip install --upgrade"? Gross.

However, you know what else needs improvement? PyPi. PyPi is downright
embarrassing compared to rubygems, npm, etc... and not only needs a facelift
but a whole redesign. This is the reason that I absolutely love to use python
but dread using a python library that I haven't heard of 1st hand. Treading
outside the standard library is an exercise in sifting through garbage.

Want to see which package for a "x" is most popular? No problem for any
dynamic language besides python. However, search for "http" on pypi and you'll
get a list of packages by weight which does not take into account popularity
or download count or views, just the score of a textual match.

Why is that important? Well when you search for "http" you get these as pypi's
top choices (no help at all):

    
    
        http               0.02
        CydraGitHTTP       0.1
        django-http-status 0.2
    

So what do we have here? A beta library (version 0.02) called "http" ranks 11
(off the charts) while kenneth reitz's insanely popular "requests", which
should be mandatory as an http client library, is nowhere to be found.

Pypi isn't even serving up the latest version of the python packages it has
some of the time (see: SCons).

All of this leads me to think the Pythonistas have given up on PyPi which does
more to hurt the community than you can imagine: It leaves a bad taste in the
mouths of those looking for quality libraries and software when they reach for
Python as their tool of choice.

~~~
talideon
PyPI is due to be replaced with something better shortly.

------
japhyr
I sure appreciate pip being included in Python 3.4. It's nice when writing
tutorials not to have to provide instructions for how to install pip, so the
user can then install packages.

~~~
makmanalp
It's annoying though, because distros like ubuntu pigheadedly decide to break
functionality by removing ensurepip from the core 3.4 package because it
violates debian policies put into place 20 years ago. As a result, there's no
straightforward ubuntu-kosher way to get a python 3 virtualenv with pip
working properly, and the users suffer for it.

edit: relevant bug and argument:
[https://bugs.launchpad.net/ubuntu/+source/python3.4/+bug/129...](https://bugs.launchpad.net/ubuntu/+source/python3.4/+bug/1290847)

------
rguldener
I recently installed numpy and scipy in a new virtualenv on my OS X 10.10
machine and was very pleasantly surprised to see them being installed directly
from wheels instead of being rebuilt (installation time was a nice 1min
compared to 10+ min before). Does anybody know if the wheel format is now
supported/standard on PyPi or was this caching of my local pip?

~~~
toyg
I understand wheels has been blessed as the way forward, so I would expect
PyPI to support it (where developers make it available).

------
Animats
At least "easy_install" and the rotten "eggs" format seem to have been
deprecated. Those tools had a failure rate above 50%. There were implicit
assumptions about where things went, and if anything went wrong, there was
neither a workaround nor a useful error message.

------
tdicola
Nice, I've been doing a bit with packaging python modules and have found the
current lay of the land incredibly confusing. This article helps quite a bit
and seems to be the most authoritative guide I've seen on the subject.

~~~
detaro
As linked above, this is the current go-to ressource for packaging:
[https://packaging.python.org/en/latest/](https://packaging.python.org/en/latest/)

------
docjigowatts
This post is quite old as others have noted. For those interested in the plans
for secure package and metadata distribution, a proposal is available now:

[http://legacy.python.org/dev/peps/pep-0458/](http://legacy.python.org/dev/peps/pep-0458/)
[http://legacy.python.org/dev/peps/pep-0480/](http://legacy.python.org/dev/peps/pep-0480/)

------
sametmax
This text seems quite old. Now :

\- wheels are more common, and popular libraries (scipy, cryptography) have
packages built at least for windows. On linux it's not a big deal since we
have package managers. \- setuptools is now the only stuff we need again,
since all has been merged in it.

It's still not very clear, but it's much better than before.

