
Packaging Python software with pbr - pmoriarty
https://julien.danjou.info/blog/2017/packaging-python-with-pbr
======
daenney
I have never understood why people feel that pbr is better than just regular
setup.py. All it seems to do is move the exact same metata into setup.cfg
instead. Maybe it's a bit easier syntactically but to pull in a whole package
just for that seems a bit much. Especially since you don't find yourself
editing setup.py that often. You still need to learn about all the fields,
what they do and what you should set them to.

I'd much more recommend reading through [https://python-
packaging.readthedocs.io/en/latest/index.html](https://python-
packaging.readthedocs.io/en/latest/index.html) to actually understand how
packaging works.

That said... packaging stuff in Python could really do with being a lot
simpler. Pbr just doesn't seem to make it simpler, it just moves the problem
to a different file.

~~~
coredog64
At first glance, pbr appears to solve the problem of having to duplicate my
dependencies in both setup.py and requirements.txt.

~~~
joshuamorton
Why are you using a requirements.txt if you have a setup.py that lists them?
`pip install -e` or `setup.py install` should just work in that case.

~~~
rajathagasthya
What if you have your own package index? You can use `--index-url` in
requirements.txt to specify that.

~~~
joshuamorton
You can use `--index-url` when calling pip:
[https://pip.pypa.io/en/stable/reference/pip_wheel/#index-
url](https://pip.pypa.io/en/stable/reference/pip_wheel/#index-url)

You can also use `dependency_links` in your setup.py to specify this, which
allows deps on github etc.

------
Walkman
Please don't do this. It's just yet another tool which makes the same thing as
setup.py yet another way with no real benefit and it's not "standard".
Everyone who did some Python packaging already know at least the basics of
setup.py, now they all should learn yet another file type syntax for the same
problem.

~~~
atarian
> now they all should learn yet another file type syntax for the same problem

Are you really implying that learning INI would be a burden?

~~~
Walkman
Yes. Obviously it doesn't seem like so at the first glance, but when you have
a product like ours, where there are hundreds of small things like this, it
can really add up. I believe that everyone should use and improve the same,
standard tools so we can have few, but high quality things. When you come up
with something new which is just slightly different from the existing one (see
Python3 first versions...) I think you are doing the wrong thing.

------
hoov
I'm not a fan of this.

First of all, requirements.txt is for development requirements. Runtime ones
belong in setup.py.

Also, the "extras" feature is already in setup.py via extras_require.

I see no need to use this nonstandard tool when the standard tooling works.

~~~
yeukhon
I do this:

requirements/

    
    
        /base.txt
        /test.txt
        /docs.txt
    

Then in setup.py just read these requirements.txt. I just find this easier to
manage all the dependencies from a single point. During development and
testing code, you'd assume base.txt is what is going to production. It takes
some care to commit this file, nonetheless.

I could keep a freeze version if I really want to have a full view (for
debugging purpose).

~~~
matt_wulfeck
Not really. A package can be installed in both base and used in test, and vice
versa, so you have no real confidence removing the package doesn't break
anything. The best you get is explicit duplication.

~~~
yeukhon
Perhaps I didn't explain well. test.txt would only consist of packages such as
"pytest". The name means a specific function. If you have a platform package
that provides APIs for test classes, and APIs for other things like wrapper
around AWS apis, then my recommendation is make them distinct package, even if
they reside in the same repository. Just have a seperate setup.py. A package
in simple terms is just a folder, module is a single file.

------
hobarrera
> automatic generation of AUTHORS and ChangeLog files based on git history

I used to think this was a good idea. Then I found a huge loophole in it: if I
copy-paste code from another FLOSS project, then that code's author should
still be listed, but won't have any commits.

Also, peer-programming.

> version management based on git tags

setuptools-scm already does this for us.

Finally, a big upside of setup.py, is that I can programatically generate
information, whereas pbr's cfg file doesn't seem to allow that.

flavors are listed as a big deal, but much like the article says, setuptools
already has this, nothing new here.

~~~
shaded-enmity
> Finally, a big upside of setup.py, is that I can programatically generate
> information, whereas pbr's cfg file doesn't seem to allow that.

The problem with package manifests that are arbitrary scripts (Python, Ruby
and probably others) is that you lose even basic introspectability into
rudimentary parameters like name, version, dependencies unless your threat
model allows for executing random code that people put into the setup.py. This
can be mitigated by using complex static analysis carefully crafted for the
specific task, but this isn't easy to implement and there is a non-
insignificant amount of cases where the data cannot be legitimately decided.
I'm just wondering how this is conceptually different from Pipfiles:

[https://github.com/pypa/pipfile](https://github.com/pypa/pipfile) (Previous
discussion
[https://news.ycombinator.com/item?id=13011932](https://news.ycombinator.com/item?id=13011932))

~~~
ubernostrum
Also, executable package scripts with arbitrary code are the reason why 'pip
install rickroll' does exactly what it sounds like it'll do.

------
matt_wulfeck
To me this doesn't really solve the most painful part of packaging and
deploying Python: portability.

You still need to install system files for those c line. You still can't have
multiple packages requiring different versions of libraries. What a mess.

~~~
delsarto
For the first part, OpenStack fairly widely uses bindep (pbr is a pretty
OpenStack ecosystem thing too). It's not really perfect, but it is a fairly
standard way for a CI environment deploying from git to ensure the binary
dependencies. If a project doesn't have a bindep.txt file, in CI it falls back
to [2] for example.

I guess your second point means something like symbol versioning for Python
libraries, which I'm not aware of any solution to, apart from just running
things in a virtualenv.

[1]
[https://docs.openstack.org/infra/bindep/readme.html](https://docs.openstack.org/infra/bindep/readme.html)
[2] [https://github.com/openstack-infra/project-
config/blob/maste...](https://github.com/openstack-infra/project-
config/blob/master/jenkins/data/bindep-fallback.txt)

------
ocschwar
PBR? People still drink that stuff?

