
A Letter to /r/python - pknerd
http://journal.kennethreitz.org/entry/r-python
======
legostormtroopr
Somehow pipenv got endorsed as an official tool by PyPA far before it was
ready for production. The documentation for pipenv and Python packaging didn’t
make it clear early on pipenv was for applications, not libraries.

Also the pipenv maintainers were very rude about closing bugs. This has lead
to a lot of reasonable criticism about the tool and how it’s maintained.

Despite marketing the tool heavily, Reitz is taking all of this personally and
is blaming others or his mental issues.

Now, Reitz has posted this 3 times on /r/python in the past 2 days, and hosted
a hangout while also saying he doesn’t have time for criticism.

And he and his PyPA buddies are all posting in all the threads and making
passive-aggressive tweets.

To top all this off Reitz has now made and mods /r/positivepython in an
attempt to imply the original is somehow not “positive”. And in my opinion to
exert more power over the python community.

Before all this I really respected him, but this childish out burst will
really make me think about how well design or endorsed his work is in the
future.

~~~
eesmith
You write "The documentation for pipenv and Python packaging didn’t make it
clear early on pipenv was for applications, not libraries."

The linked-to essay says: "There is a misnomer that Pipenv is not useful for
libraries. That is false. Try using $ pipenv install -e . and see what it does
when you lock. It augments setup.py. I, as a library developer, find that
extremely useful."

I can't help but conclude that the documentation doesn't make that clear
because the author doesn't think it's true.

~~~
pseudalopex
Reitz is talking about development; everyone else is talking about packaging
and distribution.

~~~
eesmith
I confess that I don't understand the issues. Then again, I only last month
migrated my tools from 'distutils' to 'setuptools', so I know I'm way behind
the times.

I have just now read [https://docs.pipenv.org/advanced/#pipfile-vs-
setuppy](https://docs.pipenv.org/advanced/#pipfile-vs-setuppy) on the topic
and am still confused by the terminology. Eg, "Applications are different in
that they usually are not depended on by other projects."

I am used to writing command-line applications which are used as dependencies
for other projects.

But that is neither here nor there - no need to answer. Thanks for pointing
out the nuance I did not recognize.

------
kendallpark
I did a quick browse through the threads in question and suggest others do the
same. Yes, there is negativity, but the actual debate is more complex than
simply one grumpy side slamming a poor developer.

The most relevant comment I found was by /u/bixmix:

> It's getting much worse - not better. The internal churn within pip also is
> not community driven as much as it is developer driven. It's like we didn't
> learn from the debacle of python3 vs 2.

I'm not a Pythonist so I had no idea there was such a huge debate over how to
manage deps. At various points the past few years I've googled "bundler for
python" and come up with different projects. I remember trying pipenv early on
(didn't work with conda Python install). Then I revisited it months later--the
project was much more developed--and it worked great. FWIW I like pipenv, it
solves my largest gripe with Python development.

If you have as part of your community philosophy "there should be one—and
preferably only one—obvious way to do it," it seems like you're going get a
lot of stubborn tug-of-war situations (see 2 vs 3) where each side thinks
their way is the superior way and there is less room to compromise for the
sake of keeping the community together. That said, this pattern of debate is
pretty universal across communities.

Ruby--even with it's MINASWAN motto--had it's own RVM vs rebenv drama back in
2011. Same sort of developer/community vs developer/community drama. Nowadays
I think rbenv has a larger following.

[http://www.rubyinside.com/rbenv-a-simple-new-ruby-version-
ma...](http://www.rubyinside.com/rbenv-a-simple-new-ruby-version-management-
tool-5302.html)

To contrast this drama, look at what the NPM team had to say about Facebook's
yarn: [https://blog.npmjs.org/post/151660845210/hello-
yarn](https://blog.npmjs.org/post/151660845210/hello-yarn)

~~~
jitl
As a fellow rubyist in python land, I’m also a fan of Pipenv because it uses a
model I understand very well - that of Bundler (and similar to that of Cargo
and Yarn). I do have some technical issues with it, like the strange hiding of
the virtualenv in a random-like location instead of .venv by default, but
overall it’s a large improvement over
requirements.txt/setup.py/frozen_requirements.txt and even the new toml thing.

Maybe there is some inside baseball going on here when it comes to python
foundation etc, but as an outsider looking in the vitriol seems totally
unnecessary. I wouldn’t develop Python software w/o Pipenv.

------
sverige
I don't use python so I'm not involved in the controversy either way, but it's
pretty encouraging to me that this person is open about the limitations of his
humanity and its effects on his (apparently very popular) project. He
acknowledges it and has taken steps to limit its impact on the project.

We'll probably never know many projects have withered on the vine because of
similar issues. Kudos to him for his open letter and willingness to deal with
the negativity in a meaningful way. (It always kills me when people attack
others for providing _free fucking software._ )

------
mychael
People are rightly turned off that the Python Software Foundation (which
Kenneth is a core member of) declared PipEnv the official package manager[0].
It reeks of favoritism, insider politics and the cult of personality
surrounding the tool's author. It would have been better if PipEnv proved
itself as a better tool in the open marketplace of ideas.

[0] - [https://packaging.python.org/tutorials/managing-
dependencies...](https://packaging.python.org/tutorials/managing-
dependencies/)

~~~
gkya
Python the community is full of dogma. That is why I don't like the language
and I stay away from the community. Their own ideas come before objective
truth to them.

~~~
orf
You don't like the language because of dogma you perceive to be in the
community?

~~~
gkya
Not only, of course. BTW I have consciously avoided saying I disliked it.

------
mlthoughts2018
My two cents is that conda and conda-forge is the “right” way forward, and
attention focused on pipenv is not suboptimal because of pipenv (though you
can make engineering arguments against it— after all, it is still _very_ young
and has not yet matured the way other packaging tools have).

Rather it’s bad because it’s yet another set of spec files and environment
tools to learn, potentially further fracturing the community and taking
attention away from other approaches.

To be fair, it’s not really a problem with pipenv, but rather with any
“official” community endorsement of pipenv. Saying it’s a third party,
optional tool and you’re free to have at it if it’s good for your project is
fine. But the minute there’s any degree of officialness to it, it just creates
yet another fissure in the anarchy of Python packaging.

My reaction is more malaise. Guess I’ll have to learn about this new packaging
gadget too, to be current and signal how with-it I am. But in any case I have
a choice, I’ll still just use conda and Docker for application & environment
management, supplemented by pip and virtualenv when appropriate.

~~~
kalefranz
Current conda dev lead here. Just want to reference an issue we'll be tackling
ideally on the timeline of the next 12 months.
[https://github.com/conda/conda/issues/7248](https://github.com/conda/conda/issues/7248)
Conda is not without its problems in the workflow that pipenv addresses. We
know we have a deficiency here, and addressing that deficiency is one of our
enumerated highest priorities.

~~~
mlthoughts2018
Thanks for your work! I agree these are valuable design features to add and to
get right, and also agree that pipenv offers tools for this workflow.

From my perspective, I’ve tended to prefer Docker for this. This allows me to
version control environment components like environment variables in the form
of the Dockerfile. I usually have a Makefile for my project which then uses
multistage builds or other techniques to automatically detect when
environment.yml (or package changes, like to setup.py) take place, requiring a
rebuild of the container, so that downstream things like running tests,
dropping into an interactive debugger in a full environment (with the right
python environment, environment variables, etc.), all just become simple Make
directives.

It doesn’t cover everything. Notably, for situations when there are many
target environments, like testing a library I intend to distribute cross-
platform, across Python 2/3, etc., I might still use a tool like appveyor
rather than suffering the multiplicity of a bunch of Dockerfiles and Make
commands to target multiple environments.

But for most applications I develop with an intention of targeting a small set
of production or dev environments, it turns out to be better IMO to separate
the Python environment & package definition entirely from the larger runtime
environment (ENV vars, etc.), and use Docker for that part, with Make as the
bridge.

Combined with making a library choose sensible defaults for ENV vars,
copiously documenting them, and providing examples, this has worked well
enough that I do not need or want to mix the external environment
configuration with the app’s config files or package spec, in a “separation of
concerns” sense.

Your mileage may vary, but this pattern has really worked well for me, and
makes me skeptical of solutions that approach combining environment
configuration with packaging as a first-class feature, rather than as a
special case.

------
StavrosK
I saw the first of the posts he mentions, and yes, I was very surprised to see
the negativity. There is literally no reason to be negative, if you don't like
the tool, use another. People seem to be upset that Pipenv is the
recommendation now, but it's better than pip, and it _should_ be the
recommendation until something better comes along.

I'm more upset about the whole pyproject.toml thing that means I have to
migrate everything _again_ , from Pipfile this time, even though I have barely
finished migrating _to_ it!

~~~
ubernostrum
pyproject.toml isn't another way to specify dependencies for Python code.

The long version of why it exists:

Historically, a Python package has been defined by the existence of the
setup.py file. Executing this file with various command-line arguments is how
packaging tasks are carried out: 'setup.py install' installs the package,
'setup.py sdist' builds a source distribution (.tar.gz) suitable for uploading
to a package repository, etc., etc.

But this workflow has dependencies! Most packages will only depend on
setuptools (which supplies the setup() function) and wheel (which supports
building binary packages), so the packaging tooling assumes the presence of
setuptools and wheel and you never have to specify them as explicit
dependencies (in the old _old_ days, setuptools used a drive-by installer file
bundled into the package so people who didn't have setuptools could still
install setuptools-based packages; nowadays, pip supplies it automatically).

But what if you have a workflow that requires something else? What if you use
an alternative package manager, or package build tool, or really _anything_
that isn't setuptools or wheel? How do you specify the toolchain you need to
work with your package?

That is the thing pyproject.toml solves. Since it's not an executable Python
file, it can't get stuck in a chicken-and-egg problem ("execute setup.py to
find out what you need to execute setup.py"), and it can specify the necessary
build system for a package in a standardized way.

~~~
StavrosK
Yes, I understand the purpose, but the result is that it replaces any previous
way of specifying dependencies. It's great that it exists, but it's
frustrating that it was developed right when the new thing for specifying
dependencies (Pipfile) was starting to become popular.

~~~
ubernostrum
If you think it's another "thing for specifying dependencies" in the category
of the pipenv Pipfile, you have not understood the purpose of pyproject.toml.

~~~
StavrosK
If you think that's what I think, you haven't understood my comment. Again,
I'm saying that I don't want to migrate to yet another format so soon, even if
it does much more than just specify dependencies.

~~~
ubernostrum
Suppose you are developing something which needs to make HTTP requests. So you
have a dependency on the requests library.

 _pyproject.toml is not the place where you declare that dependency_. This is
why I am pushing back on you, because your comments all very very very
strongly imply that you think pyproject.toml is a place where you would
declare that dependency.

pyproject.toml is where you'd declare that you're using an alternate
build/packaging toolchain. You would then use whatever mechanism that
toolchain provides for specifying your dependencies (in the case of the
default setuptools build/packaging toolchain: install_requires in setup.py).

~~~
StavrosK
Okay, I see what you mean now. I was confused because Poetry declares
dependencies in the pyproject.toml, but looking at the actual file, it does it
in a Poetry-specific section, so it's not compatible with any of the other
tools:

[https://github.com/sdispater/poetry/blob/master/pyproject.to...](https://github.com/sdispater/poetry/blob/master/pyproject.toml)

~~~
ubernostrum
Alternative packaging toolchains _can_ use pyproject.toml that way; it's an
extensible file.

But you've caught on to the key thing, which is that neither they, nor the
default setuptools-based toolchain, _have_ to do that. The pyproject.toml file
exists to solve the chicken-and-egg problem of specifying that you use an
alternative toolchain, when figuring that out might otherwise be dependent on
your preferred toolchain already being present and configured.

------
deedubaya
Gah, why would anyone release software into the public domain if this is the
type of thanks they get.

I mean, I have no idea if he's right, wrong, or sideways here, but what is
clear is that he's released open source software and people are hating on him
for it. It's free! You don't have to use it, do you? What gives anyone the
right to throw hate towards an open source author?

~~~
adventured
Whenever you feel that way, tell yourself that 5% of humanity is fundamentally
somewhere between horrible and really horrible, capable of doing bad things to
their fellow humans without a moment's remorse. Do that while not forgetting
that the other 95% isn't like that. You're mostly (not exclusively) going to
hear from that 5% when it comes to criticism or hostile feedback, they're loud
and they're mean, they often lust to harm others (the outward projection of
their own internal issues). You'll far more rarely hear from the kinder 95%;
I'd suspect that effect is in play here, as it so often is.

------
jedwhite
This is Kenneth's presentation about Pipenv at PyCon last week which is well
worth watching:

[https://www.youtube.com/watch?v=GBQAKldqgZs](https://www.youtube.com/watch?v=GBQAKldqgZs)

------
nceheil
My wife is bipolar type 2 and many people on my father's side of my family are
bipolar tyle 1 so I partially understand your struggles. Few people truely
understand what bipolarity does to a person and their family and I just wanted
you to know as someone who does:

Keep doing what you have got to do to make it through the day!

------
pydumpsterfire
Python as a language is incredibly productive as someone who has used it for
almost 20 years. The secret to being happy and productive using the language
is to completely ignore the noise though: PyCon, blogs, reddit, "blessed
frameworks", BDFL, etc.

This will make you as happy as reading the New York times every 30 minutes to
find out what Trump is up to, or being on Facebook or Instagram and making
yourself envious of other people's lives.

Literally NOTHING matters in Python other than things on a yearly cycle. Don't
worry be happy, and just unsubscribe to all things Python that happen at the
Micro level and your appreciation of the language will increase.

~~~
apple4ever
This has been my path. I love Python, but I ignore all this other stuff. I
have better things to worry about.

------
bmans94
I guess I’ll never know what drives people to talk about the creators and
maintainers of the tools they use in such a negative way. Especially given how
little they actually know about them! It also strikes me that the moderators
of r/python could be doing more to curb the personal attacks.

------
wyuenho
WRT to not being able to use pipenv for multiple python versions in a project
- i.e. a library, it's been a couple of years since I've writing some
substantial python libraries, but do people not use tox anymore? Why can't
people stick to the usual virtualenv + pip + tox routine?

------
SpecialistEMT
Its silly where he defends the cult of personality and explains it with his
marketing. Like there can be a good cult of personality.

------
Swissindo
it's a good solution if for your understanding

------
qop
There is such an air of entitlement with open source these days.

I guess more and more developers don't remember when all the coolest shit
WASN'T free. You had to pay for compilers, editors, manuals, etc.

If someone writes a piece of software that you invest into your pipeline,
understand that without some sort of contractual agreement, that person isn't
OBLIGATED to maintain that code, much less give a shit about your feedback.

Is Ken smug?

Yes.

Is this post full of insufferable little whiny points?

Yes.

But is Ken going to have to do what the python community wants wrt pipenv?

Hell no.

Granted, a lot of this conflict could've been done differently and with better
tact, but the gritty fact of the situation is that no matter what happens,
it's not his fault for giving away the work. Even if the whole world depended
on a module, it's still a voluntary action.

~~~
gkya
I am not familiar with this particular situation, but while baseless
entitlement is bad, criticism is good, and when you publish something with the
intention that others use it---and reading his post it seems to me that what
he does it not open sourcing a tool he made for himself, he wants people to
use pipenv---you are no more the sole owner of it. More so if other people put
effort (i.e. patches and bug reports and proposals) in to it. The OP says:

> My response to this comment is — first off, my fame, while certainly
> categorized under “cult of personality” is not necessarily accidental. It’s
> called marketing. I worked very hard at becoming well known within the
> Python community, and toiled away at it for years.

after quoting

>> If only these people didn't advertise themselves as the next best thing to
sliced bread, collecting undeserved fame, jobs, influence, and as consequence
harming very large number of people and technology in general, I wouldn't mind
them doing whatever projects they like.

(w/o attribution, unfortunately), which shows that, if pipenv is a sub-par but
popular solution (I did not ever use it), this latter quote may be a
completely valid criticism.

I am diverting from my point: if Ken _wants_ and _encourages_ the python
community to use pipenv, then he might have to obey that community, and he
better does. But if he's releasing pipenv just to give back to the community
and share his solution to his problems with others, then you're right. After
all, if the participants have no regards for each other, that's a bunch of
selfish people, not a community.

~~~
empath75
I don’t think there’s any such thing as a ‘python community’ and the vast
majority of people that use python don’t know about any of this political
bullshit.

------
robobro
Here's a radical idea, and please read it slowly:

\---------------------------------------

whatever dialog you're trying to start around

a blog post where you're talking to /r/python,

by sharing it here on ycombinator News,

is BETTER DONE by a text post on /r/python.

\---------------------------------------

In fact someone (you? don't care) already shared your blog to r/python instead
of posting your entry as a text post. so why are you sharing it here? r/Python
is already having dialog on r/Python based on the blog post where a guy talks
to r/Python.

this is really bad Rediquette, dude. not upvoting your transparent attempt at
generating SEO points

~~~
noobermin
I don't know if pknerd is the author. EDIT: In fact he isn't.

------
lsh
tl;dr: Ken is upset and amazed that disrupting the workflows and toolchains of
entrenched Python developers is causing a backlash, and that negative things
are being said about him.

The post is self-aggrandising and he has the audacity to think he knows what
is best for everybody. There is also the real chance this tool will replace
other tried and true tools and workflows with a more complicated one. It
smacks of systemd vs upstart vs sysv

~~~
rthomas6
But... You don't have to use it? How does pipenv disrupt anyone's workflow?
It's optional.

------
iamthirsty
> I’m Kenneth — a nice guy who builds nice tools and takes a lot of pictures.

 _Written with literally no knowledge of this before:_

This killed it for me.

------
anothergoogler
This guy is so full of himself.

~~~
issaria
TL;DR I'm famous and contributed to the Python community, I'm depressed (it's
real! Check this references) because people don't like me forcing them to use
my tool. these people must be awful.

------
devnonymous
Just FYI, I posted this y'day, tho didn't invoke much of a discussion -
[https://news.ycombinator.com/item?id=17105738](https://news.ycombinator.com/item?id=17105738)

------
moocowtruck
kind of thought it was going to be a letter like 'why the hell you guys still
using python' a letdown indeed

------
cat199
'I consider myself to have been mildly underpaid, considering what I was
responsible for — the entire deployment experience (including uptime) of
Python apps on Heroku, one of the largest deployment platforms in the world.'

So, you're saying you don't have enough career experience that this is
surprising to you?

Megacorps are outsourcing their core processing systems to 3rd level
subcontractors, and these are the systems that literally run the economy
itself, and have been doing so for nearly 2 decades now if not longer.

Is this a just or fair situation? no (in my view) But hardly new or
surprising.. and certainly not relevant to the point you're trying to make.

