
Python Versions Used in Commercial Projects, 2016 Edition - markoa
https://semaphoreci.com/blog/2016/11/11/python-versions-used-in-commercial-projects-2016-edition.html
======
sixhobbits
Maybe unjustified, but I'm always kind of suspicious of a post which is
largely focused on raw data (like this one) and which presents only pie-chart
visualsations [0].

Yes, Python has a pretty dirty history, with many people choosing to stick to
the Python 2.7 that they knew and loved. And yes, commercial software tends to
move waaay slower than the wider community (many banks are still running
COBOL). If you're focused on making money and pleasing clients then "it worked
for us before" is always going to be the strongest argument.

Major players in the Python eco system have pledged to move away from Python 2
[1], and if we had non pie-chart visualisations, I'm sure we'd see huge trends
towards Python 3 in the last year. Even slow-moving corporations are starting
to use Python3. Yes, MacOS defaulting to Python2 is still a problem, but
Ubuntu switching to default Python3 is already a huge step to get companies to
move forwards.

[0] [http://www.businessinsider.com/pie-charts-are-the-
worst-2013...](http://www.businessinsider.com/pie-charts-are-the-worst-2013-6)

[1] [https://python3statement.github.io/](https://python3statement.github.io/)

~~~
kikoreis
Also, we (Ubuntu) definitely won't maintain or ship Python 2.7 indefinitely;
I'm keeping a keen eye on how the ecosystem moves to decide when that happens.

~~~
module0000
I was under the impression Ubuntu is re-shipping the upstream debian python2x
and 3x packages. Are your maintainers packaging 2x from scratch? If so...why?

~~~
kikoreis
Well, our maintainers (primarily Matthias) package both for Debian and Ubuntu,
so they diverge only in the sense that the cadences and versioning are not
completely aligned.

But regardless of the packaging effort, the real overhead is tracking security
for the [overlapping] LTS releases.

------
dr_zoidberg
Wouldn't call 28% "largely ignored", rather "still lagging behind". Thing is,
3.6 looks like the real deal breaker here, what Python 3.0 should've been.
With asyncio, compact dicts and scandir I've gotten to the point where I can
almost justify to my employer why, if everything is still working, we should
move to Python 3.6 ("all will be faster" \-- I know I'm kind oversimplifying
here, but my boss pretty much doesn't understand software and just wants
results/money).

There's also the sentiment of "I don't know if [insert module here] will be
supported", which has become mostly baseless fear, but people still think Py3
support is lacking (when it's not![0]).

[0] [https://python3wos.appspot.com/](https://python3wos.appspot.com/) \--
also, take notice how 9 of this guys come from here [1]

[1]
[http://mozbase.readthedocs.io/en/latest/](http://mozbase.readthedocs.io/en/latest/)

Edit: when I posted this comment, the link was titled "Python 3 largely
ignored" (not the article per se, but the submitted link had been titled that
way). It has been changed, but this was a bit of important context for my
comment.

~~~
Siecje
I'm still waiting for supervisor, beanstalkc, and cloudsigma.

~~~
AdamN
Not sure why supervisor matters, it's just a program. It's the same thing as
Homebrew running Ruby - totally doesn't matter what version they're using.

~~~
mi100hael
It does if you're using their API

------
scrollaway
Python 3 is omnipresent in my work right now and it is so nice not to have to
worry about, for example, unicode issues.

The only thing I'm dealing with which is still not Python 3 compatible is AWS
Lambda, which only offers 2.7 right now. Supposedly Amazon is working on 3.x
but... come on.

~~~
2T1Qka0rEiPr
Python 3 doesn't remove unicode issues, although granted you probably run into
fewer issues when defaulting to unicode over a bytestring.

~~~
sevensor
Python 3 doesn't remove _all_ unicode issues, it's not magic. But the pain is
95% gone compared with Python 2. I write Python for a living, and my life has
been a lot better since we switched.

------
noir_lord
[http://i.imgur.com/EJM1viF.png](http://i.imgur.com/EJM1viF.png)

That's why Python 2.7 will be around and maintain dominance for a while.

When the latest stable releases of the primary environments are defaulting to
something you have to have very good reasons why not to use the default (even
when it's easy to change the default).

Administrators (in the old school not the new "everything in containers")
don't like using non-defaults.

~~~
qznc
The default for Debian [0] and Ubuntu [1] is Python3. The `/usr/bin/python`
symlink will be Python2 forever, since Python itself advises it. PEP 394[2]:

> for the time being, all distributions should ensure that python refers to
> the same target as python2.

[0] [https://www.debian.org/doc/packaging-manuals/python-
policy/c...](https://www.debian.org/doc/packaging-manuals/python-policy/ch-
python3.html) [1]
[https://blueprints.launchpad.net/ubuntu/+spec/core-1311-pyth...](https://blueprints.launchpad.net/ubuntu/+spec/core-1311-python3-roadmap)
[2]
[https://www.python.org/dev/peps/pep-0394/](https://www.python.org/dev/peps/pep-0394/)

~~~
oneweekwonder
Am I incorrect in recalling that ArchLinux decided against this?

edit: seems to be the case[0]

[0]:
[https://wiki.archlinux.org/index.php/python#Python_2](https://wiki.archlinux.org/index.php/python#Python_2)

~~~
qznc
Read the next line in PEP 394 ;)

> however, end users should be aware that python refers to python3 on at least
> Arch Linux (that change is what prompted the creation of this PEP), so
> python should be used in the shebang line only for scripts that are source
> compatible with both Python 2 and 3.

------
wtbob
Former Python developer here. The problem was two-fold: Python 3 broke
backwards- and forward-compatibility (in a practical sense: there were
workarounds, but libraries didn't necessarily use 'em, and Python is nothing
if not a glue language); and for the longest time there really wasn't a good
reason to switch. I think it wasn't until last year that it really started to
make sense to go with Python 3 — and by then I'd already switched to Go.

~~~
blub
Are you using Go exclusively?

All the jobs I see are Ruby + Go or Python + Go, which makes it pointless to
learn Go if you don't care about Ruby or Python.

~~~
joobus
I'd guess a lot of the X + Go jobs are porting from X to Go.

~~~
psynapse
This appears to be a common story. We ported a bunch of Python 2.7 to Go. The
improvements in speed and memory consumption are outstanding.

 _But_ we _will_ be upgrading a large code-base from 2.7 to 3.x. For things
like web data aggregation and parsing, it is hard to beat Python; the tooling
in that space is really good.

Definitely looking forward to some unicode pain alleviation from the rewrite.

------
vasco
Users generally update by default, if thousands upon thousands didn't it's
because someone else fucked up.

I find it funny how a small group of people thinks they know better than the
outer community, to the point that they feel like they should have a say in
what thousands of businesses use to run successful code.

More than this, I would argue that most people using Python 3 are those new to
the language. This is only from personal experience, so it's really just
anecdote.

PS: In a kind of jesty side-note, I know the general argument is that "python
2 was broken", but really, how broken can something be when thousands of
businesses depend on it and more than that, choose to keep using it when a
"better" alternative comes about?

~~~
rgacote
My company has been writing a significant portion of their commercial code in
Python since version 1.5 and (almost) everything new in Python 3.5.

------
drej
Coming from a country with an extended alphabet, I've always been aware of
encoding issues, especially before almost everyone settled on using UTF. So
the treatment in Python 3 (and equally in Go) is a game changer for me and
many other language users out there. I'm not saying that, say, American
developers don't need to support other alphabets, just that it's less pressing
since they don't tend to deal with them on a daily basis.

So for this feature alone, I'm all in on Python 3.

------
TheAceOfHearts
I don't follow the Python community closely, so I might be totally off the
mark here, but my general impression is that it was shipped prematurely. From
what I've read, in more recent years a lot of work has gone into easing the
transition... But it seems a bit late. As an outside I wonder: did they fail
to put in a reasonable effort at all in providing a smooth transition from the
start, or did they try and failed to account to the realities of our world?

Recklessly breaking backwards compatibility without a smooth migration plan is
hostile to developers. Although it's not a programming language, this is
something that I think React has gotten right with their current versioning
scheme [0].

[0] [https://facebook.github.io/react/blog/2016/02/19/new-
version...](https://facebook.github.io/react/blog/2016/02/19/new-versioning-
scheme.html)

~~~
ubernostrum
Python 3 was shipped with the expectation that the migration would be long.
Python 3.0 was the proof-of-concept release, basically, and it wasn't really
until 3.3 that momentum started picking up around porting. 3.5 and soon 3.6
are beginning to offer large, attractive new features people want and can't
get on 2.7, which further incentivizes the move.

But in order to make it happen, and to provide the long lead time for
migration, 3.0 had to get out the door when it did. So it wasn't "premature".

------
est
Python really should learn from PHP7.

Yes, it breaks a lot of things, but it's totally worth it for the performance
gains.

Py3k solves problems partially like Unicode or async/await but it's a non-
issue for skilled python2 developers.

People like incentives to upgrade. Period.

~~~
eeZi
Lots of incentives for us:

\- clean Unicode support which prevents mistakes which used to bite us in
production

\- native async syntax, compatible with Tornado

\- asynchronous generators (yield from a coroutine!)

\- pathlib and os.scandir

\- type annotations

\- matrix multiplication

\- chained exceptions

\- faster dicts, ordered by default

So many useful features, and with the latest Python 3 releases porting has
become really easy thanks to improved compatibility with the old syntax.

~~~
icegreentea
They are all pluses, but most of those didn't exist till the last couple
years. So effectively the first 6 or so years of Python 3 existing had much
less incentive.

I think the flip around - ~30% adoption of a newest version at 1-2 years in -
especially one that breaks backwards compatibility isn't bad.

I've started numerous python projects at work over the last year all on 2.7. I
think we just started the last one. We're finally ready to jump to 3.

~~~
dr_zoidberg
Indeed, and ordered dicts come from Py 3.6, which is in beta now and expected
for December.

If async and this had been in Py3 from the start, they could've done a far
better argument saying "hey, we broke all this stuff, but you get asyinc I/O
and faster dicts" (a note of faster dicts: since many classes and language
mechanisms use dicts, it should have an encompassing effect on all of python
performance -- maybe not 30% or 40% faster, but a few % all around, which is
kind of what micro benchmarks are showing with 3.6 betas)

------
INTPenis
I expend considerable effort that I am not obliged to expend to use Python 3
whenever possible. I'm sure that I'm not unique.

Right now in my life the major things stopping me from using python 3 for all
the things are.

    
    
      * Ansible
      * Two large in-house developed apps taken over from previous devs
      * Debian Wheezy
    

And as you can guess, at least one of those will hopefully disappear soon.

Other than that I try to use python 3 as much as possible. I know a lot of the
workaround for getting pyvenv working on various distros for example.

I also know about a lot of alternative libs like ldap3, dnspython3 and more.

~~~
taneq
> I expend considerable effort that I am not obliged to expend to use Python 3
> whenever possible.

Isn't that kind of a bad sign, though? Should it really require 'considerable
effort' to use the latest release of a fairly widely adopted language?

~~~
INTPenis
It's a good sign in the sense that people want to move forward.

But yes, it's a bad sign in the sense that more effort is required than with
Python 2.

It's getting better all the time though. Most of the effort is because of a
local Mac OS development environment, legacy Debian Wheezy systems that are
hard to replace, but also legacy Ubuntu systems that are hard to replace.

------
rtpg
this headline is a bit much. 1/3rd of new projects use Python 3! A lot of
people are excited about the language now that a lot of the warts have been
dealt with and language compat is higher than ever.

Though all it takes is one dependency to throw a wrench in the plans, the
major projects are Py3 now.

Though porting large python 2 projects is still a huuuuuge pain. This is more
a result of python 2 badness than python 3. But a lot of work.

------
makecheck
I’m not sure why but the Mac has not installed a "/usr/bin/python3" so I
continue to avoid 3.x migration (aside from minor things like "__future__"
imports for division and print_function).

I don’t want to _add_ a dependency for end users when my current code works
fine with the built-in Python at 2.x. Nor do I want to bloat the size of
downloads to embed something like an entire interpreter binary. (Swift has the
same issue; there is currently no way to refer to it externally, as it is in
flux; I do not want to embed loads of Swift libraries in every build so I will
wait to migrate until they have stabilized binaries and made them available
from a common root.)

------
binarymax
I blame in part, that the default install for OS's like Ubuntu and Mac is
Python 2.7. If you are starting python development, and want to support more
systems, and you know the default is 2.7, then that's what you will target.

~~~
teekert
Python 3 is the default in 16.04 LTS.

~~~
2T1Qka0rEiPr
I just posted that my default Python is 2 and I'm on 16.04.1, but I suppose
the upgrade from 15.10 perhaps left the default as-is?

~~~
teekert
I think the fact that python points to python 2 and python3 points to python 3
does not mean one is default and the other is not. One would think so, yes,
but this is a bit of difficult case, for historic reasons. Both versions will
need to be present because many things depend on 2 still (ufw for one I
believe), python 2 was just there first and hence earned the badge "python".
To now switch to explicitly calling it python2 and calling python 3 python
would break many things probably.

On a clean 16.04 install, python 3 is present, python 2 is not, however, you
still need to call python 3 with "python3" [0]

[0]
[https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3](https://wiki.ubuntu.com/XenialXerus/ReleaseNotes#Python_3)

Edit: See qznc's remark on the pep, apparently it's even advised the keep
"python" pointing to python 2.

------
perlin
Anecdotally, we had a simple events API running Python 2.7/Flask/uwsgi/nginx
that needed to do quick I/O operations on a growing # of HTTP requests. To
increase throughout, we experimented with Python 3 for its async/await style
concurrency. It didn't seem to help much and we ended up just rewriting that
API in Go and see way better throughout-to-resource ratios by shipping 5mb
binaries to Docker & Kubernetes.

Point is, we're a hybrid shop that does all first pass services in Python 2.7
and then move them to Go when they become suffiently trafficked and/or
critical.

------
knlje
My colleagues at academia (scientific computing) mostly default to Python 2.
Personally I'll switch to Python 3 when approximately over 50% of my
colleagues use it or when some mind blowing new feature or library is
introduced only for Python 3. Makes things simpler this way. Also, I don't
currently have enough spare time to port all my code.

~~~
rglullis
Don't be so scared of porting your code. On my current work, we have a few
thousand lines of python (spanning different services and supporting
libraries) throughout in ~18 months and we were blocked by pika (a module for
communication with AMQP servers).

When pika got a python3 compatible version, we ran out of excuses to not use
python3, but still my colleague was worried that "it would take too long to
port things, and our-code-is-working-so-why-bother". I decided to try it
anyway, and it took me less than a lazy Friday afternoon to run 2to3 on all
the packages and get the tests to pass.

Given that your work is with scientific computing and (presumably?) you have
control over the deployment environments, I'd really recommend that you avoid
do the switch sooner than later.

~~~
dr_zoidberg
> Don't be so scared of porting your code.

I stuck with that line, because maybe his colleagues are also waiting for "50%
of colleagues using Py3" and it's just a deadlock until some few brave enough
starts to port and tip the balance.

------
ntoll
Python 3 is the default teaching language for the RaspberryPi. The BBC
micro:bit runs MicroPython - a reimplementation of Python 3 for
microcontrollers.

Teachers and kids are already ahead of the game.

When today's kids graduate they'll view Python2.7 in perhaps the same way I
view Delphi or VB6 ("you're still using _that_..?" etc,).

~~~
mavhc
And then Codecademy go and use Python 2, sigh

------
swalsh
I've recently started using python a lot more, it was a natural progression
when I moved from using a windows dev machine to a mac book. One day I needed
to do something quick, and python was the quickest way to do it. I typed
python into my terminal, saw 2.7, and built from that. I had no incentive to
look into upgrading, I had everything I needed to accomplish what I wanted. My
good experience then has grown to the point where it's probably the second
most common tool in my toolbag, but it's still an "I need to do xyz, and I
don't care how it looks I just want to see if this will work". So I still
stick with the defaults. I only think about using the latest greatest tools if
I think the code I'm writing has long term potential.

------
spdy
From my personal opinion just porting a project from 2->3 we will get there.
Python 3.6 offers to much to let is pass.

Ecosystem has moved far enough its just a matter of time when big frameworks
etc. deprecate python 2.7 and this mostly happen around the real EOL of it.

------
d_theorist
I'm actually surprised (pleasantly) to see that adoption is that high. I would
have guessed at less than 10%.

I suppose the sample set (projects using a tool like Semaphore) is likely to
be biased towards more forwards-looking teams, but still.

------
danielsamuels
Where I work we're just waiting on Ansible. Everything else we write and use
works with Python 3, except the tool we use to deploy it all. Frustrating.

~~~
randlet
Why not use both? I'm shipping python 3 software using Ansible and it works
great.

~~~
danielsamuels
Using version 2.2?

~~~
AdamN
Ansible is just a program - I'm not sure where the confusion lies. You can use
Chef (Ruby) to deploy Python apps after all ... the version of Python used by
Ansible is totally irrelevant to the interpreter you use for your codebase.

~~~
danielsamuels
As far as I remember, if your local virtual environment is Python 3 Ansible
doesn't work (might not even install).

~~~
neokya
well, you can install ansible outside local virtualenv. It's a single program
like Firefox, so probably you won't need different versions and could be
installed as such.

------
verytrivial
Interesting data-point, but what is the _trend_? We already know 2.7 is a
victim of its own success, right?

------
Siecje
What would a project that supports both Python2 and Python3 show up as in
these charts?

How many of the projects support both?

------
upofadown
As languages go, Python 3 is pretty successful. It unfortunately shares a name
with the language it is a variant of so people tend to compare the popularity
of the two. The popularity of Python 3 should be judged against all similar
languages, not just the insanely popular Python 2.

------
rlpb
Commercial projects are the laggards. I'd expect them to be the last to move
over. More interesting I think is the progress in distributions. This matters
because commercial projects use distributions too, and will need to follow
them in the end.

------
denfromufa
Sentry is also still Python 2.7, thanks to Armin and his friends:

[https://github.com/getsentry/sentry/issues/1152](https://github.com/getsentry/sentry/issues/1152)

~~~
chuamo
looks to me they have maybe changed their stance? See commits.
[https://github.com/getsentry/sentry/tree/master/src/sentry/m...](https://github.com/getsentry/sentry/tree/master/src/sentry/middleware)

------
pjc50
Let this (and Perl 6) be a lesson to language designers: _do not make
backward-incompatible changes, ever_. It cripples adoption. Java managed this
much better by being able to mark APIs as "deprecated".

~~~
bhaak
I actually don't know why it's been such a big pain with Python 3.

Ruby 1.9 also fixed the accurate handling of strings potentially encoded with
multiple bytes.

Fast forward to today, we didn't have such a turmoil like with Python and now,
not even Ruby 1.9 is supported by the official devteam anymore. And there is
nobody holding out on 1.8 anymore.

Compare that to Python 2 which will still be supported until 2020!

~~~
dagw
One big difference between Ruby and Python (and one of the big reasons that I
think python3 failed to take off) is that the Ruby community has been much
better at making sure that their major third party libraries are in sync with
the core language. I don't remember the exact timeline, but basically when
Ruby 1.9 came out you could more or less instantly start using it with RoR so
there was no reason to wait with experimenting with it.

When python 3.0 was released both django and numpy where still years away from
getting official python 3 support into their releases so no one bothered to
invest too much time into it, and this caused a major loss of momentum that is
only today starting to slowly return.

------
fdgdasfadsf
Python2 is stable. This is actually a major advantage in some applications.

------
cakeface
Is python's upgrade difficulties from 2.7 to 3.x due to being interpreted
rather than compiled? I feel like the types of problems seen in a non
compatible upgrade are exactly what I consider easy problems in Java. Look for
the compilation issues, fix them, done.

In Java I am largely talking about library upgrade issues because Java made
the decision to remain language backwards compatible. Something that has it's
costs but also some real benefits. See Linux and it's promises to never break
user space.

------
w8rbt
I think this shows just how good Python 2 is ;)

------
fermigier
How do they define "commercial" ? How do they define "new" ?

------
joobus
My office has moved new projects to Go. We were using twisted web server which
still isn't p3 compatible. It also doesn't help that Python is basically
single-threaded.

~~~
xapata
Stop spreading misinformation. Python is fine for multithreading despite the
GIL. It's simply that there's no free lunch -- no one strategy for parallelism
is best for all programs.

------
lmm
Flamebait title, and doesn't match the article. 30% adoption, while not
exactly a big success, is nothing to sniff at.

------
lovelearning
Title should be changed to "Python Versions Used in Commercial Projects, 2016
Edition". The data pertains only to Semaphore's commercial CI customers.

------
vuanotino
The only thing I haven't liked when comparing 2 to 3, is how 3 is even slower.

CPython is so slow, they really should start taking performance more
seriously.

~~~
majewsky
Taking a guess here: Maybe all the people who are concerned about performance
have moved to PyPy?

~~~
vuanotino
They don't support 3 :(

~~~
majewsky
What, PyPy? Their homepage states 3.3.5 compatibility:
[http://pypy.org/](http://pypy.org/)

~~~
xapata
And they're skipping 3.4 going straight to 3.5

------
someguydave
I'm glad that python 2 is unpopular with cool kids. Otherwise they might be
inserting broken feature every minor revision.

------
bdg
Of course it is. Python is full of language snobs who insist their language is
beautiful and the quality is great. They insist there's no reason to change,
except for some reason, to go.

