
Raising severity to serious for some Python 2 leaf packages with no Python 3 - mkesper
https://lists.debian.org/debian-python/2019/10/msg00043.html
======
mjw1007
Historically, Debian hasn't particularly objected to packaging obsolete
versions of programming languages without upstream support.

I doubt anyone is checking for potential security problems in the Algol 68 and
Fortran 77 implementations that Debian ships, and I don't think the people
using those packages are particularly inconvenienced by that.

It seems a shame that the social pressure to persuade people to port their
code to Python 3 means that Debian is going to have weaker support for
10-year-old Python than 40-year-old Fortran.

In particular, there are ongoing efforts to try to make it the normal thing
for scientists to make the programs they ran on their data available so that
their results can be reproduced; aggressively dropping older programming
language implementations rather gets in the way of that.

~~~
DCKing
What? This isn't about "languages". It's about software!

Algol 68 and Fortran 77 may have stale (but maintained) compilers or
interpreters in the package repository. Starting very soon - Python 2 will
have an entire set of unmaintained runtime and libraries in the package
repository. You know - actual, offically, unmaintained software! Unmaintained
software that other packages, including Calibre in this example, further build
on. Of course they're throwing this out.

If you're concerned about language support there are multiple solutions like:

* Port your stuff to Python 3 and enjoy first class support

* Running Debian Buster

* Running CentOS 8, which will probably profit from RHEL 8's Python 2 support-with-an-asterisk until 2029 [1].

* Installing it manually on your system

* Installing it manually in a container

This is how it feels to run legacy software - it's bloody inconvenient. But
that's not a problem Debian needs to solve for anyone.

[1]: [https://developers.redhat.com/blog/2018/11/14/python-in-
rhel...](https://developers.redhat.com/blog/2018/11/14/python-in-rhel-8/)

~~~
CJefferson
Python 2 is only unmaintained because the Python foundation is refusing to let
anyone "officially" take over support. There are thousands of programs in
Debian with less core developers than Python 2 working just fine.

~~~
maxerickson
It's understandable that the PSF wouldn't endorse a maintainer.

They can't stop someone from releasing something that runs just like Python 2
and has a different name.

~~~
CJefferson
That's true, but it might make packaging tricky -- lots of the old Python 2
software people want to run will have '#!/usr/bin/env python' hard-wired in,
and having to fix those would be annoying.

~~~
takeda
Upgrading the OS is also annoying, so why not just stay in one place for next
5, 10, 50? years. If you don't upgrade your machine it probably will continue
to work, python 2 won't disappear.

I wish PSF would pull the plug from python 2 much sooner so we wouldn't have
to deal with whiners for 10 f...ing years. The problem with python 3 wasn't
the incompatibility, the problem was keeping python 2 alive for so long. Other
languages usually gave only 1 year and everyone moved, because they didn't
want to stay behind.

~~~
CJefferson
Which other languages?

My C89 code, C++98 code and Fortran 77 code all still compile on a modern
Ubuntu, and none of these are expected to break any time soon. The compilers
just support multiple versions.

~~~
joshuamorton
I'd strongly suggest watching "What is C++" [0] from this year's cppcon.
There's a lot of good content, but the relevant bit is that c especially
(remember, c++98 is younger than python2!) values backwards compatibility very
highly. This is sometimes useful, but it has costs. Performance costs, safety
costs, and especially ergonomic/complexity costs for future development (the
kitchen-sink-ness of modern cpp).

People aren't wrong to make different tradeoffs than you. Especially given
that strict backwards compatibility is impossible (especially in a language
like c/++, people will rely on things that aren't intentionally public,
because they can introspect your bytes).

Compiler changes break code all the time, it's just almost always semantic and
not syntactic. I find this amusing, since the syntactic changes are usually
easier to detect and fix.

And python2 still works, it just won't get new features.

[0]: [https://youtu.be/LJh5QCV4wDg](https://youtu.be/LJh5QCV4wDg)

~~~
hermitdev
> And python2 still works, it just won't get new features.

Probably more importantly, it wont get security fixes.

Yeah, Python 2 will still work. Ita not like there's a kill switch or
anything. It just wont receive any official or supported updates. Effectively,
Py2 will be abandonware.

------
strenholme
I have seen multiple people post to Ycombinator claiming that porting to
Python3 is always trivial and easily done. And, yeah, sure, for a 380-line
script written after Python3 came out, porting the code so it runs the same
both in Python2 and Python3 is trivial.

However, for an over 6000-line script written in the mid-2000s, before Python3
even existed, porting to Python3 is non trivial. Trust me, I tried.

First, the syntax sugar stuff was easy enough to fix: Do a simple search and
replace so that all cases of stuff like

    
    
        print "foo"
    

Become

    
    
        print("foo")
    

I had to do something similar with the way this script calls raise(). Not to
mention adding a hack so Python2 uses “xrange” while Python3 uses “range”.
After doing all of this, the code still does not run in Python3. As far as I
can tell, the issue is that Python3 does implicit float-to-int conversions
differently than Python2; the code will not raise an error in Python3, but it
will silently run differently, until the state is different enough that the
Python3 version raises an error while the Python2 version works as expected.

At this point, I just threw my hands in the air and gave up.

For reference, the offending code is here:

[https://github.com/samboy/misc-
civ4-mapscripts/blob/master/T...](https://github.com/samboy/misc-
civ4-mapscripts/blob/master/TotestraRG32.py)

~~~
CivBase
Devil's advocate:

Many people aren't even bothered to _try_. The company I work for still writes
new stuff in Python _2.7.3_ - not even a recent release of 2.7. Some of their
stuff will be difficult to convert, but a lot of it really is trivial. They
just don't care to put any effort into it until things start to break down.

I can understand why people get frustrated by situations like this and are
tempted to call out programs still running Python 2 and those who advocate for
keeping it.

I'm a huge fan of some of the features introduced by Python 3, so I switched
and am not interested in going back. However, I think there is room for other
interpreters to step in and continue supporting Python 2 even if the PSF wont.

~~~
Izkata
> Python _2.7.3_

Familiar number.

Maybe they tried, and stuff broke in weird and unexpected ways, so they gave
up. For example, the first time we upgraded from that to (I think) .8, lots of
strange errors started coming in from our webapp. Apparently kombu, one of the
dependencies of celery, was importing an underscore-function from the standard
library, which no longer existed in the new version.

~~~
CivBase
This is definitely the case for a few apps. It's why I raise hell anytime I
see someone use a private attribute or function from a builtin module.

However, I'm painfully aware that it does not apply to _most_ of our apps and
it does not justify using an outdated interpreter for writing _new_ apps.

~~~
strenholme
There are occasional cases where, with some Python libraries, I need to use a
“hidden” _function to do my job. The one time I had to do that, I made sure to
contact the authors to let them know what I did and why:

[https://github.com/cobrateam/splinter/issues/591](https://github.com/cobrateam/splinter/issues/591)

------
doubleunplussed
I'm running Calibre on Python 3 (via an Arch package provided by one of the
people working on the port). It and mercurial/tortoisehg and the only
remaining reasons Python 2 is on my system. All of these things will be on
Python 3 soon enough.

Why doesn't Debian just ship the unreleased Python 3 port of Calibre? It's not
like Debian restricts themselves to only formal upstream releases, they have
never hesitated to backport patches to unreleased versions. If lacking Python
3 support is such a massive bug, wouldn't using patches that have not been
officially released, that fix the bug, be acceptable? Debian testing isn't
Debian stable, there is not a reasonable expectation that the code shipped is
perfect.

Mercurial also works on Python 3, though it's only a beta. I could understand
not shipping it because version control bugs can be serious, but hey,
someone's gotta test the beta. I've used it and it not encountered any bugs,
but went back to py2 because tortoisehg won't work with Python 3 mercurial yet
(and then bitbucket dropped mercurial support so now I'm moving everything to
git, grumble grumble, probably will sadly stop using mercurial altogether
within the next year).

~~~
_delirium
It's definitely possible for Debian to package unofficial releases out of the
upstream VCS. You can sometimes tell that's what happened because it'll have
something like ~git20191005.f8ab33 in the Debian package's version number.

In this particular case, the Debian maintainer seems to prefer to wait for
upstream to release an official Python3-supporting version of Calibre though:
[https://github.com/norbusan/calibre-
debian/blob/master/debia...](https://github.com/norbusan/calibre-
debian/blob/master/debian/NEWS)

It's basically up to the maintainer either way. The freeze for the next Debian
release is at least months away (no date set yet), so it's not too urgent, and
waiting for upstream is still reasonable at this point.

------
ridaj
One can sense the frustration dripping from the thread. This feels like pretty
aggressive communication ("frankly, I don't care", etc.)... it could've been a
misunderstanding about what "removal from testing" means... In particular
since the packages remain available in unstable, the argument raised about
users paying the price isn't really valid.

~~~
ldng
I think you're overreading it. Wanting to remove python2 dependancy now with
the next Debian release as target (probably 2 year from now) seem reasonable
to me. The aggressivity you perceive is to me just being firm. Refuse and push
back any exceptions. The work has to be done, end of story.

Personnally, I approve, otherwise it'll spiral down in nitpicking and special
cases. Would not be the first time with Debian.

~~~
marcosdumay
There's this part:

> If the package has still many users (popcon >= 300), or is needed to build
> another package which cannot be removed, document that by adding the
> "py2keep" user tag

So they are not even that firm. Overall, it seems very well handled.

~~~
stOneskull
I am impressed by the leadership. It needs to happen and it needs a firm
leadership like this. No dilly-dallying. On with the job.

------
sethish
Can someone link the list of packages that needed to be ported? I get the
impression from this thread that newly ported packages won't be included in
the next testing release, but hey, there's always unstable.

~~~
deckar01
Python 2 Removal (All): [http://bugs.debian.org/cgi-
bin/pkgreport.cgi?tag=py2removal;...](http://bugs.debian.org/cgi-
bin/pkgreport.cgi?tag=py2removal;users=debian-python@lists.debian.org)

No Python 3 Port: [https://bugs.debian.org/cgi-
bin/pkgreport.cgi?tag=py3noport;...](https://bugs.debian.org/cgi-
bin/pkgreport.cgi?tag=py3noport;users=debian-python@lists.debian.org)

------
sprash
Has anybody interest in forking the python project and continuing to maintain
2.7 until forever? And by "maintain" I mean no new features, just bug fixes
and performance improvements. Maybe if we find enough volunteers we can pull
that off.

There is still much useful and valuable legacy software out there that many
people use which is getting abandoned for no other reason than some people
(like the authors of calibre) are not very fond of the direction python is
going. Others can't upgrade to 3.x because you have standard libraries
especially in the academic sector that people literally put their whole phd
into and that can not "easly" be upgraded just like that because of high
complexity, lack of staff and time.

~~~
carapace
I plan to do exactly that.

 _It 's vaporware at the moment,_ but uh:
[https://git.sr.ht/~sforman/Bladders](https://git.sr.ht/~sforman/Bladders)

> Maintenance-mode fork of the officially-deprecated Python 2 interpreter and
> standard library. Bugs fixed, security holes patched, and maybe some
> performance improvements, but no changes to the syntax or semantics (for
> that see the Tauthon project, unaffiliated.)

\- - -

Sometime between now and the end of the year I'm going to fork Py2 and set up
build jobs to run the test suites. (Maybe later today, it's not hard, it just
takes a few spare hours.)

Going forward I plan to work on things like improvements to the docs, curating
additional packages and libraries (additional to the standard lib), and
language tooling (I'd like to keep things like Nuitka, Snakefood, etc.
working.) I also want to run fuzzers and other code quality tools over it all.

~~~
sundarurfriend
Before it's too late, I'd like to humbly suggest a name change. Even as a
Blackadder fan, the name sounds terrible at a glance, especially for such a
(potentially) high profile project. Naming isn't everything, of course, but it
has more significance than we consciously realize. It can subconsciously
impact decisions in favour of or against the project, as irrational as that
is.

~~~
carapace
I appreciate the sentiment but it's deliberate. I don't want to become high
profile. The Python folks have made it pretty clear that they don't want
continuity of Python 2 to "dilute the brand" of Python-as-Python-3, so I
wanted to wave a big "Don't take this too seriously!" flag over this fork.
(Also for the BBC, eh? It would be a little on the nose to call it
"Blackadder". Or "Baldrick".)

------
philshem
This is turning out to be the py2k bug

------
Congeec
> No special pleading. No whining. No excuses. > No buts and no exceptions. No
> "middle ground".

This sounds strong to me. Not to be provocative since I am not familiar with
the mailing list, I would like to ask is it common to communicate this way in
this Debian community? Is it effective to push things moving forward?

~~~
_se
I think it's perfectly reasonable. Package maintainers are stomping their feet
and huffing about making a change that is in everyone's best interest. Why
shouldn't the distribution set strong guidelines and enforce them?

Too much is often sacrificed at the altar of backward compatibility. Move
forward and leave the laggards behind.

------
breatheoften
How did a project like Babel not evolve to compile python2 into (horrific
code) that runs with python2 compatible semantics on the python3 runtime ...?
Is that not possible for some reason?

~~~
_bxg1
There's this:
[https://docs.python.org/2/library/2to3.html](https://docs.python.org/2/library/2to3.html)

But the thing with Babel is that it only _adds_ features to JavaScript, it
doesn't translate between two _disjoint_ languages. I'd assume that has an
effect on its viability as a project.

~~~
nickm12
2to3 is such a small part of the solution. It only takes care of basic stuff,
but to really get things right you need to know the types.

------
olliej
This is why I continue to think that Python 3 being needlessly backwards
incompatible was one of the dumber things to happen.

Think of the lost productivity that has resulted.

People complain about JS all the time, but JS hasn’t broken existing code and
has improved.

JS supports ‘let’ declarations and variables named ‘let’, python could have
improved without breaking everything.

------
ridiculous_fish
gclient (necessary for building chromium, among other things) got an "early
version" working with Python3 just a few weeks ago. Apparently this stuff is
still being ported.

[https://groups.google.com/a/chromium.org/forum/#!topic/chrom...](https://groups.google.com/a/chromium.org/forum/#!topic/chromium-
dev/Azd9hNkav6c)

------
ahbyb
The last thing I expected to see in 'lists.debian.org' was emojis.

~~~
perch56
Do you mean the loupe magnifier? I wouldn’t consider it an emoji.

~~~
daxelrod
[https://emojipedia.org/right-pointing-magnifying-
glass/](https://emojipedia.org/right-pointing-magnifying-glass/) is part of
Unicode Emoji 1.0 [https://unicode.org/Public/emoji/1.0/emoji-
data.txt](https://unicode.org/Public/emoji/1.0/emoji-data.txt) as 1F50E and
it's represented on the page not via an image embed, but as a character.

Why wouldn't you consider it an emoji?

~~~
true_religion
It barely shows any emotion to put a magnifier down, maybe?

~~~
TazeTSchnitzel
The word “emoji” does not come from “emotion”, unlike “emoticon”.

~~~
tuukkah
Had to look it up as a reminder:

> _Originally meaning pictograph, the word emoji comes from Japanese e (絵,
> "picture") + moji (文字, "character"); the resemblance to the English words
> emotion and emoticon is purely coincidental._
> [https://en.wikipedia.org/wiki/Emoji](https://en.wikipedia.org/wiki/Emoji)

------
dooglius
Why is Debian removing support for Python 2 at all? It's not like there are
new releases they need to test against or updates they need to merge.

~~~
belorn
A package with no active developer gets removed unless someone takes up the
role of maintaining it.

Since python software foundation will stop developing it there is no upstream
support after 2020. In theory a new upstream for python 2 could pop up, and
there is speculation that Redhat might do it, but Debian do not want to become
the new upstream for python 2. If they included python 2 in next release they
would have to support it for as long as next release get support. What will
happen instead is that they will keep supporting python 2 in older releases
but when those support periods are over then its over.

~~~
wyldfire
> there is speculation that Redhat might do it

If they're going to do it, for the sake of the community they should announce
that soon. Even if they don't plan on taking up the responsibility until the
middle or end of 2020 it might be something that distros and companies would
consider when making their plans.

~~~
diminoten
They did and they have. I won't link to it because I strongly disagree with
their decision to do so, but if you care about this you can find it.

------
jancsika
Javascript: Hey you kids: quit doing cartwheels on the heads of 20 friends who
are also all doing cartwheels twenty levels deep. No cartwheels more than 3
levels deep or someone's going to get hurt! You've been told over and over
now.

Python: Hey you kids: sit in this cubicle for the entire summer and do 20,000
long division problems _just to keep the school 's lights on_. And don't get
too encouraged because there's no way in hell we can finish all the long
division before the beginning of the school year. Some of your classrooms will
be dark. p.s. our principal got burned out and quit.

~~~
333c
Can you explain the JavaScript comment? I don't follow that world.

~~~
sciurus
[https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/](https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/)

~~~
onei
I never actually read the whole story behind this. To me, it looks like the
original author of leftpad was in the right - radically changing a package can
break someone's build. Maybe he went a bit too far with his next action, but
if NPM don't care about its end-users enough to ensure packages remain
relatively stable (within the limited control they have), why should an
individual contributor?

