On: The Python Unicode Mess

[loeg]

How is that different from Python 2? You have to deal with it either way.

[ddevault → loeg]

Python 2 encourages the unaware to silently write broken, anglocentric code. Python 3 puts the problem in your face.

[loeg → ddevault]

I don't see how that is responsive to the question about using bytes/Py2 string for path names.

[ddevault → loeg]

Python 2 encourages the unaware to silently write broken, anglocentric code [which interacts with paths].

[loeg → ddevault]

Python 3 encourages the unaware to write broken code in a utf8-centric way instead. The unaware are going to write broken code.

[ddevault → loeg]

Not true. Compare these:

https://docs.python.org/2/library/os.html#os.listdir

https://docs.python.org/3.5/library/os.html#os.listdir

[loeg → ddevault]

What specific difference are you trying to highlight? The only distinction I see is that Python 3.3 adds support for file descriptors.

[joshuamorton → loeg]

Likely the existence and explicit mention of os.fsencode, as well as the api giving what you put in, so accepting bytes if you do things safely.

[loeg → joshuamorton]

The 2 API also returns the same type you put in.

os.fsencode() was changed in Python 3 longer after the 3.0 fork and isn't inherent to 3 -- the same change could be made to Python 2. It only wasn't because Python.org has been intentionally neglecting Python 2 since 2015.

(The Python 2 equivalent of os.fsencode() is just encoding with sys.getfilesystemencoding(). The primary difference is that fsencode() uses the surrogateescape error handler by default. Since this is a relaxed behavior, it seems like it could be added to Python 2 without regressing existing programs.)

[joshuamorton → loeg]

Yes, so the documentation and language explicitly have tools to avoid encoding issues and make you aware of them. It encourages writing good, unbroken, code.