
If I were designing Python's import from scratch - ingve
http://www.snarky.ca/if-i-were-designing-imort-from-scratch
======
the_mitsuhiko
I wrote about this many years ago and I think my conclusions back then are
still valid: [http://lucumr.pocoo.org/2009/7/24/singletons-and-their-
probl...](http://lucumr.pocoo.org/2009/7/24/singletons-and-their-problems-in-
python/)

Turns out other languages have evolved in the mean time and came up with much
better solutions. In particular with dynamic languages ES6 modules in
combination with npm's approach to package dependency management is the
current benchmark. ES6 exposes bindings instead of values which solves a ton
of issues that Python will never be able to fix with it's import system.

~~~
lukasm
I'd say Go is the benchmark in case of import system.

\- consistent syntax \- no circular dependencies \- you can only import a
module, only one symbol is polluting the namespace

I wonder what will be the winning solution for versioning.

npm and 260 char limit for path on Windows was driving me nuts.

~~~
TheDong
Go's import system has tons of problems.

1) inconsistent syntax: import "github.com/fsouza/go-dockerclient";
docker.NewClient...

Notice how you import something with the trailing path being "go-dockerclient"
but the actual package name is "docker"?

In npm, if you type x = require("foo"), you always refer to it as
'x.something'. You don't have to guess or read docs.

2) build flags. What the fuck? // +build fuckme

3) Canonical import paths. Double what the fuck? For those of you who don't
know, if you have "github.com/foo/bar" but write 'package bar // import
"github.com/google/bar"' then a user of your package will not be able to
compile it unless they move it to the directory you gave.

Yeah, no joke.... I'm in awe at this stupidity

4) Poor project-based importing (something actually worse than virtualenv;
nothing near as good as npm).

5) init side effects, e.g. look at how pprof works. Literally you do an
underscore import and it mutates the behaviour of the http package. Thanks.

~~~
Vendan
1) Yes, that is messed up, but that's just cause that package isn't following
standard idiom

2) Build flags are very useful, on the verge of required, for some situations.
For instance, code that needs seperate implementations for each os.

3) This is very useful for packages that may be on github, but served through
an alternate path. If they import the github version, you are now stuck with
github. If you move to self hosted git or something, their existing codebase
won't know how to do an update, and will require a rewrite to use the new
version.

4) I'll be the first to admit it isn't the greatest, but, it's rendered mostly
moot by the fact that you build single binaries. Deal with a slightly annoying
import system in exchange for effortless deployments and not having to worry
about libraries and such on the servers? Heck yeah!

5) I agree on this one, it should be something more like doing
pprof.Register(*ServeMux) in an init in your program.

------
aiiane
I totally agree with nixing *-imports and attribute imports. Not only do they
hide where something came from but they also tend to actively hinder
refactoring by making it harder to tell how packages are being used.

~~~
tgb
Nice for REPL work though.

~~~
jzwinck
You can use "import longmodulename as mn"\--many people do this today, and it
isn't nearly as harmful as "*".

~~~
zo1
Overall, I think many would agree that "import _" and "from x import _" are
more harmful than any of the alternatives. It's a laziness code-smell and will
come back to bite you.

------
scotty79
If I were designing import in JS I'd give option to inject dependency into
imported module that overrides the one that language would normally load from
file.

Like this:

    
    
        // model.js:
        var db = require("db");
    
    
        // controller.js:
        var model = require("model");
    
    
        // test.js:
        var model = require("model", { db: require("db_mock") });
    
    

I could probably implement something like that as webpack plugin.

------
stuaxo
The docs for importlib are a nightmare right now.

So many things are deprecated that it is hard to work out which things you
should use.

The deprecated bits should either be moved to the end of the page or to
another page.

------
EvanPlaice
Very interesting write-up, I'm glad to see a core developer thinking deeply on
this topic because Python has a lot of room for improvement.

This touches briefly on 1 of 3 major issues I have with python. I'll try my
best to articulate them from smallest-to-greatest impact.

 __1. _import_ semantics __

It would be amazing to see the _import_ module be abstracted out of the _sys_
module. Not just for the reasons mentioned in the article. Ideally, it would
allow developers to override/extend the implementation of _import_ and
experiment with new ways to handle.

For instance, why can't a developer write a module to experiment with using
es6-module style imports.

``` import {attr1, attr2} from module@[version]/submodule import * from
module2 ```

When I wrote the pypreprocessor lib, I simply wanted to add the ability to
import and conditionally-compile code using inline c-style preprocessor
directives. It works by blocking the import, preprocessing the code, then
importing the postprocessed version of the code.

Implementing it is ugly because there's no way to inline the preprocessor
step. If I had the ability to extend and inline the preprocessor step as a
custom import module, the specifics would be transparent to the user.

With a preprocessor it would be possible to write python2/python3 code side-
by-side making the transition much easier for library developers. Implementing
it as a custom import add-on would allow developers to extend the platform
without polluting core (ie since GvR is vehemently against adding preprocessor
capability to core).

Why does python default to the mentality of 'one true way'? There is a huge
ecosystem of developers willing to build/extend and experiment with the
language in ways that aren't immediately apparent to core design committee.

 __2\. Packages violates 'Explicit is better than implicit' __

Package management as a whole is a terrible, broken experience for library
developers and consumers of those libraries alike. Global-only package
installations, dependency hell, lack of version configuration, inability to
automate dependency loading, etc.

"Vendoring is not hard thanks to relative imports and most projects don't seem
to need it."

I highly disagree. Expecting users to either install dependencies globally or
manually copy them to the package contents sucks. The former introduces
potential environmental-level side effects such as dependency conflicts and
requires virtualenv to isolate packages. The latter is prone to error, adds
maintenance overhead as dependencies are updated, and makes it unclear to
developers who want to add dependencies without polluting source control.

Just look at the NPM ecosystem for inspiration. Vendoring, incl versioned
dependency management is done on a per-project level by default and the
benefits are clear. I can execute `npm install` from the CLI and the package
manager will download+install all of a projects dependencies (ie and
dependencies of dependencies) automagically. There's a reason the NPM eclipsed
every other packaging ecosystem in such a short period. It 'just works'.

Virtualenv is a hack. Instead of supporting package-level dependencies, it
forces developers to limit the environment to the package-level. It introduces
all of the same problems inherent to using globally installed dependencies,
except at a local level.

This violates II of 'The Twelve Factor App'
[http://12factor.net/dependencies](http://12factor.net/dependencies)

 __Pets vs Cattle __

While the current model works well with 'pets', ie long-running processes and
persistent environments. It causes a lot of problems when it comes to
'cattle', ie transient/disposable environments because it requires developers
to re-create an identical copy of the development environment on the target
for every deployment. Not only does this add a lot of provisioning
overhead/complexity but is highly prone to error if/when the environment
changes (ex modules added, configuration changes).

DevOps is hard enough, even without the overhead of cloning environment-
specific details of the language platform.

 __3\. Religious adherence to 'the one true way' __

Specifically, python is a platform implementation when it 'should' be a
language.

Don't get me wrong, Python is a significant improvement over the everything-
and-the-kitchen-sink approach used by platforms like .NET/Java. I understand
the need for including enough core functionality to get users up and running
quickly. What I don't agree with is the difficulty of extending/overriding the
default functionality.

Along those lines, I'd include a requirements: \- expose the core modules as
classes (ie not just _import_ ) that can be extended/overridden \- take some
time to engineer a really good package management tool \- setup.py shouldn't
require hand-coding, pip should be capable of generating it \- unify the
configuration at the package-level \- additional package install steps can be
added as pre/post hooks in the configuration

\-------

I have some obvious biases that come from the JS/Node.js development
community. Don't get me wrong, I love python as a language and will continue
to use it in the distant future. Without good package/version management I
don't see it as a good alternative for building anything but trivial
applications.

~~~
andybak
Good post but I call hyperbole on one of your closing remarks:

> Without good package/version management I don't see it as a good alternative
> for building anything but trivial applications.

Even if we agree Python has mediocre package management then it is still more
of an annoyance/hinderance than a factor that realistically rules out
"building anything but trivial applications".

~~~
EvanPlaice
You're right. Python is definitely useful for more than 'trivial
applications'. My statement was more a matter of subjective personal
preference.

The emphasis on package management isn't so much about building non-trivial
applications as it is about deploying non-trivial applications.

My ideal is a system that can be built in a composable manner using the
standard tools.

A deployment would include: \- cloning the source \- installing dependencies
\- setting env vars \- running tests \- launching the application

It should be trivial to repeat this process to deploy to multiple targets as
well as automate continuous integration testing.

With the current tool chain there's no clear path to accomplish this without
relying on a higher level abstraction like Docker.

------
n0us
not to quibble but the author spelled "import" as "imort" in the url slug.

