I use this for my Python-based Lisp: https://github.com/shawwn/pymen/blob/ml/importer.py
It's so cursed. I love it.
"from stackoverflow import quick_sort will go through the search results of [python] quick sort looking for the largest code block that doesn’t syntax error in the highest voted answer from the highest voted question and return it as a module. If that answer doesn’t have any valid python
code, it checks the next highest voted answer for code blocks."
I once implemented a custom importer as part of a system where the Python interpreter never touched the filesystem.
But, I did make an interactive tutorial here: https://docs.ycombinator.lol/
If you have any questions about it, I'd be happy to answer. This stuff is pure fun mixed with a shot of professionalism.
For what it's worth, as someone with narcolepsy, I relate quite a lot to your chronic pain. (https://twitter.com/theshawwn/status/1392213804684038150) For me, it mostly translated into wandering aimlessly from job to job, since I thought no one would have me. I hope that you find your way -- there's nothing wrong at all with taking it slow and spending years on something that takes others a few months. Everyone is different, and it's all about the fun.
It evaluates to -500000, as you’d expect.
(Just kidding, it’s -50000. Amusingly, the https://docs.ycombinator.lol version gets it right, since it has to; every expression is actually evaluated in the browser.)
This sounds very much like the good kind of magic, though.
> that you create
As in existing language supplied eg. Perl, Java, etc., or literally anything? Like bootstrapping your own home made language from scratch?
It starts with reader.l: https://github.com/shawwn/pymen/blob/ml/reader.l where the raw character stream is turned into a bunch of nested arrays. E.g. (+ 1 2) becomes ["+", 1, 2]
Then it's punted over to compiler.l https://github.com/shawwn/pymen/blob/ml/compiler.l where it's passed through `expand`, which does a `macroexpand` followed by a `lower`. E.g. (do (do (do (print 'hi)))) becomes ["print", "\"hi\""]
Then the final thing is thrown to the compile function, which spits out print("hi") -- the final valid Python code that gets passed into the standard python `exec` function.
Works with all the standard python things, like async functions and `with` contexts. Been screwing around with it for a few years.
Should work with anything as long as you can ultimately generate Python bytecode (and provide a module object). The import system is not simple, but it's really open.
Edit: A must have for any prolific dynamic language. But now I’m not sure that’s true, because even though it apparently works in Python, it’s certainly not widely used. In NodeJS this feature is used quite heavily for typescript, coffeescript (etcetera) interop.
I mean, it is used—even in the standard library—but often for alternative packaging (e.g., loading python modules in zip files) rather than alternative languages. It may be used less prominently than in Node, but it definitely is used for a variety of things.
That doesn't sound cursed to me, just flexible.
In the first .com wave we used a similar mechanism, just with TCL instead of Python, to ship our scripts encrypted.
Only our modified loader, written in C, could handle them.
$ touch foo.l
>>> import foo
ModuleNotFoundError: No module named 'foo'
git clone https://github.com/shawwn/pymen -b ml
You need nodejs to be installed too, ha.
Number one is that relative imports are weird. My intuition about imports is good enough that I never bothered to learn all the rules explicitly, but sometimes something simple is just not possible and it bites me. I think the case is importing files relative to a script (and not running with python -m ...).
Number two is, in order to do package management, you have to create a fake python installation and bend PYTHONPATH. Virtualenvs are the canonical way to do it, but to me it feels like a hack - the core language seems to wants all packages installed in /usr. So now I have all these virtualenvs lying around and they are detached from the scripts.
Why couldn't the import system resolve versions, too? You could say `import foo >= 1.0` and it would download it to some global cache, and the import the correct versions from the cache.
Coming from JS, that was a pretty frustrating realization.
It does not though, unless you have altered the default sys.path to always contains `.`.
When running a file / script, Python will add that file's directory as the first lookup path, so these two invocations should have the exact same sys.path, and thus the same module resolution throughout.
If you have added `.` to sys.path (which is not the default), then the first invocation will also have the "grand-parent" folder on the sys.path, while the second won't.
This also doesn't seem to have anything to do with relative imports, you need to already be inside a package for relative imports to do anything.
 that is not the case of `-c` and `-m`, both add CWD to the path, and they differ in what they do: `-c` adds `''` to sys.path, which is the in-process CWD. `-m` stores the actual value of CWD at invocation, so changes to CWD while the program runs don't influence module resolution
Assuming `foo.bar` is a script inside the package `foo` that you want to both import and run directly (without `python -m ...`), this lets you do so without too much hassle.
IMO that's also one of the main issues of Bash – you can't modularize a script unless you make sure your working directory is the directory containing the script. (And good luck with finding out the latter! )
I want to organize my code logically in directories. As a script grows, I want the ability to spin out parts of that file to separate files.
In order to do that in python, I need separate directories between the script and the spun-out functionality. This ends with a script that says "do function from module" and all code being in the module.
Having code in different directories for no reason except "the import system" sucks. How is this supposed to go?
I used to include a simple stub Python program (~10 lines) as a "script" in my setup.py that would import the right code and call it.
Then I learned that "entry_points" implemented most of the dispatch behavior I wanted.
I no longer have those scripts, just an entry that basically says "from abc.cli.prog1 import main; main()". The prog1.py, prog2.py, etc. look like normal command-line scripts, assuming the usual:
if __name__ == "__main__":
Because unless I'm missing something, the changes to the import system make this more reliable, not less: in Python 2, implicitly relative imports means `import x` might depend on the file's location. In Python 3, it does not, it depends solely on the setup of the sys.path.
from . import foo
from .foo import bar
from foo import bar
Exactly that: in Python 3, if an import doesn’t start with a . it will always be absolute aka looked up from sys.path.
In python 2, it would try performing a relative import (so importing a sibling module) first.
There's also ~/.local/lib/python3/site-packages (or whatever your distribution made of that). Virtualenvs are only necessary if you want to isolate dependencies between environments. That's useful if you have projects with conflicting dependencies, because Python doesn't allow you to install multiple versions of the same package, for better or worse. However, if you've written some simple scripts that don't care much about the exact version of their dependencies, it's perfectly fine to install their dependencies glboally.
I wrote a Python script yesterday which calls out to a couple of external commands (`mlr` and `xq`), with a shebang like this:
#!nix-shell -i python3 -p python3 -p miller -p yq
Say you have a main package "mylib" with a subpackage "mylib.utils". Typically I like to see "mylib.utils" imports as being in one of (roughly) 4 categories:
- standard imports, that I would put first in the file (e.g. "import logging")
- external imports, that I would put second in the file (e.g. "import requests")
- library local imports, bits and pieces from the "mylib" package that you want to reuse in "mylib.utils", but are external to the current package (e.g. "from mylib.email import client")
- and package local imports, which I see as implementation details of the current subpackage, and should be agnostic from the overall architecture of "mylib" (e.g. "from .helpers import help_function")
The last category are modules that only make sense from within "mylib.utils", should relocate with it even if it is renamed or moved elsewhere, and shouldn't require a change whatever the structure of "mylib" becomes, which is why I would use "mylib.utils" relative imports in there.
This is not supported in python. For reasons beyond my understanding, you are supposed to put the script with python, or with the shebang, in a different directory.
Alternatively, you can always use `python -m` to run your code.
Come again? This is what "import" does, obviously. This is not a clear explanation of your problem.
I say script here because the apparent intention is that scripts and modules are separate. That is why it's not easy to import functions from a file in the same directory as a script.
This restriction is very unexpected. And it is not borne out well by the fact that it is not obvious this distinction exists.
When you run a module, the root package containing it is on the path by definition and imports work.
Does that still work if you call the script from a different directory (so `python bar/foo.py`)?
Imma guess it doesn't work with symlinked scripts. But I don't think it should without doing an install, in which case a module does make sense.
I don't think Python will resolve symlinks here, so if that needs to work, you probably need to do it yourself before touching other code / files.
If the package uses absolute paths then I would need to tweak all the imports for the new absolute path.
Also the way that JS imports are just relative paths is very nice because it means that the imports are statically determinable, your editor can understand them and fix them automatically and you can trust that refactoring. Python has turing complete imports because there's so much dynamic messing about with sys.path that goes on in Python due to inadequacies of the import system.
Yes, just like absolute imports...
> relative paths is very nice because it means that the imports are statically determinable, your editor can understand them and fix them automatically and you can trust that refactoring
1) Just like absolute imports?
2) I absolutely contest that relative imports are easier for an IDE to refactor. I've never had VSCode hang while refactoring a Java package; I've had VSCode hang while refactoring "create-react-app" app...
env PYTHONPATH="foo/bar:$PYTHONPATH" python ...
Yes in Java it's fine because Java has a build stage. So tooling can figure out where imports point to from your static configuration.
However from the tool's perspective there is nothing better than a relative path. Relative paths require no messing about with configuration files at all to resolve. It's just a path to another file or directory on disk.
When a tool sees
import c from "./a/b/c"
So what is the advantage of absolute imports really? If import lines are mainly written and maintained by tooling then shouldn't we pick the representation that is easiest for the tooling? Then we can have more and better tools and the tools will be more reliable.
And it turns out that, relative paths are easy for humans to understand too. The same configuration-free resolution algorithm also works in your head when you are reading code! At least when the language doesn't overcomplicate them too much (JS is guilty of this to a certain extent, although nowhere as bad as Python)
Well whatever the language or runtime, you need to tell it how to find its dependencies. Be it with a configuration file, environment variables or parameters.
Your statement is not exactly correct either. You don't need a fake installation _and_ bend PYTHONPATH.
Virtualenv leverages the fact that "../lib/python<version>/site-packages/" relative to the interpreter is a default import path (default value of PYTHONHOME). It doesn't use PYTHONPATH.
> Why couldn't the import system resolve versions, too?
Not sure that would be really great. I prefer to have my dependencies all grouped together in a setup.py rather than scattered in various files at import time.
If I understand what you mean by package management in this context, I wonder if editable installs will help you.
What do you do about conflicts? Or say you have `import foo >= 1.0` in a file, and `import foo == 2.4`, but the latest version is 2.5, so the first import grabbed the latest version, and you later realize you need 2.4?
Imagine running a report generator for 5 hours, only to have the formatting module require a conflicting version of something and erroring out at run time...
I think the thing to remember about using relative imports is that it requires modules. Using relative imports to import something into a script will fail because the scripts don't belong to modules.
> Number two is, in order to do package management, you have to create a fake python installation and bend PYTHONPATH. Virtualenvs are the canonical way to do it, but to me it feels like a hack - the core language seems to wants all packages installed in /usr. So now I have all these virtualenvs lying around and they are detached from the scripts.
This is a pain. You can abstract this away with with pyenv or virtualenvwrapper, though.
It lead me to read the source of the Python "types" standard library module, which really does just create a bunch of different Python objects and then use type() to extract their types: https://github.com/python/cpython/blob/v3.9.6/Lib/types.py
Some examples from that file:
async def _ag():
_ag = _ag()
AsyncGeneratorType = type(_ag)
def _m(self): pass
MethodType = type(_C()._m)
BuiltinFunctionType = type(len)
BuiltinMethodType = type(.append) # Same as BuiltinFunctionType
- three modules cannot depend on each other in a circular way
- relative imports are fragile ("module not found")
- the __all__ definitions in the __init__ file make modules available under different full names
- how to reload a module in a jupyter notebook if edited
and so on.
When I do have a nasty circular dependency Webpack usually does a bad job telling me what’s wrong.
Though I should still treat circular imports as, at the very least, an organization code smell.
Circular imports are only ever a problem when you have code running when the module loads. Then you run into module load ordering issues. So avoid any side effects on module load and make all
Python is built upon namespaces and cycles in dependencies, either viewed as graph theory or kSAT introduce that fun NP-complete problem of version hell.
Karp's 21 and/or SAT will catch up with you at some point if you don't respect the constraints that make the problem tractable.
Note I am not saying I prefer or like pythons choices...but that they had to make one.
No it doesn't. __all__ just defines which objects are imported when doing a star import.
- understand difference between Python in IDE and Python in shell
So many times people do `pip install <>` and still not able to use in IDE
What would the purpose of circular modules like this be? You may as well collapse into a single module and the situation would not be any different would it?
What is the purpose of modules? You may as well collapse into a single script and the situation would not be any different, would it?
I'm not being facetious here. The answer to the second is the answer to the first.
A common example might go like this. You have a module for each kind of thing you have in the database. But now if someone loads a Company, they need to get to Employees. And if someone loads Employee they need to get to Accounting for the salary, payments, etc. And Accounting needs to be able to get to Company.
Those are all large and complicated enough that it makes sense to make them modules. But you just created a circular dependency!
The standard solution is to load a base library that loads all kinds of objects so they all can assume that all the others already exist and don't need a circular dependency. But of course someone won't like wasting all that memory for things you don't need and ...
Only if those things not only need to be able to “get to” each other, but also need to know, at compile time, about the concrete implementation of the others.
That's possible to be a real need, but its also something that often happens because of excessive and unnecessary coupling.
However your "compile time" point is important. There is another solution, which is to implement lazy loading of those classes.
So you put your import in the method of each that needs to know the other. This breaks the circular dependency and needs more up front memory. However it can also become a maintenance issue where a forgotten import in one function is masked by a successful import in another, until something changes the call and previously working code mysteriously goes boom.
It's all tradeoffs.
So parts of the system can be managed independently.
> The answer to the second is the answer to the first.
Clearly not - since circular modules cannot be managed independently!
In the example that I gave, the design described will handle complex edge cases such as a part time employee working for multiple companies. And will do so without programmers having to think through the whole system at all points.
Independence of modules has no importance in a codebase that ships as a whole. But modularity does.
The common approach to solving this is pulling everything that is used by all the modules into leaf libraries, effectively creating a directed acyclic graph, but this is not obvious nor easy to do the first time.
Boom, circular dependency.
Happens in basically all corporate code bases that grow over the years, with varying path lengths.
Throwing all potentially circular types into one big module isn't a great solution.
(In practice, we tend to rely on run-time imports to make it work. Not really great, but better than throwing several 10k or 100k lines of code into a single module).
Suppose you have some method of A which does something special if it gets an instance of B, and vice versa. Now you have a circular import problem; glhf
from a import A
But why not collapse into a single module at this point if you can’t avoid dependency? What are the separate modules adding at this stage forward?
I ran into this when A and B had many derived classes. I wanted to put A and it’s derived classes in one module, and B and it’s derived classes in another. It was messy.
I wound up putting A and B in a single module and having a separate one for the derived classes. Not ideal.
A and B both need to know about the other's base definition, neither cares about the details about the other's derived classes. Splitting it into three modules shares as little surface area as possible.
While that’s in rare circumstances the right thing to do, it's mostly an anti-pattern—you should be taking an object supporting a protocol, with the behavior difference depending on a field or method of (or actually implemented in a method of) that protocol. If you do that, you don't create a dependency on a concrete class that happens to require the special behavior.
Besides, if you are in a "well, fuck it, deadline is tomorrow" mode, you can always do something horrible like:
if 'classB' in type(obj).__name__: ...
But the truth is sometimes it has happened to me and the only solution I found was creating an small module with maybe one or two functions which is not exactly ideal.
If you keep doing that for a while, a circular dependency will happen.
This is the dumbest thing thing in Python. All other languages I know have solved it.
- protocols and abcs. Depend only on builtins, this sits at the top of the hierarchy and can be used anywhere
- top-level types/models, data structures. Only depends on external libraries
- config and logging. Depends on pydantic, which handles all init config validation, sets up log level. Many things import this
- specific class implementations, pure functions, transforms. imports from all the above
- dependency-injected stuff
- application logic
- routers and endpoints
I've had some close calls since I type check heavily, but protocols (since 3...6? 3.7 for sure) are a fantastic tool for both structuring and appeasing mypy.
So put them in the same module? Circular modules don’t give you the benefits of modules, do they? Not an expert in modularity.
In case you didn't know, each source code file is a module in Python.
# Run the interpreter with -u so that stdout isn't buffered.
"exec" "python3" "-u" "$0" "$@"
curdir = os.path.dirname(os.path.realpath(sys.argv))
# Add enough .. to point to the top-level project directory.
sys.path.insert(0, '%s/../..' % curdir)
Your main program starts here ...
This suggests that there is more than one entry point to the Python project?
While I'm sure there are good reasons for this, and while I'm not criticising your instance of this specifically, as a general point of advice I've found this sort of thing to be a bit of an anti-pattern.
Having one entry that handles things like path setup and other global concerns, before delegating out to subcommands or whatever construct works best makes it much easier to keep the whole codebase aligned in many ways.
Django has a system for this and while it has its flaws, it is nice to have it. Using this, on our main Python codebase of ~400k lines, we have a single manual entry point, plus one server entry point. Coordinating things like configuration, importing, and the application startup process, are therefore essentially non-issues for us for almost all development, even though we have a hundred different operations that a dev can run, each of which could have been a separate tool like this.
For additional bonus points, have your single entry point exhibit a CLI that properly documents everything the developer can do, i.e. what features are available, what environment variables and config flags can be set etc. That way, the code essentially documents itself and you know longer have to keep your README file updated (which people always tend to forget).
- everything is structured as a module
- options and args are stored in their own module for easy reuse
- the whole stack has one cleo.Application, with however many subcommands. Usually of the form "mytool domain verb" e.g. "mytool backend start."
- cleo args/options are parsed into pydantic objects for automatic validation (you could do this with argparse and dataclasses to skip the deps but it's more work)
- each subcommand has a `main(args: CliArgsModel)` which takes that parsed structure and does its thing. This makes it super easy to unit test
I install into a venv with `poetry install` or `pip install -e` for editable installs.
It all just works, no fuss, and so damn modular.
Also, say what you will about Perl and esoteric global variables, but it's kinda nice to be able to toggle buffered output on and off on the fly. Is there really no way to do this in python without re-executing the script like that?
For adding current directory to front of path, the sys.path.insert() call is a pretty sound way of doing it.
(To clarify, using "$0" with bash is just standard method to invoke the same script with an interpreter - sys.argv will work with or without bash exec part.)
"exec" "python3" "-u" "$0" "$@"
#!/usr/bin/env python3 -u
The contraption I wrote allows adding arbitrary parameters - I was burnt one too many times by Python silently buffering my debug messages, so I use it to always add "-u".
#!/usr/bin/env -S python $ARGS
#!/usr/bin/env -S python -i
print("Entering interactive mode...")
Honestly that error is misnamed. It should be `ModuleImportRefusedError`.
And the frustration caused by getting PyTest to work in a project is likely responsible for a large percentage of the untested python projects in the world...
the second occurs if you want your test files outside of the package directory (e.g., project/src/foo_app/foo.py, project/test/test_foo.py)
I ended up just giving up. I read programming in lua, and rewrote my entire project in lua and actually finished it.
Some day I'd like to go back and maybe learn Python but I really didn't enjoy my experience with it. I even found C headers easier to figure out than Python imports.
⏵ cat module.py
⏵ cat script.py
⏵ python3 script.py
⏵ cat script.py
from module import foo2
⏵ python3 script.py
"ImportError: attempted relative import with no known parent package"
Looks like I'm back to having to learn a bunch of stuff about scripts and modules again?
I was using Python 2 at the time. Python 3 was still relatively new. Not sure how much difference it makes for your example, but the import systems are different between 2 and 3.
I'm sure if I'd taken the time to try and fix it I eventually could have and at this point i've had more experience with a bunch of different languages, so I'm sure it's not as bad as I remember.
I imagine it's one of those cases where if i were to go back and laugh about how stupid I was, but ya know, those first impressions.
If you know how a path and relative path work from the shell, there is only one thing: touch __init__.py.
These are on the import doc page and beginners tutorial.
And I think that's for the best. I'd much rather have a happy path that I stay on than use some sort of dark magic that nobody who comes after me will understand.
- Entrypoints? Getting there.
- Namespace packages? Ehh. Murky
- site-packages/mypackage.pth - I get it but I don't know why sometimes it appears and other times not
- c extensions? Ehhh.
- .so loading? Kinda magic.
- the confluence of editable installs, namespace packages, foo.pth, PYTHONPATH, sys.path, relative imports, entry points, virtualenvs, LD_LIBRARY_PATH, PATH, shell initialization, setup.py vs pyproject.toml? Um yeah that's some heavy wizardry.
Tbf, you don't need the vast majority of that to be effective in python.
I now have the rule of, only ever use pip inside a venv. If your venv is more than a little bit complex, write a requirements.txt file so you can generate it. So it's something like
$ cat > requirements.txt << EOF
$ echo venv > .gitignore
$ python3 -m venv venv
$ venv/bin/pip install -r requirements.txt
$ . venv/bin/activate
(venv)$ pip install -r requirements.txt
$ rm -r venv
$ python3 -m venv venv
$ venv/bin/pip install -r requirements.txt
Either of these rules works fine. The thing that works poorly is using pip install outside of a venv (with or without root).
Skip the relative imports in the article. No need to read it entirely anymore ;-)
Basically instead of
from magicimport import magicimport
tornado = magicimport("tornado", version = "4.5")
It's nice to have something that "just works" without having to install it. I like to call it Level 5 autonomous software -- software that can figure out how to run itself with zero complaints.
I actually use this for a lot of personal non-production scripts, I can just clone the script on any system and just run it, it will figure itself out.
Also, packages with version dependencies fuck up other packages with other version dependencies, unless you set up virtualenvs or dockers or condas for them, and those take additional steps.
magicimport.py uses virtualenv behind the scenes, so when it imports tornado 4.5 because some script wants 4.5, it won't mess up the 6.0 that's already on your system. In some sense it just automates the virtualenv and pip install process on-the-fly, to give the effect that the script "just works".
Not something I'd use in production, but it's a very clear way to see how both "finding a module" and "loading a module" works under the covers.
Edit: As an aside, I much prefer the way Perl does things in this space. It's much easier to define multiple packages either within a single file or across different files, and much clearer about what's happening underneath.
There are some lessons here that other languages would do well to learn. Trouble importing 3rd party libraries must be a kiss of death for beginner engagement.
You have internalised all these quirks, and know how to work with/around them. Beginners haven't.
Start with pip, the official package manager. You don't have to start with virtual envs: I didn't.
I haven't ran into any function which I "should not use".
You don't need an entry point, you can just write code in a file. Besides that, I don't see how much it differs from other languages with implicit entry points, where you need to match a certain name in your function.
Imports: yes, those are annoying and confusing. I still struggle with those.
Aditionally: package managers and virtual envs are a pain in the ass. Every year we're getting a new one which is supposed to solve the problems from the previous one, but doesn't, and the cycle goes on. The language should really solve this at the core instead of requiring community fixing, as it is a core part of any serious development.
Dependency resolution just works. Editable installs just work. Building just works.
Before that, I only had problems with virtual envs once, and it was due to bad hygiene with system python libs and deps. Moral of the story: don't. Unless it's necessary to bootstrap virtualenv or compiled libs, don't system install python. Good ol' get-pip.py and virtualenv.
And then I was quite shocked by the state of package managers in python. You need to learn pip, venv (with references to "virtualenv"), these are too low level, so you find pipenv, which is unbelievably slow (installing a single dependency can take 10 minutes), so you need to learn to use it with "--skip-lock", but then you lose locking ...
I've never appreciated node's bundled "npm" so much before which mostly "just works".
On the contrary, C projects tends to build with 3 commands and C# (often way bigger) with a single command, and without having to do magic things around "virtual environments".
I've been programming in Python for quite a while but didn't really understand how the import system works: what modules and packages are exactly; what relative imports are relative to; what's in sys.path and so on. My goal with this post was to answer all these questions.
I welcome your feedback and questions. Thanks!
One of the simplest import systems I've seen were in q/kdb (like with most things in that language, everything is as simple as possible)
Imports work by simply calling `\l my_script.q` which is similar to simply taking the file `my_script.q` and running it line by line (iirc, it does so eagerly, so it reruns the entire file whenever you do `\l my_script.q`, even if the file has been loaded before, which may affect you state. By contrast, Python `import` statements are no-ops if the module has already been imported).
The main disadvantage is that you risk your imported script overwriting your global variables. This is solved by following the strong (unenforced) conversion that scripts should only affect their own namespaces (which works by having the script write declare \d .my_namespace at the top of the script)
I never found this system limiting and always appreciated its simplicity - whenever things go wrong debugging is fairly easy.
What does Python gain by having a more sophisticated import system?
Suppose that you are importing a larger project. Where your one import (say of your standard database interface) pulls in a group of modules (say one for each table in your database), all of which import a couple of base libraries (to make your database objects provide a common interface) that themselves pull in common Python modules (like psychopg2) which themselves do further imports of standard Python modules.
The combinatorial explosion of ways to get to those standard Python modules would make the redundant work of parsing them into a significant time commitment without a caching scheme.
From the point of view of a small script, this kind of design may seem like overkill. But in a large project, it is both reasonable and common.
These are slightly more complicated than "load the script at this path".
There's probably a more detailed answer, in that historically, decisions were made that we're now stuck with. Python packages can and sometimes intentionally have import-time side effects, for example. They must be only run once, without relying on convention, or we break existing code.
If you need to start installing your own user packages, you need `pip` and then `venv` and then things get ugly, but for the usual case where the sysadmin deals with all that (or you're on Windows), it works quite well.
He specifically describes “strict modules” and efforts to improve the efficiency of Python imports at 16:30 of this episode of Django Chat:
Unless anyone knows of magical tools to help solve import issues?
Maybe some think this is only a theoretical problem and doesn't happen with "well-written" libraries. Well, here is one example which bit me in the past: https://stackoverflow.com/a/4706614/767442
Topological sorting? Always wondered why programming languages can't do what package managers do.
Actually I'm not even sure what kind of error we're talking about here. If two modules import each other and they both need access to the other's contents upon initialization, there is no ordering that will work. And if at most one needs access to the other, it will always work, no matter in which order they are imported. So I don't really know what the OP was talking about.
The implementation is surprisingly straightforward, once you've come to terms with the basic idea, see  and the rest of the `clearml.binding` package.