The deepest problem I see w/ the Python dependency system is the need to be able to declare substitute or alternative dependencies.
For instance you should be able to install a numpy that is compiled to take advantage of your specific CPU. You should be able to pick at install time which one you want WITHOUT patching the the source code directory in any way that would make it back to version control.
Assuming you don't want to actually patch Numpy and just want to make sure it's compiled with support for processor features you have available like AVX-512 and SSE2 and what not, you can already do this by just telling pip to prefer source, which means it'll compile it at install time and the underlying build system will detect and compile in support for whatever processor features you can use.
I suspect it's the features of the BLAS that will matter, though, a lot more than Numpy itself. The Numpy build system by default prefers MKL, OpenBLAS, then ATLAS, and I don't remember what after that, and it's up to you to compile those with the features you need.
I DO NOT want to get anywhere near compiling dependencies at install time, that is how you make your Python build process entirely irreproducible!
I didn’t feel in control of Python builds until I had a wheels only policy. (For one things you can download about 2% of a wheel and get the dependencies to put into an SMT solver and globally solve the dependencies before you build anything. pip, poetry and similar systems use eggs and have to run setup.py to figure out deps which means they solver locally and eagerly and back themselves into corners that they may or may not be able to get out of.)
If I have to build from source I want to build a wheel, archive it, then reference the wheel in my repository. I don’t quite have the tooling to make it as easy as it really is.
Do you have an example of how it would be solved in other languages?
In python I would create different requirements files together with freeze versions of those. One could contain the standard "numpy" and another would be "numpy @ my-custom-built-numpy-url". I haven't tried it and don't know how it would work in practice.
For instance you should be able to install a numpy that is compiled to take advantage of your specific CPU. You should be able to pick at install time which one you want WITHOUT patching the the source code directory in any way that would make it back to version control.