Show HN: deptry 0.14.0 – detect unused Python dependencies up to 10 times faster

[flo12392]

https://github.com/fpgmaas/deptry/releases/tag/0.14.0

deptry 0.14.0 was just released, bringing significant speed improvements: It is now up to 10 times faster than the previous release!

For those unfamiliar with deptry; deptry is a command line tool to check for issues with dependencies in a Python project, such as unused or missing dependencies.

GitHub: https://github.com/fpgmaas/deptry

For some benchmarks of the new release, see the release notes:

https://github.com/fpgmaas/deptry/releases/tag/0.14.0

The performance improvement was achieved by leveraging Rust to parse the AST and extract the import statements from .py files, rather than using Python's ast module.

The addition of Rust to the project also opens up doors for more optimizations in the future, so stay tuned!

[LordKeren]

I inherited a legacy code base that suddenly had a lot more attention on it and had to be rapidly overhauled. Deptry was invaluable for quickly removing a large amount of unused dependencies.

The newest update was a delightful surprise since deptry is now fast enough to run during automated pull-request analysis via GitHub actions

[flo12392 → LordKeren]

Thanks for the positive feedback! Happy to hear that it turned out to be useful for you.

[passion__desire → flo12392]

Does it provide a tree output of the dependencies as they do in Scala/Java land?

[flo12392 → passion__desire]

If I understand your question correctly; no, that is out of scope for deptry. Many of the dependency management tools already provide a way to do so, e.g. for PDM `pdm list --tree`.

[passion__desire → flo12392]

You could provide diff of current tree vs tree after applying your suggested removals? It is more complex than that ?

[BerislavLopac → passion__desire]

Use pipdeptree for that

[infocollector]

Speed is impressive. Accuracy could use some work. Input for authors:

1. DEP003 'click' imported but it is a transitive dependency (In my code base click is used @click - which it doesn't look like it sees)

2. DEP003 'numcodecs' imported but it is a transitive dependency Usage was open(...,compressor=numcodecs.xxx,...)

Perhaps these warnings/errors should be removed?

[eatox → infocollector]

On the contrary, it does see those imports, and indicates that those dependencies are used, but are not direct dependencies in your dependency tree, and instead installed because other dependencies rely on them: https://fpgmaas.github.io/deptry/rules-violations/#transitiv...

[globular-toast → eatox]

That's really useful. I find it really common for people to import transitive dependencies in python. A common one is putting something like matplotlib in deps then directly importing numpy. Seems to happen all the time.

[infocollector → eatox]

Thank you for catching this! Indeed its not a bug.

[marcooliv]

Congrats and thank you! I have a ticket on my backlog to clear our main project non-used dependencies. haha

[flo12392 → marcooliv]

Don't tell your colleagues about deptry, take the afternoon off and tell your manager you manually removed each dependency one by one and ran the unit tests to check if the dependency was needed ;)

[VeejayRampay]

I feel like cases where the import is different from the package name yield false positives

i.e. if I have

  beautifulsoup4[html5lib,lxml]==4.12.3

in my requirements.txt and then import with

  from bs4 import BeautifulSoup

it would complain

is there a way to configure that somehow?

aside from that, the tool is extremely fast

[flo12392 → VeejayRampay]

This is not an issue when using `deptry`; it does not only match the import name against the dependency name; it first tries to find the modules that are provided by the dependency `beautifulsoup4`. If you run `deptry -v .`, you can see this in the verbose logs:

` Dependency 'beautifulsoup4' with top-levels: {'bs4'}.`

If you ever run into a situation where this does not work, you can use the `pacakge_module_name_map` parameter: https://fpgmaas.github.io/deptry/usage/#package-module-name-...

Hope this helps! Florian

[singhrac]

I have used deptry for my most recent set of projects and it is so useful to make sure your project is movable to another venv (or PEP518 compliant).

[ditsuke]

I recently introduced tweag's fawltydeps for this use-case at work, looking forward to try deptry when I run into this problem again..

[wodenokoto]

I'm a bit confused as to what it does.

Does it compare requirements.txt against imports, or checks if imports are actually used?

[kylegalbraith]

Does this work with uv as well?

[flo12392 → kylegalbraith]

Good question, I have not looked into this myself yet. I would say; definitely, since it uses the dependencies from a `requirements.in` or a `pyproject.toml` file to generate a set of locked dependencies. Since `deptry` supports both formats, this should be no problem.

One thing to keep in mind, that would also be good for `deptry` to think about; if users decide to create a `requirements.txt` file from a `requirements.in` file, `deptry` will automatically use the `requirements.txt` file. However, this is wrong: It should use the `requirements.in` file, since the `requirements.txt` file also contains the transitive dependencies. Users can change this using the `requirements-txt` flag though (https://fpgmaas.github.io/deptry/usage/#requirements-txt).

Thanks for raising this, I will probably create an issue in our project to see how we should feal with this :)

[flo12392 → flo12392]

There we go: https://github.com/fpgmaas/deptry/issues/608

[verdverm]

VS Code does this for me.

What use case is this after? Linting in CI or similar?

[flo12392 → verdverm]

I am not aware of functionality within VS Code that does the same thing as `deptry` does. What most IDE's support is removing unused imports, which sounds similar but is different from what deptry aims to do: deptry tries to remove unused dependencies. So if you have e.g. `pandas` in your `requirements.txt`, but you never import (from) `pandas` in your project, `deptry` will tell you that you should remove it from your `requirements.txt`. Does that answer your question?

[hereonout2 → flo12392]

This is a bit more useful than things like vscode or flake8 (my preferred option) telling you that you "import xxx" but don't use it.

I'd love a deeper way to identify and selectively reduce the cruft from projects though. For example currently I have about 180 dependencies in my project's .venv - most of them transitive.

These include things like the google-auth library when there is no code in my project that would ever need to talk to Google.

Mostly this is just wasteful and not a big issue, but then come the conflicting dependencies you need to resolve by pinning version numbers. This is doubly annoying if you have no idea what the dependency is needed for or even if it's needed at all.

This is not just a python issue, it's apparent in all languages I've worked in.

[parpfish → flo12392]

This is particularly helpful if at one point somebody replaced a nice succinct requirements.txt with the output of ‘pip freeze’