7 ms·
Show HN: deptry 0.14.0 – detect unused Python dependencies up to 10 times faster
deptry 0.14.0 was just released, bringing significant speed improvements: It is now up to 10 times faster than the previous release!
For those unfamiliar with deptry; deptry is a command line tool to check for issues with dependencies in a Python project, such as unused or missing dependencies.
GitHub: https://github.com/fpgmaas/deptry https://github.com/fpgmaas/deptry
For some benchmarks of the new release, see the release notes:
https://github.com/fpgmaas/deptry/releases/tag/0.14.0 https://github.com/fpgmaas/deptry/releases/tag/0.14.0
The performance improvement was achieved by leveraging Rust to parse the AST and extract the import statements from .py files, rather than using Python's ast module.
The addition of Rust to the project also opens up doors for more optimizations in the future, so stay tuned!
- ditsuke 3y agoI recently introduced tweag's fawltydeps for this use-case at work, looking forward to try deptry when I run into this problem again..
- wodenokoto 3y agoI'm a bit confused as to what it does. Does it compare requirements.txt against imports, or checks if imports are actually used?
- VeejayRampay 3y agoI feel like cases where the import is different from the package name yield false positives i.e. if I have beautifulsoup4[html5lib,lxml]==4.12.3 in my requirements.txt and then import with from bs4 import BeautifulSoup it would complain is there a way to configure that somehow? aside from that, the tool is extremely fast
- flo12392 3y agoThis is not an issue when using `deptry`; it does not only match the import name against the dependency name; it first tries to find the modules that are provided by the dependency `beautifulsoup4`. If you run `deptry -v .`, you can see this in the verbose logs: ` Dependency 'beautifulsoup4' with top-levels: {'bs4'}.` If you ever run into a situation where this does not work, you can use the `pacakge_module_name_map` parameter: https://fpgmaas.github.io/deptry/usage/#package-module-name-map https://fpgmaas.github.io/deptry/usage/#package-module-name-... Hope this helps! Florian
- verdverm 3y agoVS Code does this for me. What use case is this after? Linting in CI or similar?
- flo12392 3y agoI am not aware of functionality within VS Code that does the same thing as `deptry` does. What most IDE's support is removing unused imports, which sounds similar but is different from what deptry aims to do: deptry tries to remove unused dependencies. So if you have e.g. `pandas` in your `requirements.txt`, but you never import (from) `pandas` in your project, `deptry` will tell you that you should remove it from your `requirements.txt`. Does that answer your question?
- parpfish 3y agoThis is particularly helpful if at one point somebody replaced a nice succinct requirements.txt with the output of ‘pip freeze’
- hereonout2 3y agoThis is a bit more useful than things like vscode or flake8 (my preferred option) telling you that you "import xxx" but don't use it. I'd love a deeper way to identify and selectively reduce the cruft from projects though. For example currently I have about 180 dependencies in my project's .venv - most of them transitive. These include things like the google-auth library when there is no code in my project that would ever need to talk to Google. Mostly this is just wasteful and not a big issue, but then come the conflicting dependencies you need to resolve by pinning version numbers. This is doubly annoying if you have no idea what the dependency is needed for or even if it's needed at all. This is not just a python issue, it's apparent in all languages I've worked in.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- LordKeren 3y agoI inherited a legacy code base that suddenly had a lot more attention on it and had to be rapidly overhauled. Deptry was invaluable for quickly removing a large amount of unused dependencies. The newest update was a delightful surprise since deptry is now fast enough to run during automated pull-request analysis via GitHub actions
- flo12392 3y agoThanks for the positive feedback! Happy to hear that it turned out to be useful for you.
- passion__desire 3y agoDoes it provide a tree output of the dependencies as they do in Scala/Java land?
- flo12392 3y agoIf I understand your question correctly; no, that is out of scope for deptry. Many of the dependency management tools already provide a way to do so, e.g. for PDM `pdm list --tree`.
- passion__desire 3y agoYou could provide diff of current tree vs tree after applying your suggested removals? It is more complex than that ?
- BerislavLopac 3y agoUse pipdeptree for that
- deleted 3y ago[deleted]
- infocollector 3y agoSpeed is impressive. Accuracy could use some work. Input for authors: 1. DEP003 'click' imported but it is a transitive dependency (In my code base click is used @click - which it doesn't look like it sees) 2. DEP003 'numcodecs' imported but it is a transitive dependency Usage was open(...,compressor=numcodecs.xxx,...) Perhaps these warnings/errors should be removed?
- eatox 3y agoOn the contrary, it does see those imports, and indicates that those dependencies are used, but are not direct dependencies in your dependency tree, and instead installed because other dependencies rely on them: https://fpgmaas.github.io/deptry/rules-violations/#transitive-dependencies-dep003 https://fpgmaas.github.io/deptry/rules-violations/#transitiv...
- infocollector 3y agoThank you for catching this! Indeed its not a bug.
- globular-toast 3y agoThat's really useful. I find it really common for people to import transitive dependencies in python. A common one is putting something like matplotlib in deps then directly importing numpy. Seems to happen all the time.
- marcooliv 3y agoCongrats and thank you! I have a ticket on my backlog to clear our main project non-used dependencies. haha
- flo12392 3y agoDon't tell your colleagues about deptry, take the afternoon off and tell your manager you manually removed each dependency one by one and ran the unit tests to check if the dependency was needed ;)
- kylegalbraith 3y agoDoes this work with uv as well?
- flo12392 3y agoGood question, I have not looked into this myself yet. I would say; definitely, since it uses the dependencies from a `requirements.in` or a `pyproject.toml` file to generate a set of locked dependencies. Since `deptry` supports both formats, this should be no problem. One thing to keep in mind, that would also be good for `deptry` to think about; if users decide to create a `requirements.txt` file from a `requirements.in` file, `deptry` will automatically use the `requirements.txt` file. However, this is wrong: It should use the `requirements.in` file, since the `requirements.txt` file also contains the transitive dependencies. Users can change this using the `requirements-txt` flag though (https://fpgmaas.github.io/deptry/usage/#requirements-txt https://fpgmaas.github.io/deptry/usage/#requirements-txt). Thanks for raising this, I will probably create an issue in our project to see how we should feal with this :)
- flo12392 3y agoThere we go: https://github.com/fpgmaas/deptry/issues/608 https://github.com/fpgmaas/deptry/issues/608
- singhrac 3y agoI have used deptry for my most recent set of projects and it is so useful to make sure your project is movable to another venv (or PEP518 compliant).