7 ms·
Ask HN: Has Anyone Successfully Run DeepSeek-R1 on an Nvidia Jetson Nano?
Hey HN,
I am curious if anyone has successfully run Deepseek-R1 on a Nvidia Jetson Nano, especially for programmatic use.
If so, did it work well?
Any tips on what to do, what to avoid for it to run smoothly?
Thanks
- DrPhish 2y agoThe Nano only has 4GB VRAM and DS-R1 is 671B FP8 parameters (equivalent to 671GB model size). You need something with about 800GB to run the full model with context. You'd still need 400GB to even run a half-sized Q4 quant of R1, so there is no reasonable way that it would work.
- florianbansac 2y agoOK, I understand the flagship model is huge, It seems to be far from local use. Anyone did it with a smaller/distilled version, and getting good performance?
- n0vella 2y agoJust curious, what it's the cheapest card do you think that would be needed to run this model or something like llama 3.3-70B? Only nvidia cards are compatible or AMD ones also could work?