5 ms·
Or just wait for the NVIDIA Digits PC later this year which will cost the ~same amount and can fit on your desk
by zelcon 2y ago
Or just wait for the NVIDIA Digits PC later this year which will cost the ~same amount and can fit on your desk
- dzogchen 2y agoThat one can handle up to 200B parameters according to NVIDIA.
- zelcon 2y agoThat's a shame. I suppose you'll need 4 of them with RDMA to run a 671B, but somehow that seems better to me than trying to run it on DDR4 RAM like the OP is saying. I have a system with 230G of DDR4 RAM, and running even small models on it is atrociously slow.