6 ms·
You don't need all of the model in VRAM. 1 or 2 RTX Pro 6000s will do. $50K will get you there very nicely, and on a 1600 watt PSU if you go for the MAX-Q versi
by theYipster 3mo ago
You don't need all of the model in VRAM. 1 or 2 RTX Pro 6000s will do. $50K will get you there very nicely, and on a 1600 watt PSU if you go for the MAX-Q versions. (The same wattage PSU I'm typing this on, and have been using over the last 5 years.)
- Tepix 3mo agoIf you want decent performance (more than say 20 tokens/s) for your dev team, you absolutely do need all of the model in VRAM.