6 ms·
Yeah, for a single GPU inference, considering the higher VRAM and FP4 support on the RTX 6000, it should fit larger models as well than the H100.
by k2so 7mo ago
Yeah, for a single GPU inference, considering the higher VRAM and FP4 support on the RTX 6000, it should fit larger models as well than the H100.