5 ms·
That depends on the workload. When buying for on-prem inference, MI355x is quite capable. It still requires a ton more ops time, not every dc handles it coolin
by DiskoHexyl 2mo ago
That depends on the workload. When buying for on-prem inference, MI355x is quite capable.
It still requires a ton more ops time, not every dc handles it cooling-wise, and you have to be careful with the model selection, but when it fits, it fits well