6 ms·
Microsoft recently announced that they run chatgpt 3.5 & 4 on mi300 on Azure and the price/performance is better. https://www.amd.com/en/newsroom/press-release
by oelang 2y ago
Microsoft recently announced that they run chatgpt 3.5 & 4 on mi300 on Azure and the price/performance is better.
https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd-instinct-mi300x-accelerators-power-microsoft-a.html https://www.amd.com/en/newsroom/press-releases/2024-5-21-amd...
- sigmoid10 2y agoI've used ChatGPT on Azure. It sucks on so many levels, everything about it was clearly enforced by some bean counters who see X dollars for Y flops with zero regard for developers. So choosing AMD here would be about par for the course. There is a reason why everyone at the top is racing to buy Nvidia cards and pay the premium.
- dexterac 2y ago"Everyone" at top is also developing their own chips for inference and providing APIs for customers to not worry about using CUDA. It looks like the price to performance of inference tasks gives providers a big incentive to move away from Nvidia.
- sigmoid10 2y agoThere are only like 3 AI building companies who have the tech capability and resources to afford that and 2 of them don't even offer their chips to others or have gone back to Nvidia. The rest is manufacturers desperately trying to get a piece of the pie.