6 ms·
The AMD tinybox is on hold until we can build and run the firmware on our GPUs
- yinser 2y agoIdentifying MES & CP as the barriers to an AMD tinybox and letting the community know is a huge service and is still great engineering even if they decide it's an immoveable barrier and walk away.
- Havoc 2y agoUnfortunate but understandable. AMD needs to move faster on software support in AI space if they want any of that money.
- brucethemoose2 2y agoBut is this going to blow over in a few days? Again? I can certainly appreciate frustration with the AMD stack, but be blunt, I was not impressed with Hotz's YouTube rant from before.[1] It didn't give the impression of a stable framework, and this doesn't either. Also (at least from the end user llm inference side of things) ROCm is not nearly as unusable as it used to be. We would certainly be renting MI300s over A100s (or even H100s) if we could get any, and we use a number of different inference backends. 1: https://news.ycombinator.com/item?id=36193625 https://news.ycombinator.com/item?id=36193625
- jauntywundrkind 2y agoThe post PC era of big expensive hardware you can't even buy if you do have the money is upon us, and mercy this is a scary scary time in computing for me/us.
- smoldesu 2y agoBesides the datacenter stuff, what exactly are people struggling to source these days? The 30/40-series prices should be fairly stable relative to the MSRP these days.
- brucethemoose2 2y agoUsed 3090 prices are absolutely outrageous. And the 4090 MSRP was outrageous to begin with.
- brucethemoose2 2y agoI was talking about renting! There are some boutique hosts like Hot Aisle serving MI300s (who I really should reach out to), but for the immediate future our little startup is stuck with the big cloud providers. No MI300s for us mere mortals, not even to rent.
- pjmlp 2y agoYet another thing to go back to 1990's ecosystem. We already have timesharing again, now we have the prices as well.
- whalesalad 2y agohe's always been nails on a chalkboard for me. rant, whine, cry, repeat. reminds me of larry david with a CS degree.
- brucethemoose2 2y agoI never followed Hotz, so perhaps I missed something cool. But I never understood the hype myself.
- deleted 2y ago[deleted]
- zachbee 2y agoWhen they originally announced tiny corp and the tinybox, the entire pitch was that AMD hardware were great, but their software was bad. [1] Now they're giving up on AMD hardware because the software is bad. Wasn't the whole point to solve that problem? I'm hopeful for tinygrad as a piece of software, but I'm skeptical about the future of the tinybox if they keep waffling on the hardware so much. [1] https://geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...
- wmf 2y agoThe software is now fixed but that only revealed that the firmware is bad.
- 1oooqooq 2y agonext stop: microcode.
- throwawaymaths 2y agoThere is a texture between hard and soft
- snissn 2y agogooey?
- patmorgan23 2y agoFirme(ware)
- patmorgan23 2y agoFirm(ware)
- jrflowers 2y agoSilken
- 2y ago
- chaostheory 2y agoDoes Nvidia have any real competition that’s already shipped?
- pjmlp 2y agoThat is the thing, they can allow themselves to have all that proprietary stuff, because Intel, AMD just can't get their act together. Even the whole OpenCL versus CUDA, they had years to ship something that was great tooling alternative, instead they did everything but that.
- whywhywhywhy 2y agoWhy would there be? The writing was on the wall that this was important 10 years ago and no one moved on it, in fact AMD and Apple both fumbled OpenCL then proceeded to waste several more years after that. I know some people don't like Nvidia but like their competition had their opportunity and what needed to be done spelt out to them and did nothing.
- brucethemoose2 2y agoThe MI300 is the best accelerator you can buy, for many current workloads. It's technically way more advanced. Not as outrageously priced as an H100 either.
- lostmsu 2y agoTBH I don't know what they were counting on. 4090 has almost 3x BF16 tensor ops/s vs 7900XTX. So you can just buy a regular PC with 2x4090 for half the price and have basically the same training performance with much less headache.
- renewiltord 2y agoFrequently people say on HN you should use AMD but looks like it's not going to work. I am glad to stick to straightforward Nvidia GPU / Epyc CPU stack. Don't want to innovate for this.
- convolvatron 2y agothere is no room for innovation in high performance tensor evaluation, cost-performance, or usability. we're just done.
- renewiltord 2y agoThere is room, but if you are not working on building the framework, it's not worth building the framework. The time cost is high.
- brucethemoose2 2y agoIt's not either or, you can use different vendors for different tasks. tinygrad isn't in the realm of production ready though, AFAIK.
- fisf 2y agoYes but the same could be said about rocm.
- wmf 2y agoSomething that isn't really talked about is that Tiny Corp has been kind of working against AMD's interests. Tinybox is/was about replacing MI300s with much cheaper 7900 XTXs. I'm not terribly surprised to discover that AMD is not investing in ROCm on consumer cards (which are a different architecture) even though they're technically supported.
- throwaway48476 2y agoNo one would be buying A100's if CUDA hadn't been supported on all desktop Nvidia cards for years. Desktop cards provide an accessible on ramp to the ecosystem. PhD grads with boxes of desktop cards turn into the purchasers of data center chips.
- cherioo 2y agoNvidia was forced to do that because it was so early, they had no customers except PhDs. I see no evidence AMD wants to do that right now, and instead focusing on extracting value from deep pocket enterprise customers. The way things go, I think the AMD consumer card experience will only get better once AMD manage to gimp consumer cards’ ML throughput or RAM.
- DaiPlusPlus 2y ago> The way things go, I think the AMD consumer card experience will only get better once AMD manage to gimp consumer cards’ ML throughput or RAM. Que? Making things worse will make things better?
- sjsdaiuasgdia 2y agoThey'll let you do some things on a consumer card as soon as they can make sure that you can't effectively use the consumer card in place of an enterprise card.
- DaiPlusPlus 2y ago
- whalesalad 2y agoThis was destined to be a hard problem. Surprised to see they are giving up so easily.
- caycep 2y agowhat's so tiny about a box that has 6 gpus?
- zitterbewegung 2y agoWhat advantage will remain if tensorflow and PyTorch works on AMD cards? https://www.xda-developers.com/nvidia-cuda-amd-zluda/ https://www.xda-developers.com/nvidia-cuda-amd-zluda/ https://pytorch.org/ https://pytorch.org/ has a rocm support . This doesn’t make the outlook on this company very good …
- alecco 2y ago> We are also (sadly) exploring a 6x4090 box. $12k for 6 4090 for 144GB GDDR vs $20k H100 PCIe 80GB HBM2 (price likely dropping later this year when B100 is released). And H100 has a lot of features like async and loading directly to tensor cores not present in consumer cards. I want to root for the little guy, but it seems the AI hardware landscape will be Nvidia for the next few years. And us GPU poors accessing it via cloud (shudders).
- mdaniel 2y agogiven his background in the jailbreak community, I look forward to him jailbreaking the AMD GPU firmware load process :-D
- mnau 2y agoJailbreaking is not the problem. The problem is reverse engineering a large firmware that operates on HW they have no docs for and fixing the firmware (ie. doing it better that original manufacturer that has access to everything). The job of firmware is so close to the hw that it's nearly impossible to decode. You need to decode a custom CPU instructions for a IP block the microcode is running on (it's custom, no ARM/RISC/MIPS..). After that you have to decode what firmware actually does. It writes something to this.... What does it mean? It's completely opaque number written to a opaque memory... cache control? Delay? And you do that why? So you can ship tinybox (i.e. cheap consumer GPU). Let's says you succeed. Do the same thing Next gen will be similar challenge, except firmware will be better locked, because AMD will want consumer HW to be segregated from data center GPU, the same way NVidia does. The task itself is basically impossible and waste of time. There is the reason why NVidia driver driver for Linux was used basically only to install official driver.
- mdaniel 2y agorelevant to his tenstorrent mention: https://news.ycombinator.com/item?id=39658787 https://news.ycombinator.com/item?id=39658787 and an bunch more https://hn.algolia.com/?q=tenstorrent https://hn.algolia.com/?q=tenstorrent