6 ms·
Looks like a pretty useful offering, 128Gb Memory Unified, with the ability to be chained. IN the Uk release price looks to be £2999.99 Nice to see AI Inference
by maxbaines 10mo ago
Looks like a pretty useful offering, 128Gb Memory Unified, with the ability to be chained. IN the Uk release price looks to be £2999.99 Nice to see AI Inference becoming available to us all, rather than using a GPU ..3090etc.
https://www.scan.co.uk/products/asus-ascent-gx10-desktop-ai-supercomputer-gb10-blackwell-superchip-1tb-ssd-128g-lpddr5x-200gb-connec https://www.scan.co.uk/products/asus-ascent-gx10-desktop-ai-...
- BoredPositron 10mo agoI would hold my horses and see if the specs are actually true and not overblown like for the spark otherwise there are better options.
- exasperaited 10mo agoAnd if waiting six months is possible, do that. Asus make some really useful things, but the v1 Tinker Board was really a bit problem-ridden, for example. This is similarly way out on the edge of their expertise; I'm not sure I'd buy an out-there Asus v1 product this expensive.
- eightysixfour 10mo agoThis is a Spark, so it is not going to be any different.
- atwrk 10mo agoAll Sparks only have a memory bandwidth of 270 GB/s though (about the same as the Ryzen AI Max+ 395), while the 3090 has 930 GB/s. (Edit: GB of course, not MB, thanks buildbot)
- buildbot 10mo agoI believe you mean GB/s?
- postalrat 10mo agoThe 3090 also has 24gb of ram vs 128gb for the spark
- Gracana 10mo agoYou'd have to be doing something where the unified memory is specifically necessary, and it's okay that it's slow. If all you want is to run large LLMs slowly, you can do that with split CPU/GPU inference using a normal desktop and a 3090, with the added benefit that a smaller model that fits in the 3090 is going to be blazing fast compared to the same model on the spark.
- Jackson__ 10mo agoEh, this is way overblown IMO. The product page claims this is for training, and as long as you crank your batch size high enough you will not run into memory bandwidth constraints. I've finetuned diffusion models streaming from an SSD without noticeable speed penalty at high enough batchsize.
- cmxch 10mo agoAt that price (roughly 4000 USD), one could build a full HBM powered Xeon system from the Sapphire Rapids generation. Either build a single socket system and give it some DDR5 to work alongside, or go dual socket and a bit less DDR5 memory.