7 ms·

Nvidia DGX Spark as a daily driver

102 points by plun9 2mo ago

midnightbobarun 2mo ago
Super-powerful (if rather pricy) Linux desktop that happens to play games while doing everything else... that man is living my dream :'D
bryanlarsen 2mo ago
It's interesting how many of these issues don't appear to be specific to the DGX Spark but to the standard "Nvidia GPUs suck on Linux" type of issues that afflict a lot of people.
- a-dub 2mo ago
  it's actually very stable on x86_64 these days, even with optimus.
cogman10 2mo ago
I strongly considered it, but the one thing that scares me away from wanting to do the spark is you basically have to use nvidia's linux (from what I've read) and it doesn't appear the nvidia is interested in upstreaming their kernel changes. I'm avoiding where possible buying electronics where support is controlled by the manufacturer and not me.
- willis936 2mo ago
  A good instinct. There are a lot of things a $500 AMD GPU can do in linux that a $5000 DGX cannot.
  - ARandomerDude 2mo ago
    I know nothing about this topic but your comment piqued my curiosity. What would a $500 AMD GPU do better than DGX?
    - bigyabai 2mo ago
      AMD has Mesa drivers for graphics, which are better-optimized than Nvidia's proprietary Linux Vulkan drivers. It can be fixed in software, but Nvidia's only barely started to catch up. The focus for Nvidia's GPU stack on Linux is getting CUDA working, which means that some traditional raster features get neglected.
      - pjmlp 2mo ago
        NVIDIA also cares about Hollywood and CAD/CAM shops. I learnt three times that AMD isn't necessarily better in Linux as advertised, now I don't care any longer.
        abe_m 2mo ago
        I'm not aware of any commercial CAD/CAM software running Linux that Nvidia would have a money interest in supporting. Which current CAD/CAM is running on Linux with graphics? The closest I'm aware of it FEA packages allowing remote calculation on Linux machines while the GUI runs on Windows. Is that what you're referring to?
        pjmlp 2mo ago
        BricsCAD, ARES Commander and Platicity come to mind.
        bigyabai 2mo ago
        A lot of Hollywood studios use renderers that jump straight to CUDA as their backend. Cycles is a popular one, as well as Renderman and Octane.
        pjmlp 2mo ago
        Linux is part of the VFX reference platform, and you forgot Hyperion, for example.
  - girvo 2mo ago
    Although one of the ones it cannot do is have/address 128GB of video memory, so it depends on what you want to achieve.
    - seemaze 2mo ago
      AMD Strix Halo was half the price at launch, same 128GB unified memory and nearly identical memory bandwidth. It is x86, so run whichever OS your heart desires. You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed..
      - htrp 2mo ago
        I'm seeing it for 4k and up for similar specs as the spark?
        adrian_b 2mo ago
        Yes, due to the huge increase in memory price and also exploiting the demand for local LLM inference, the price for a mini-PC with Strix Halo has become more than double in comparison with the first months after launch (when you could pay something like $1800 for a system with 128 GB). For those who are interested mainly in GPU performance, the NVIDIA system may be preferable. Otherwise, the Ryzen CPU is significantly stronger in multi-threaded performance, much stronger when running applications that are aware of AVX-512, so Strix Halo is preferable when the CPU performance matters. Also for things like video decoding or transcoding, the DGX Spark GPU is likely to be weak.
      - girvo 2mo ago
        >You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed.. I mean that's not as true as you'd think: in fact its the whole benefit IMO of the spark-alikes, and if you're not running TP=2 or above you're sort of wasting it's potential
  - colordrops 2mo ago
    Fair. Honest caveat - I keep seeing "a good instinct" everywhere now. Is this humans acquiring new phrases from Claude? Is there a name for this phenomenon yet?
    - knollimar 2mo ago
      If not I propose HLRF.
      - colordrops 2mo ago
        Nice
  - mycall 2mo ago
    DGX does parallel inference with llamacpp/vLLM much better than AMD GPU at 128GB VRAM.
- cyril-crutches 2mo ago
  If I understand your comment correctly, I think he addresses that in the first few paragraphs: > The DGX Spark runs “DGX OS” but it is in fact just plain old Ubuntu 24.04 with some additions. If you want, you can just install another Linux distribution easily (Fedora works well), although there may be a couple of weird bugs with the Realtek Ethernet driver so the NVIDIA version of the Linux kernel has a couple of patches. Unlike some other ARM devices, the DGX Spark is all ACPI rather than device tree based, so regular Linux builds for arm64 work just fine.
  - cogman10 2mo ago
    Well that is better than what I gleened. I thought I'd read that the GPU needed extra kernel patches to properly work. If it's just the Ethernet driver that seems a lot more appealing. I thought it was device tree as well, so great that it's actual ACPI.
- deleted 2mo ago
  [deleted]
- dllu 2mo ago
  Many other distros almost work out of the box (as in they boot and run without any modifications). The custom kernel patches you mentioned address mainly non-critical bugs such as a bug where the Realtek r8127 stops working after a reboot (but it works if you turn it off and on again) [1] [2]. I'd consider it in a way better state than trying to run other Linux distros on certain device tree-based devices like, say, Qualcomm Snapdragon machines. The regular NVIDIA drivers with the open source kernel modules work just fine. Talos Linux supports DGX Spark since version 1.12 [3]. I also know of people using Fedora and nixOS successfully. [1] https://github.com/NVIDIA/NV-Kernels/compare/ea55925ab430f1e930e25249a33e303c240ff53a...cbaa7deb383e0007520ad09045c1763a8e1aea7e https://github.com/NVIDIA/NV-Kernels/compare/ea55925ab430f1e... [2] https://forums.developer.nvidia.com/t/realtek-r8127-ethernet-nic-disappears-on-every-reboot-only-recovers-with-full-power-cycle/360654/3 https://forums.developer.nvidia.com/t/realtek-r8127-ethernet... [3] https://github.com/siderolabs/talos/issues/12170 https://github.com/siderolabs/talos/issues/12170
  - rbanffy 2mo ago
    Why are those patches not in mainline? They appear to be generic enough not to give out any Nvidia secret sauce.
    - dllu 2mo ago
      iirc the bug that the R8127 patch was addressing was fixed in the R8169 driver upstream in kernel 6.18 and later. Not super sure what other patches there are and what the status of those are...
trentor 2mo ago
I'm genuinely disappointed with my Spark. I don't know how anyone can claim it performs decently with LLMs or diffusion models. Back when I worked in VFX in the early 2000s, we had a saying: "Render time is coffee time" and if you try to run this thing with a usable context size, you'll be drinking a lot of coffee. Most of the optimizations it relies on for inference simply aren't available for training, so it crawls like a snail on almost every model. An RTX 6000 Blackwell would have been the better investment for an AI enthusiasts and for general computing there are cheaper offerings.
- mapontosevenths 2mo ago
  If you bought it for inference you made a mistake. They aren't good at that. Use it to train models and experiment with ML. It's much better at that. If you just want local inference buy a Mac. If you bought early on, like I did, the Spark is probably worth double what you payed now. I think I paid $3,000 retail for mine and the last time I looked they were fetching close to $6k on ebay. I'm not sure if that's still the case, but you can buy a very nice Mac with $6k.
  - trentor 2mo ago
    Mhm... maybe read my full comment?
    - mapontosevenths 2mo ago
      Are you saying that it's slow for training? Sorry, your comment is confusingly worded to me. I've not had any issues in that regard, but I'm working with LLM's not training diffusion models. Are you following one of the Nvidia provided recipes or inventing something on your own? The last time I looked into it they benchmarked very well, but we both know that doesn't always mean much.
      - trentor 2mo ago
        Yes it is slow for training/fine-tuning decently sized LLMs and unusable for diffusion models. What are you training were you get usable speeds?
ramshanker 2mo ago
Yes. Waiting for the Windows Version myseflf. RTX Spark Desktop.
- dijit 2mo ago
  I guess copilot needs all the help it can get?
aftbit 2mo ago
Neat blog! I was intrigued by this bullet point mentioned in passing: >my four hard drive USB 3.2 ZFS raidz2 array with four 24 TB drives Can you speak more about this? Which USB array did you choose? How well does it work? I've been slowly planning a transition away from my power-hungry surplus enterprise gear in the 19" rack towards a smaller, quieter, lower power setup ... but storage is the real kicker right now. I have a 12x18TB array in raidz2 built into a 1U NAS case, and I just can't quite figure out a better way to package something like that. I would need three USB arrays if I want to reuse the existing drives, which I think I do given how expensive storage is today.
- dllu 2mo ago
  It's an Orico 9948C3 with four Seagate Barracuda 24TB drives. They were on sale last year [1]. Unfortunately, the enclosure doesn't work super well on Linux. There is a weird bug where the drives don't enumerate when I boot up my computer. This happens on both my x86_64 AMD machine running Linux, and on the DGX Spark. The solution is... simply power cycle the enclosure a couple of times by toggling the power button on it and then it works. Once all four drives show up in lsblk, I can `sudo zfs import ...` manually. This is really gross and annoying. Replacing the USB cable, flipping the USB-C cable 180 degrees, hot plugging it, etc, all didn't work, both on the DGX Spark and the other Linux machine. I've also read reports of it being unstable in UAS mode on Linux but I haven't found a big difference in stability between enabling UAS or falling back to usb-storage. Once it starts up correctly though, the drives are fast. I store my huge amount of 100 megapixel photos on it. The Seagate Barracudas are helium-filled HAMR/CMR drives and are apparently rebranded/binned Exos drives. They aren't rated for 24/7 use but then neither are the refurbished Exos drives. [1] https://www.reddit.com/r/buildapcsales/comments/1p29pm8/hdd_seagate_barracuda_24tb_st24000dm001_35_hard/ https://www.reddit.com/r/buildapcsales/comments/1p29pm8/hdd_...
  - aftbit 2mo ago
    Yeah... that's been my past experience with USB docks, at least any with more than one slot. I've never had great luck with them. "Can recover from power outage without being touched" is a key requirement for my NAS so I'll give that one a pass and stick with my "SATA drives directly attached to a SAS controller" strategy for now. Thanks for the reply. As for the 24/7 use, yeah so be it. The I in RAID stands for Inexpensive. If they fail after 10 years at 24/7, so be it. I have drive level redundancy and frequent offsite backups of anything critical.
rvz 2mo ago
I would avoid the DGX Spark. For that price and its performance on running local models it is a complete scam. This tweet says it all [0] [0] https://xcancel.com/petergostev/status/1978230978725507108 https://xcancel.com/petergostev/status/1978230978725507108
haunter 2mo ago
This is something I'd do if I've had the disposable income lol >Non-Steam games have a lower chance of working Wonder if it's true for GOG games because they are usually installed in a neatly packaged folder without any bloat.
MrVitaliy 2mo ago
I do appreciate how Nvidia tries to say close to vanilla with Linux and Android (nvidia shield). Instead of trying to build a shitty moat like Samsung with all their garbage software. If nvidia ever releases Android smartphone, I'd probably stand in line to get one.
- dietr1ch 2mo ago
  After leaving a few of great-on-paper SoCs as paperweights I've learnt that I just don't want to deal with anyone's custom platform as I'll eventually be left with an outdated system that's annoying and time-consuming to maintain.
  - deleted 2mo ago
    [deleted]
- m463 2mo ago
  I thought they did that once. hmmm... about 2010-ish: NVIDIA Tegra mobile phones include the Motorola Droid X2, LG Optimus 2X, Samsung Galaxy R, HTCOne X, and Xiaomi Mi 3. https://www.androidauthority.com/nvidia-phones-1234416/ https://www.androidauthority.com/nvidia-phones-1234416/
- ColdStream 2mo ago
  It is funny how they make some really decent hardware but the uptake can be fairly limited. Tegra range is a good example. Yes there is the Nintendo Switch but beyond that it become very limited. Surface RT, Ouya, Nvidia shield and then a few small bits here and there.
jubilee33 2mo ago
This is an interesting review. I have a Chinese strix halo box that's isnt available in the west (favm faex1) I've been able to do some ok graphical gen, or some decent agentic tasks as a fallback for when some of the APIs are overloaded during business hours, but nothing amazing for sure, and also not both at the same time. But here's the thing...it cost me 1800usd two months ago....and it's runs x86. I am struggling to see why people pay +2x more for the Arm Nvidia version, despite the slightly higher bandwidth it still does basically the same AI tasks and alot fewer high end general computing tasks... I like my box but I wouldn't find it useful enough to pay more than I did for it or get more of them and cluster for instance. Can anyone explain the allure of the Nvidia box, other than brand name?
- embedding-shape 2mo ago
  > isnt available in the west [...] Can anyone explain the allure of the Nvidia box The first part might answer the second one. Otherwise, the lack of CUDA and the nvidia ecosystem of tooling could also explain why it doesn't seem so interesting for AI tasks.
  - jubilee33 2mo ago
    But strix halo boxes themselves are available, just not that one. And despite my concerns about what's said about Cuda and ROCm I have never had a problem running any model, for image or text or voice, the community has done great work in making things work. So the point of the question stands. It's also interesting that the most high end Chinese equipment, both prosumer things like these boxes but also the Huawei professional stack is just not available in the places it would be most appreciated. Not sure if thats china tit for tat, or western "we don't want your commie hardware anyways" But for a lot of people it sucks cause nobody should be paying 4.2k for this product. The value isn't there.
    - embedding-shape 2mo ago
      > I have never had a problem running any model, for image or text or voice, the community has done great work in making things work There is a whole world of other tooling and stuff that isn't just for hobbyists to run inference with ML models, but also how to do profiling, debugging and gathering data when you run distributed workloads, and so on. The nsight toolkit seems miles ahead of the competition on other platforms, as just one example.
bullen 2mo ago
I have been running uConsoles with CM5 (2712 and 3588 with 16GB RAM) for 6 months as daily drivers. They are ~$500* and present the same ARM problems/opportunities. But they are completely silent (no fan, the case is the heat sink). My 6600(3050) desktop from 2016(2024) with replaced SSD(2021)/RAM(2025) (they age like milk) now gets little use and M$ will soon sleep with the fishes. *Hard to get now as the 3588 that has linux for uConsole is out of stock and the Raspberry one is rare and more expensive by the day.
- jubilee33 2mo ago
  Got 2 of these early on. They are wonderfully designed, unfortunately I've been caught in the "building things" trap for few months now and they have been relegated to being used as retro gaming/ computing learning boxes for my sons. They really don't appreciate it at all (yet)....but I can hope they will remember it in the future. Teaching them to get to terminal and run the emulator was great fun...reminded me of MSDoS and the hours of troubleshooting to run games with limited memory and drivers back in the day. I just worry that with LLMs the whole point of teaching them basic terminal/troubleshooting skills might be lost soon. We will see.
biddit 2mo ago
Please don’t buy a DGX Spark unless all three of these are true: - You value simplicity more than performance or price-to-performance. - You accept that the hardware will depreciate rapidly. - You’re prepared to buy two or four of them. OR: - You want to run frontier models right now as cheaply as possible - You want to run high-parameter models on a 15a breaker/line Otherwise, get a normal, high-bandwidth GPU. A single Spark gives you roughly 115 GB of usable memory compared with the 24–32 GB found on many lower-cost GPUs. It's certainly a big increase, but in practice it does not unlock dramatically better models. - One Spark: More memory, but mostly enough for poor-quality, extremely low-bit quants of larger models. - Two Sparks: Enough for mid-tier parameter models at reasonable quants, such as DeepSeek V4 Flash and HY3. - Four Sparks: Enough for GLM 5.2 at a reasonable quant. You'll need a $1000+ switch too. The problem is that Sparks are slow compared with almost everything else in their price range. Many factors affect inference speed, but memory bandwidth is one of the biggest. A $4,000-plus DGX Spark provides only 273 GB/s. | GPU | Memory bandwidth | VRAM | Approx. price | | ------------------- | ---------------: | -------------: | ------------: | | DGX Spark | 273 GB/s | ~115 GB usable | $4,000+ | | RTX 5060 | 448 GB/s | 16 GB | $600 | | Radeon AI Pro R9700 | 640 GB/s | 32 GB | $1,200 | | RTX 4000 Pro | 672 GB/s | 24 GB | $2,300 | | RTX 4500 Pro | 896 GB/s | 32 GB | $3,500 | | RTX 3090 | 936 GB/s | 24 GB | $1,200 | | RTX 5000 Pro | 1,344 GB/s | 48 GB | $6,000 | | RTX 5090 | 1,792 GB/s | 32 GB | $4,000 | | RTX 6000 Pro | 1,792 GB/s | 96 GB | $12,000 | Yes, the Spark has substantially more memory. But going from roughly 24 GB to 115 GB does not necessarily unlock substantially better model quality. In many cases, it only lets you load heavily compressed 2-bit versions of larger models, such as DeepSeek V4 Flash, with serious quality degradation. 24–32 GB is currently a sweet spot. Models such as Qwen 3.6 27B and 35B-A3B: - Perform far above what their parameter counts suggest. - Fit comfortably within 24–32 GB of VRAM at reasonable quantization levels. A 4-bit quant of Qwen 3.6 27b (18 GB) will out-perform a 2-bit quant of DeepSeek v4 Flash (90gb). Instead of the Spark, if I had a roughly $4,000 budget... Assuming I already had a reasonably modern desktop: - One RTX 5090, RTX 5000 Pro, or RTX 4500 Pro. - Two RTX 3090s, RTX 4000 Pros, or R9700s, provided the motherboard can bifurcate two physical x16 slots into x8/x8. If I were building a system from scratch: - A DDR4- or PCIe 4.0-era consumer CPU and motherboard that supports x8/x8 bifurcation. - Two RTX 3090s, RTX 4000 Pros, or R9700s. If I were already planning to buy a new Mac: - A MacBook Pro M5 with 64 GB or 128 GB of unified memory. For context, these are the systems I currently run: - EPYC Turin with four RTX 6000 Pro Max-Qs. - EPYC Milan with four RTX 3090s. - AM4 with two RTX 3090s. - AM4 with two RTX 3090s. - Intel Raptor Lake with two RTX 5060 Ti. - MacBook Pro M3 128GB Unified
ciupicri 2mo ago
Somehow related: "The end of my AArch64 [Ampere Altra Q80-30] desktop experiment", https://news.ycombinator.com/item?id=48728599 https://news.ycombinator.com/item?id=48728599 / https://marcin.juszkiewicz.com.pl/2026/06/26/the-end-of-the-aarch64-desktop-experiment/ https://marcin.juszkiewicz.com.pl/2026/06/26/the-end-of-the-...
- dllu 2mo ago
  The Ampere cpu in that post has a "lack of single core CPU speed" despite ripping through compilation tasks with its 80 cores. But the DGX Spark's CPU has fairly decent single core speed and is more like Apple Silicon in this respect.
skolos 2mo ago
"Multi-token prediction gives a free speedup of up to 2x on many models" - at the expense of halving prompt processing speed
- hedgehog 2mo ago
  MTP has no effect on prompt processing.
  - skolos 2mo ago
    Interesting - in my setup (llama.cpp rtx5090 qwen-3.6 27b) prompt processing with mtp is almost half vs non mtp. Sounds like I need to investigate what is wrong.
    - hedgehog 2mo ago
      Maybe enabling MTP causes some weights to be displaced to host memory? MTP itself doesn't do anything during prefill so that should be exactly unchanged, decode will vary depending on settings but with 2-4 proposals depending on workload I've never seen an overall slowdown. edit: I recommend building recent llama.cpp from source, I've been updating about once a week, as there has been a fair amount of work related to MTP recently. If you're running a lot of tool calling on Qwen you might also benefit from one of the bugfixed chat templates like the Froggeric version.
- tingletech 2mo ago
  it works quite will for me in llama.cpp, but I get more like 20% to 40% speedup on tokens per second. I generally use spec-type = draft-mtp,ngram-mod spec-draft-n-max = 4 I have not observed any effect on prompt processing, which is usually an order of magnitude faster than generation on my spark.
ShipVoicedev 2mo ago
Is Nvidia better than Intel
zer0zzz 2mo ago
This is cool. Can I run the same fedora as the asahi Linux I have on my M2 Ultra and use the M2 to do builds and the DGX to launch kernels? That'd be the ideal fully arm at-home coding setup I think.
InTheArena 2mo ago
I have a DGX and a Ryzen AI Max 395 - while I love both of them, there are a few critical things that leave the DGX in use, while the Ryzen "just" is my primary homelab server. The biggest thing is prefil numbers, and the performance impact of higher context sizes. Qwen 27b is a great model, nemotron is decent, gemma is workable. But all of them need reasonable context for reasonable outputs. Unfortunitly, as others have noted, the DGX OS experience... sucks. My hope is that the RTX Spark (which looks to be the exact same stack, sans the high capacity network interface) will help this get a bit more attention, but nVidia's long long long war with the open source community is not helping. Focusing on mainlining kernel support would go a long way to getting the community to be supportive. Of course, a massive regression just hit Linux 7+/7.1 plus for ROCm hosts, so it's just rough everywhere.
- deleted 2mo ago
  [deleted]
enbarca 2mo ago
I purchased my Spark back in March before the RAM price increase. My use case is local self-sovereign AI in Healthcare (https://hcvc.net https://hcvc.net), so I haven't done any gaming. Its been a great driver for Hermes Agent, serving local MoE and dense models up to 122B. It now handles about 90% of my inference load (more complex devops/coding tasks are still routed to cloud). LoRa works well. All my models are open source. There is no NVIDIA dependency in my build. Its also a backend server for my PoC app (https://proforta.com https://proforta.com). Zero issues so far. Extremely reliable and gives me straight path to production for VPS deployment.

102 points by plun9 2mo ago

Super-powerful (if rather pricy) Linux desktop that happens to play games while doing everything else... that man is living my dream :'D

bryanlarsen 2mo ago

It's interesting how many of these issues don't appear to be specific to the DGX Spark but to the standard "Nvidia GPUs suck on Linux" type of issues that afflict a lot of people.

a-dub 2mo ago
it's actually very stable on x86_64 these days, even with optimus.

cogman10 2mo ago

I strongly considered it, but the one thing that scares me away from wanting to do the spark is you basically have to use nvidia's linux (from what I've read) and it doesn't appear the nvidia is interested in upstreaming their kernel changes. I'm avoiding where possible buying electronics where support is controlled by the manufacturer and not me.

willis936 2mo ago
A good instinct. There are a lot of things a $500 AMD GPU can do in linux that a $5000 DGX cannot.
- ARandomerDude 2mo ago
  I know nothing about this topic but your comment piqued my curiosity. What would a $500 AMD GPU do better than DGX?
  - bigyabai 2mo ago
    AMD has Mesa drivers for graphics, which are better-optimized than Nvidia's proprietary Linux Vulkan drivers. It can be fixed in software, but Nvidia's only barely started to catch up. The focus for Nvidia's GPU stack on Linux is getting CUDA working, which means that some traditional raster features get neglected.
    - pjmlp 2mo ago
      NVIDIA also cares about Hollywood and CAD/CAM shops. I learnt three times that AMD isn't necessarily better in Linux as advertised, now I don't care any longer.
      - abe_m 2mo ago
        I'm not aware of any commercial CAD/CAM software running Linux that Nvidia would have a money interest in supporting. Which current CAD/CAM is running on Linux with graphics? The closest I'm aware of it FEA packages allowing remote calculation on Linux machines while the GUI runs on Windows. Is that what you're referring to?
        pjmlp 2mo ago
        BricsCAD, ARES Commander and Platicity come to mind.
      - bigyabai 2mo ago
        A lot of Hollywood studios use renderers that jump straight to CUDA as their backend. Cycles is a popular one, as well as Renderman and Octane.
        pjmlp 2mo ago
        Linux is part of the VFX reference platform, and you forgot Hyperion, for example.
- girvo 2mo ago
  Although one of the ones it cannot do is have/address 128GB of video memory, so it depends on what you want to achieve.
  - seemaze 2mo ago
    AMD Strix Halo was half the price at launch, same 128GB unified memory and nearly identical memory bandwidth. It is x86, so run whichever OS your heart desires. You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed..
    - htrp 2mo ago
      I'm seeing it for 4k and up for similar specs as the spark?
      - adrian_b 2mo ago
        Yes, due to the huge increase in memory price and also exploiting the demand for local LLM inference, the price for a mini-PC with Strix Halo has become more than double in comparison with the first months after launch (when you could pay something like $1800 for a system with 128 GB). For those who are interested mainly in GPU performance, the NVIDIA system may be preferable. Otherwise, the Ryzen CPU is significantly stronger in multi-threaded performance, much stronger when running applications that are aware of AVX-512, so Strix Halo is preferable when the CPU performance matters. Also for things like video decoding or transcoding, the DGX Spark GPU is likely to be weak.
    - girvo 2mo ago
      >You give up 200GB networking, but anyone 'daily driving' one of these things are probably not needing to cluster multiples of them at high speed.. I mean that's not as true as you'd think: in fact its the whole benefit IMO of the spark-alikes, and if you're not running TP=2 or above you're sort of wasting it's potential
- colordrops 2mo ago
  Fair. Honest caveat - I keep seeing "a good instinct" everywhere now. Is this humans acquiring new phrases from Claude? Is there a name for this phenomenon yet?
  - knollimar 2mo ago
    If not I propose HLRF.
    - colordrops 2mo ago
      Nice
- mycall 2mo ago
  DGX does parallel inference with llamacpp/vLLM much better than AMD GPU at 128GB VRAM.
cyril-crutches 2mo ago
If I understand your comment correctly, I think he addresses that in the first few paragraphs: > The DGX Spark runs “DGX OS” but it is in fact just plain old Ubuntu 24.04 with some additions. If you want, you can just install another Linux distribution easily (Fedora works well), although there may be a couple of weird bugs with the Realtek Ethernet driver so the NVIDIA version of the Linux kernel has a couple of patches. Unlike some other ARM devices, the DGX Spark is all ACPI rather than device tree based, so regular Linux builds for arm64 work just fine.
- cogman10 2mo ago
  Well that is better than what I gleened. I thought I'd read that the GPU needed extra kernel patches to properly work. If it's just the Ethernet driver that seems a lot more appealing. I thought it was device tree as well, so great that it's actual ACPI.
deleted 2mo ago
[deleted]
dllu 2mo ago
Many other distros almost work out of the box (as in they boot and run without any modifications). The custom kernel patches you mentioned address mainly non-critical bugs such as a bug where the Realtek r8127 stops working after a reboot (but it works if you turn it off and on again) [1] [2]. I'd consider it in a way better state than trying to run other Linux distros on certain device tree-based devices like, say, Qualcomm Snapdragon machines. The regular NVIDIA drivers with the open source kernel modules work just fine. Talos Linux supports DGX Spark since version 1.12 [3]. I also know of people using Fedora and nixOS successfully. [1] https://github.com/NVIDIA/NV-Kernels/compare/ea55925ab430f1e930e25249a33e303c240ff53a...cbaa7deb383e0007520ad09045c1763a8e1aea7e https://github.com/NVIDIA/NV-Kernels/compare/ea55925ab430f1e... [2] https://forums.developer.nvidia.com/t/realtek-r8127-ethernet-nic-disappears-on-every-reboot-only-recovers-with-full-power-cycle/360654/3 https://forums.developer.nvidia.com/t/realtek-r8127-ethernet... [3] https://github.com/siderolabs/talos/issues/12170 https://github.com/siderolabs/talos/issues/12170
- rbanffy 2mo ago
  Why are those patches not in mainline? They appear to be generic enough not to give out any Nvidia secret sauce.
  - dllu 2mo ago
    iirc the bug that the R8127 patch was addressing was fixed in the R8169 driver upstream in kernel 6.18 and later. Not super sure what other patches there are and what the status of those are...

trentor 2mo ago

I'm genuinely disappointed with my Spark. I don't know how anyone can claim it performs decently with LLMs or diffusion models. Back when I worked in VFX in the early 2000s, we had a saying: "Render time is coffee time" and if you try to run this thing with a usable context size, you'll be drinking a lot of coffee. Most of the optimizations it relies on for inference simply aren't available for training, so it crawls like a snail on almost every model. An RTX 6000 Blackwell would have been the better investment for an AI enthusiasts and for general computing there are cheaper offerings.

mapontosevenths 2mo ago
If you bought it for inference you made a mistake. They aren't good at that. Use it to train models and experiment with ML. It's much better at that. If you just want local inference buy a Mac. If you bought early on, like I did, the Spark is probably worth double what you payed now. I think I paid $3,000 retail for mine and the last time I looked they were fetching close to $6k on ebay. I'm not sure if that's still the case, but you can buy a very nice Mac with $6k.
- trentor 2mo ago
  Mhm... maybe read my full comment?
  - mapontosevenths 2mo ago
    Are you saying that it's slow for training? Sorry, your comment is confusingly worded to me. I've not had any issues in that regard, but I'm working with LLM's not training diffusion models. Are you following one of the Nvidia provided recipes or inventing something on your own? The last time I looked into it they benchmarked very well, but we both know that doesn't always mean much.
    - trentor 2mo ago
      Yes it is slow for training/fine-tuning decently sized LLMs and unusable for diffusion models. What are you training were you get usable speeds?

ramshanker 2mo ago

Yes. Waiting for the Windows Version myseflf. RTX Spark Desktop.

dijit 2mo ago
I guess copilot needs all the help it can get?

aftbit 2mo ago

Neat blog! I was intrigued by this bullet point mentioned in passing: >my four hard drive USB 3.2 ZFS raidz2 array with four 24 TB drives Can you speak more about this? Which USB array did you choose? How well does it work? I've been slowly planning a transition away from my power-hungry surplus enterprise gear in the 19" rack towards a smaller, quieter, lower power setup ... but storage is the real kicker right now. I have a 12x18TB array in raidz2 built into a 1U NAS case, and I just can't quite figure out a better way to package something like that. I would need three USB arrays if I want to reuse the existing drives, which I think I do given how expensive storage is today.

dllu 2mo ago
It's an Orico 9948C3 with four Seagate Barracuda 24TB drives. They were on sale last year [1]. Unfortunately, the enclosure doesn't work super well on Linux. There is a weird bug where the drives don't enumerate when I boot up my computer. This happens on both my x86_64 AMD machine running Linux, and on the DGX Spark. The solution is... simply power cycle the enclosure a couple of times by toggling the power button on it and then it works. Once all four drives show up in lsblk, I can `sudo zfs import ...` manually. This is really gross and annoying. Replacing the USB cable, flipping the USB-C cable 180 degrees, hot plugging it, etc, all didn't work, both on the DGX Spark and the other Linux machine. I've also read reports of it being unstable in UAS mode on Linux but I haven't found a big difference in stability between enabling UAS or falling back to usb-storage. Once it starts up correctly though, the drives are fast. I store my huge amount of 100 megapixel photos on it. The Seagate Barracudas are helium-filled HAMR/CMR drives and are apparently rebranded/binned Exos drives. They aren't rated for 24/7 use but then neither are the refurbished Exos drives. [1] https://www.reddit.com/r/buildapcsales/comments/1p29pm8/hdd_seagate_barracuda_24tb_st24000dm001_35_hard/ https://www.reddit.com/r/buildapcsales/comments/1p29pm8/hdd_...
- aftbit 2mo ago
  Yeah... that's been my past experience with USB docks, at least any with more than one slot. I've never had great luck with them. "Can recover from power outage without being touched" is a key requirement for my NAS so I'll give that one a pass and stick with my "SATA drives directly attached to a SAS controller" strategy for now. Thanks for the reply. As for the 24/7 use, yeah so be it. The I in RAID stands for Inexpensive. If they fail after 10 years at 24/7, so be it. I have drive level redundancy and frequent offsite backups of anything critical.

rvz 2mo ago

I would avoid the DGX Spark. For that price and its performance on running local models it is a complete scam. This tweet says it all [0] [0] https://xcancel.com/petergostev/status/1978230978725507108 https://xcancel.com/petergostev/status/1978230978725507108

haunter 2mo ago

This is something I'd do if I've had the disposable income lol >Non-Steam games have a lower chance of working Wonder if it's true for GOG games because they are usually installed in a neatly packaged folder without any bloat.

MrVitaliy 2mo ago

I do appreciate how Nvidia tries to say close to vanilla with Linux and Android (nvidia shield). Instead of trying to build a shitty moat like Samsung with all their garbage software. If nvidia ever releases Android smartphone, I'd probably stand in line to get one.

dietr1ch 2mo ago
After leaving a few of great-on-paper SoCs as paperweights I've learnt that I just don't want to deal with anyone's custom platform as I'll eventually be left with an outdated system that's annoying and time-consuming to maintain.
- deleted 2mo ago
  [deleted]
m463 2mo ago
I thought they did that once. hmmm... about 2010-ish: NVIDIA Tegra mobile phones include the Motorola Droid X2, LG Optimus 2X, Samsung Galaxy R, HTCOne X, and Xiaomi Mi 3. https://www.androidauthority.com/nvidia-phones-1234416/ https://www.androidauthority.com/nvidia-phones-1234416/
ColdStream 2mo ago
It is funny how they make some really decent hardware but the uptake can be fairly limited. Tegra range is a good example. Yes there is the Nintendo Switch but beyond that it become very limited. Surface RT, Ouya, Nvidia shield and then a few small bits here and there.

jubilee33 2mo ago

This is an interesting review. I have a Chinese strix halo box that's isnt available in the west (favm faex1) I've been able to do some ok graphical gen, or some decent agentic tasks as a fallback for when some of the APIs are overloaded during business hours, but nothing amazing for sure, and also not both at the same time. But here's the thing...it cost me 1800usd two months ago....and it's runs x86. I am struggling to see why people pay +2x more for the Arm Nvidia version, despite the slightly higher bandwidth it still does basically the same AI tasks and alot fewer high end general computing tasks... I like my box but I wouldn't find it useful enough to pay more than I did for it or get more of them and cluster for instance. Can anyone explain the allure of the Nvidia box, other than brand name?

embedding-shape 2mo ago
> isnt available in the west [...] Can anyone explain the allure of the Nvidia box The first part might answer the second one. Otherwise, the lack of CUDA and the nvidia ecosystem of tooling could also explain why it doesn't seem so interesting for AI tasks.
- jubilee33 2mo ago
  But strix halo boxes themselves are available, just not that one. And despite my concerns about what's said about Cuda and ROCm I have never had a problem running any model, for image or text or voice, the community has done great work in making things work. So the point of the question stands. It's also interesting that the most high end Chinese equipment, both prosumer things like these boxes but also the Huawei professional stack is just not available in the places it would be most appreciated. Not sure if thats china tit for tat, or western "we don't want your commie hardware anyways" But for a lot of people it sucks cause nobody should be paying 4.2k for this product. The value isn't there.
  - embedding-shape 2mo ago
    > I have never had a problem running any model, for image or text or voice, the community has done great work in making things work There is a whole world of other tooling and stuff that isn't just for hobbyists to run inference with ML models, but also how to do profiling, debugging and gathering data when you run distributed workloads, and so on. The nsight toolkit seems miles ahead of the competition on other platforms, as just one example.

bullen 2mo ago

I have been running uConsoles with CM5 (2712 and 3588 with 16GB RAM) for 6 months as daily drivers. They are ~$500* and present the same ARM problems/opportunities. But they are completely silent (no fan, the case is the heat sink). My 6600(3050) desktop from 2016(2024) with replaced SSD(2021)/RAM(2025) (they age like milk) now gets little use and M$ will soon sleep with the fishes. *Hard to get now as the 3588 that has linux for uConsole is out of stock and the Raspberry one is rare and more expensive by the day.

jubilee33 2mo ago
Got 2 of these early on. They are wonderfully designed, unfortunately I've been caught in the "building things" trap for few months now and they have been relegated to being used as retro gaming/ computing learning boxes for my sons. They really don't appreciate it at all (yet)....but I can hope they will remember it in the future. Teaching them to get to terminal and run the emulator was great fun...reminded me of MSDoS and the hours of troubleshooting to run games with limited memory and drivers back in the day. I just worry that with LLMs the whole point of teaching them basic terminal/troubleshooting skills might be lost soon. We will see.

biddit 2mo ago

Please don’t buy a DGX Spark unless all three of these are true: - You value simplicity more than performance or price-to-performance. - You accept that the hardware will depreciate rapidly. - You’re prepared to buy two or four of them. OR: - You want to run frontier models right now as cheaply as possible - You want to run high-parameter models on a 15a breaker/line Otherwise, get a normal, high-bandwidth GPU. A single Spark gives you roughly 115 GB of usable memory compared with the 24–32 GB found on many lower-cost GPUs. It's certainly a big increase, but in practice it does not unlock dramatically better models. - One Spark: More memory, but mostly enough for poor-quality, extremely low-bit quants of larger models. - Two Sparks: Enough for mid-tier parameter models at reasonable quants, such as DeepSeek V4 Flash and HY3. - Four Sparks: Enough for GLM 5.2 at a reasonable quant. You'll need a $1000+ switch too. The problem is that Sparks are slow compared with almost everything else in their price range. Many factors affect inference speed, but memory bandwidth is one of the biggest. A $4,000-plus DGX Spark provides only 273 GB/s. | GPU | Memory bandwidth | VRAM | Approx. price | | ------------------- | ---------------: | -------------: | ------------: | | DGX Spark | 273 GB/s | ~115 GB usable | $4,000+ | | RTX 5060 | 448 GB/s | 16 GB | $600 | | Radeon AI Pro R9700 | 640 GB/s | 32 GB | $1,200 | | RTX 4000 Pro | 672 GB/s | 24 GB | $2,300 | | RTX 4500 Pro | 896 GB/s | 32 GB | $3,500 | | RTX 3090 | 936 GB/s | 24 GB | $1,200 | | RTX 5000 Pro | 1,344 GB/s | 48 GB | $6,000 | | RTX 5090 | 1,792 GB/s | 32 GB | $4,000 | | RTX 6000 Pro | 1,792 GB/s | 96 GB | $12,000 | Yes, the Spark has substantially more memory. But going from roughly 24 GB to 115 GB does not necessarily unlock substantially better model quality. In many cases, it only lets you load heavily compressed 2-bit versions of larger models, such as DeepSeek V4 Flash, with serious quality degradation. 24–32 GB is currently a sweet spot. Models such as Qwen 3.6 27B and 35B-A3B: - Perform far above what their parameter counts suggest. - Fit comfortably within 24–32 GB of VRAM at reasonable quantization levels. A 4-bit quant of Qwen 3.6 27b (18 GB) will out-perform a 2-bit quant of DeepSeek v4 Flash (90gb). Instead of the Spark, if I had a roughly $4,000 budget... Assuming I already had a reasonably modern desktop: - One RTX 5090, RTX 5000 Pro, or RTX 4500 Pro. - Two RTX 3090s, RTX 4000 Pros, or R9700s, provided the motherboard can bifurcate two physical x16 slots into x8/x8. If I were building a system from scratch: - A DDR4- or PCIe 4.0-era consumer CPU and motherboard that supports x8/x8 bifurcation. - Two RTX 3090s, RTX 4000 Pros, or R9700s. If I were already planning to buy a new Mac: - A MacBook Pro M5 with 64 GB or 128 GB of unified memory. For context, these are the systems I currently run: - EPYC Turin with four RTX 6000 Pro Max-Qs. - EPYC Milan with four RTX 3090s. - AM4 with two RTX 3090s. - AM4 with two RTX 3090s. - Intel Raptor Lake with two RTX 5060 Ti. - MacBook Pro M3 128GB Unified

ciupicri 2mo ago

Somehow related: "The end of my AArch64 [Ampere Altra Q80-30] desktop experiment", https://news.ycombinator.com/item?id=48728599 https://news.ycombinator.com/item?id=48728599 / https://marcin.juszkiewicz.com.pl/2026/06/26/the-end-of-the-aarch64-desktop-experiment/ https://marcin.juszkiewicz.com.pl/2026/06/26/the-end-of-the-...

dllu 2mo ago
The Ampere cpu in that post has a "lack of single core CPU speed" despite ripping through compilation tasks with its 80 cores. But the DGX Spark's CPU has fairly decent single core speed and is more like Apple Silicon in this respect.

skolos 2mo ago

"Multi-token prediction gives a free speedup of up to 2x on many models" - at the expense of halving prompt processing speed

hedgehog 2mo ago
MTP has no effect on prompt processing.
- skolos 2mo ago
  Interesting - in my setup (llama.cpp rtx5090 qwen-3.6 27b) prompt processing with mtp is almost half vs non mtp. Sounds like I need to investigate what is wrong.
  - hedgehog 2mo ago
    Maybe enabling MTP causes some weights to be displaced to host memory? MTP itself doesn't do anything during prefill so that should be exactly unchanged, decode will vary depending on settings but with 2-4 proposals depending on workload I've never seen an overall slowdown. edit: I recommend building recent llama.cpp from source, I've been updating about once a week, as there has been a fair amount of work related to MTP recently. If you're running a lot of tool calling on Qwen you might also benefit from one of the bugfixed chat templates like the Froggeric version.
tingletech 2mo ago
it works quite will for me in llama.cpp, but I get more like 20% to 40% speedup on tokens per second. I generally use spec-type = draft-mtp,ngram-mod spec-draft-n-max = 4 I have not observed any effect on prompt processing, which is usually an order of magnitude faster than generation on my spark.

ShipVoicedev 2mo ago

Is Nvidia better than Intel

zer0zzz 2mo ago

This is cool. Can I run the same fedora as the asahi Linux I have on my M2 Ultra and use the M2 to do builds and the DGX to launch kernels? That'd be the ideal fully arm at-home coding setup I think.

InTheArena 2mo ago

I have a DGX and a Ryzen AI Max 395 - while I love both of them, there are a few critical things that leave the DGX in use, while the Ryzen "just" is my primary homelab server. The biggest thing is prefil numbers, and the performance impact of higher context sizes. Qwen 27b is a great model, nemotron is decent, gemma is workable. But all of them need reasonable context for reasonable outputs. Unfortunitly, as others have noted, the DGX OS experience... sucks. My hope is that the RTX Spark (which looks to be the exact same stack, sans the high capacity network interface) will help this get a bit more attention, but nVidia's long long long war with the open source community is not helping. Focusing on mainlining kernel support would go a long way to getting the community to be supportive. Of course, a massive regression just hit Linux 7+/7.1 plus for ROCm hosts, so it's just rough everywhere.

deleted 2mo ago
[deleted]

enbarca 2mo ago

I purchased my Spark back in March before the RAM price increase. My use case is local self-sovereign AI in Healthcare (https://hcvc.net https://hcvc.net), so I haven't done any gaming. Its been a great driver for Hermes Agent, serving local MoE and dense models up to 122B. It now handles about 90% of my inference load (more complex devops/coding tasks are still routed to cloud). LoRa works well. All my models are open source. There is no NVIDIA dependency in my build. Its also a backend server for my PoC app (https://proforta.com https://proforta.com). Zero issues so far. Extremely reliable and gives me straight path to production for VPS deployment.