6 ms·

This feels fluff to me on the part of the author (whose work I don’t want to trivialize) but I don’t think they’ve actually looked deeper than a paper spec shee

by dagmx 3mo ago

This feels fluff to me on the part of the author (whose work I don’t want to trivialize) but I don’t think they’ve actually looked deeper than a paper spec sheet on this. 1. Yes it has the same number of cores as a 5070 mobile. It’s also running at a shared peak of 2/3 the bandwidth and a shared peak of 2/3 the TDP. The GPU by itself will likely perform at half the dedicated units performance 2. Apple may not have SVE2 but they do have the AMX (private) and SME. I don’t see why he thinks the SVE2 will give him more performance than the SME. 3. He mentions a single core type but doesn’t mention the total makeup. We already have known for a year how the DGX Spark compares to Apple chips. For CPU it’s roughly equivalent to an M3 Pro and for GPU compute (not rasterization) it’s between an M4 Pro and M4 Max without considering bandwidth. The real advantage to these is that they run CUDA. That’s it. Otherwise when they launch they’ll be 2-3 generations behind where Apple is and 1 gen behind AMD. The other super power of the DGX Spark was the NIC for pairing them together. But that’s been removed here too.

llm_nerd 3mo ago
It is absolutely fluff, and the only reason this worthless tweet is on the front page of HN is that this audience has a habit of canonizing certain people, and treating each of their bowel movements as prophetic. Guy suddenly became aware of a chip that the rest of the industry long knew about, seems completely unaware of the competitors, and posts about how it's a BEAST and will be a GAME CHANGER. Like the DGX Spark was a game changer? Eh, it has mostly been a massive disappointment. An overpriced nvidia laptop isn't going to change the equation an iota.
- trympet 3mo ago
  Yes. This reads like a LinkedIn post rehashing old news. I’m not even in the industry.
storus 3mo ago
> GPU compute (not rasterization) it’s between an M4 Pro and M4 Max without considering bandwidth You are likely thinking about token generation which is dependent on memory bandwidth where Apple has an edge. Spark's GPU compute is way higher than even M5 Max (17 FP32 TFlops), around 2x FP32 TFlops... It's literally 6144 CUDA cores like desktop 5070, slowed down by slow memory and lower TDP (29.7 vs 31 FP32 TFlops on 5070).
- dagmx 3mo ago
  That’s only if you consider FP32 specifically. On average the M5 Max will pull ahead for tasks like GPU raytracing (it’s currently the fastest mobile GPU for Blender rendering) and token generation and other things that benefit from the higher memory bandwidth. I’d also mention that you’re comparing peaks which the RTX Spark won’t be hitting. The top TDP is less than that of the DGX Spark. I just think anyone calling this a beast and a game changer are conflating/extrapolating from different form factors and constraints
  - deleted 3mo ago
    [deleted]
  - well_ackshually 3mo ago
    > fastest mobile GPU for Blender rendering cool story, but nobody cares about mobile GPUs for blender. A 4080 eats an M5 Max alive for breakfast. The 5080 in my machine that cost me 1500€ runs circles around an M5 Max that would cost me over 6000€. And when in 5 years the 5080 isn't enough, I can upgrade it to a 7080 or whatever, which will remain compatible. If you're a professional, soldered products like the RTX Spark or Apple's offering are a dead end. They are literally never worth it.
    - Foobar8568 3mo ago
      And nobody cares about 5080.
      - well_ackshually 3mo ago
        Exactly. A 5080 is just an enthusiast gamer card. If you're part of a large company that requires you to run Blender/3DSMax/etc (read: disney/pixar sized), you're going to have an A6000 in there, or even just a render farm. Game dev & asset work is probably happy with a 5080 and that's what most rendering/dev machines would have. The addressable market of "i have 6000 to blow and i need meh performance on anything related to 3D rendering" is small, and benchmarks make it look bigger than it really is.
wmf 3mo ago
Lemire is very narrowly interested in CPU SIMD so within that niche it may be interesting. As you said, overall the Spark is good but not great.
- oofbey 3mo ago
  I cannot fathom why he brings up CPU SIMD as a potential comparative weakness on the NVIDIA Spark when it has teraflops of CUDA sitting right there.
  - well_ackshually 3mo ago
    Because you won't run your sorting algorithm that runs every frame on a CUDA kernel. CPU performance matters more than however many tflops of CUDA you have under hand as soon as you do silly things like "run an OS" and "use your PC for anything but shitting out tokens"
cthalupa 3mo ago
Prefill is another advantage vs. Apple. It's way way way way faster on a spark than it is even on an m5 max. Same model, same quant, same query, as close to as matched settings as I can get from vllm, and for workloads with large prompts + low cacheability, one of my sparks will often be done responding before the mbp is done with prefill.

by dagmx 3mo ago

llm_nerd 3mo ago

It is absolutely fluff, and the only reason this worthless tweet is on the front page of HN is that this audience has a habit of canonizing certain people, and treating each of their bowel movements as prophetic. Guy suddenly became aware of a chip that the rest of the industry long knew about, seems completely unaware of the competitors, and posts about how it's a BEAST and will be a GAME CHANGER. Like the DGX Spark was a game changer? Eh, it has mostly been a massive disappointment. An overpriced nvidia laptop isn't going to change the equation an iota.

trympet 3mo ago
Yes. This reads like a LinkedIn post rehashing old news. I’m not even in the industry.

storus 3mo ago

> GPU compute (not rasterization) it’s between an M4 Pro and M4 Max without considering bandwidth You are likely thinking about token generation which is dependent on memory bandwidth where Apple has an edge. Spark's GPU compute is way higher than even M5 Max (17 FP32 TFlops), around 2x FP32 TFlops... It's literally 6144 CUDA cores like desktop 5070, slowed down by slow memory and lower TDP (29.7 vs 31 FP32 TFlops on 5070).

dagmx 3mo ago
That’s only if you consider FP32 specifically. On average the M5 Max will pull ahead for tasks like GPU raytracing (it’s currently the fastest mobile GPU for Blender rendering) and token generation and other things that benefit from the higher memory bandwidth. I’d also mention that you’re comparing peaks which the RTX Spark won’t be hitting. The top TDP is less than that of the DGX Spark. I just think anyone calling this a beast and a game changer are conflating/extrapolating from different form factors and constraints
- deleted 3mo ago
  [deleted]
- well_ackshually 3mo ago
  > fastest mobile GPU for Blender rendering cool story, but nobody cares about mobile GPUs for blender. A 4080 eats an M5 Max alive for breakfast. The 5080 in my machine that cost me 1500€ runs circles around an M5 Max that would cost me over 6000€. And when in 5 years the 5080 isn't enough, I can upgrade it to a 7080 or whatever, which will remain compatible. If you're a professional, soldered products like the RTX Spark or Apple's offering are a dead end. They are literally never worth it.
  - Foobar8568 3mo ago
    And nobody cares about 5080.
    - well_ackshually 3mo ago
      Exactly. A 5080 is just an enthusiast gamer card. If you're part of a large company that requires you to run Blender/3DSMax/etc (read: disney/pixar sized), you're going to have an A6000 in there, or even just a render farm. Game dev & asset work is probably happy with a 5080 and that's what most rendering/dev machines would have. The addressable market of "i have 6000 to blow and i need meh performance on anything related to 3D rendering" is small, and benchmarks make it look bigger than it really is.

wmf 3mo ago

Lemire is very narrowly interested in CPU SIMD so within that niche it may be interesting. As you said, overall the Spark is good but not great.

oofbey 3mo ago
I cannot fathom why he brings up CPU SIMD as a potential comparative weakness on the NVIDIA Spark when it has teraflops of CUDA sitting right there.
- well_ackshually 3mo ago
  Because you won't run your sorting algorithm that runs every frame on a CUDA kernel. CPU performance matters more than however many tflops of CUDA you have under hand as soon as you do silly things like "run an OS" and "use your PC for anything but shitting out tokens"

cthalupa 3mo ago

Prefill is another advantage vs. Apple. It's way way way way faster on a spark than it is even on an m5 max. Same model, same quant, same query, as close to as matched settings as I can get from vllm, and for workloads with large prompts + low cacheability, one of my sparks will often be done responding before the mbp is done with prefill.