5 ms·

I just built a workstation with a 32 core Threadripper Pro, 128 GB ECC RAM, Thunderbolt 4, and an RTX 4090 for $6,500.

• sliken 3y ago
  Likely better on any workloads that are GPU heavy and fit in 24GB of vram. Two data points from Apple: a) M2 Ultra 24 cores, 128GB ram, 2TB storage = $5,200 b) M2 ultra 24 cores, 192GB ram, 4GB storage = $6,600 Likely 1/4th the size, 1/4th the power consumption, and 4x the ram bandwidth. Have you by chance played with any LLMs? Just saw a post that someone managed 5 tokens/sec with the llama 65B model.
  • USB5 3y ago
    I don't care about playing with LLMs, so this is simply meaningless to me.
    • kamranjon 3y ago
      But for many people training or running inference on large models is important and the apple approach actually makes that feasible in a desktop solution which is pretty awesome.
  • deleted 3y ago
    [deleted]