charleshn

6 ms·

1.
by charleshn 3mo ago
Thanks for the article. I have two questions/comments: 1. The N-ring fade-in is quite neat. I guess without the constraint of hash parity rendezvous hashing [0] could have been an elegant approach since it has support for weights (and
2.
Workload isolation using shuffle-sharding(aws.amazon.com)
3 points by charleshn 4mo ago|0 comments
3.
by charleshn 4mo ago
> Some people like to parrot "next token prediction", "LLMs can only interpolate", and other nonsense Thank you for illustrating my point.
4.
by charleshn 4mo ago
Yes, they can. Some people like to parrot "next token prediction", "LLMs can only interpolate", and other nonsense, but it is obviously not true for many reasons, in particular since we introduced RL. Humans do not have
5.
by charleshn 4mo ago
Not sure what you mean: OpenEvolve is an open source implementation of AlphaEvolve: https://huggingface.co/blog/codelion/openevolve
6.
by charleshn 4mo ago
They'll likely make it available at some point, but for now one can use OpenEvolve [0] which is not quite as good but should be a good start to use the same LLM-driven evolutionary framework. [0] https://github.com/algo
7.
by charleshn 8mo ago
> Application pods learn the current assignment through a library called the Slicelet (S for server side). The Slicelet maintains a local cache of the latest assignment by fetching it from the Dicer service and watching for updates. When
8.
What Does a Database for SSDs Look Like?(brooker.co.za)
148 points by charleshn 9mo ago|121 comments
9.
by charleshn 9mo ago
They should be reintroducing the 3D vcache [0] variants (X) in EPYC, with a higher cache/core ratio, that was present in EPYC4 (e.g. 9684X [1]) they for some reason wasn't available in EPYC5. Makes a massive difference at high den
10.
by charleshn 9mo ago
It's fundamentally because of verifier's law [0]. Current AI, and in particular RL-based, is already or will soon achieve super human performance on problems that can be - quickly - verified and measured. So maths, algorithms, etc
11.
by charleshn 9mo ago
> std::hardware_destructive_interference_size Exists so you don't have to guess, although in practice it'll basically always be 64. Unfortunately it's not quite true, do to e.g. spacial prefetching [0]. See e.g. Folly&#x27
12.
by charleshn 10mo ago
> There's a good reason so much research is done on Nvidia clusters and not TPU clusters. You are aware that Gemini was trained on TPU, and that most research at Deepmind is done on TPU?
13.
Collective Communication for 100k+ GPUs(arxiv.org)
1 points by charleshn 11mo ago|0 comments
14.
by charleshn 11mo ago
I can relate. I had tinnitus for over 10 years. My tinnitus was not the usual ringing type, it was some sort of humming, low frequency noise. The frequency was not constant, it could vary. It could sometimes stop for 5-10 minutes, e.g. afte
15.
by charleshn 1y ago
A few questions if the authors are around! > Is hardware agnostic and uses TCP/IP to communicate. So no RDMA? It's very hard to make effective use of modern NVMe drives bandwidth over TCP/IP. > A logical shard is furthe
16.
by charleshn 1y ago
Interesting that neither the article nor the comments mention the CALM theorem [0], which gives a framework to explain when coordination-free consistency is possible, and is arguably the big idea behind SEC. [0] https://arxiv.org
17.
by charleshn 1y ago
You can have a look at the DeepSeek paper, in particular section "2.2 DeepSeek-R1-Zero: Reinforcement Learning on the Base Mode". But generally the idea is that it's, you need some notion of reward, verifiers etc. Works reall
18.
by charleshn 1y ago
> We cannot add more compute to a given compute budget C without increasing data D to maintain the relationship. > We must either (1) discover new architectures with different scaling laws, and/or (2) compute new synthetic data t
19.
by charleshn 1y ago
Yes, 450GB/s is the per GPU bandwidth in the nvlink domain. 3.2Tbps is the per-host bandwidth in the scale out IB/Ethernet domain.
20.
Demystifying NCCL: An In-Depth Analysis of GPU Communication Protocols and Algos(arxiv.org)
1 points by charleshn 1y ago|0 comments
21.
by charleshn 1y ago
Could you check the value of your kernel's net.ipv4.tcp_slow_start_after_idle sysctl, and if it's non zero set it to 0?
22.
by charleshn 1y ago
You can now add getting gold at IMO [0] to the above list. [0] https://x.com/alexwei_/status/1946477742855532918
23.
by charleshn 1y ago
Frontier models went from not being able to count the number of 'r's in "strawberry" to getting gold at IMO in under 2 years [0], and people keep repeating the same clichés such as "LLMs can't reason" or &
24.
by charleshn 1y ago
I mentioned algorithms, not software engineering, precisely for that reason. But the next step is obviously increased formalism via formal methods, deterministic simulators etc, basically so that one could define an environment for a RL age
25.
by charleshn 1y ago
I'm always surprised by the number of people posting here that are dismissive of AI and the obvious unstoppable progress. Just looking at what happened with chess, go, strategy games, protein folding etc, it's obvious that pretty
26.
by charleshn 1y ago
We do already have ASICs, see Google's TPU to get some cost estimates. HBM is also very expensive.
27.
by charleshn 1y ago
You might want to look into TCP BBR [0], it might help. Easy to try on Linux, simple sysctl. [0] https://en.m.wikipedia.org/wiki/TCP_congestion_control#TCP_B...
28.
by charleshn 1y ago
As can be seen from other comments, people tend to focus on the consistency implications, but something not discussed often in the context of distributed systems is that caches tend to introduce bimodality and metastability [0] [1]. See e.g
29.
by charleshn 1y ago
See e.g. https://www.alibabacloud.com/blog/async-fork-mitigating-quer...
30.
by charleshn 1y ago
> Fork on Linux should use copy-on-write vmpages now, so if you fork inside python it should be cheap. No, that's exactly the point I'm making, copying PTEs is not cheap on a large address space, woth many VMAs. You can run a s

More ›