refibrillator

5 ms·

1.
by refibrillator 12d ago
Proving FLT was such a profoundly emotional and spiritual experience for Andrew Wiles, it almost brought a tear to my eye: https://news.ycombinator.com/item?id=49203626 It is truly saddening to think that machines will depr
2.
by refibrillator 14d ago
So OpenAI employees run massively distributed CyberGym evals on an unpublished and “unaligned” model. For days the agent swarm communicates via their internal infra, even crashing Artifactory where 95% of messages were being passed through,
3.
Brain motion is driven by mechanical coupling with the abdomen(nature.com)
3 points by refibrillator 4mo ago|0 comments
4.
by refibrillator 4mo ago
All the author’s comments here are straight from Claude too :/
5.
by refibrillator 4mo ago
Previous discussion here (with links to actual primary source): https://news.ycombinator.com/item?id=48023079 No technical report published yet, unlikely code or weights will be either given VC funding.
6.
by refibrillator 8mo ago
Hmm in distributed computer systems similar patterns exist, e.g. adding jitter to avoid thundering herd effects. This feels like an essential pattern of the universe or something…
7.
by refibrillator 9mo ago
Sometimes I wonder if anyone else feels there is a halo effect around certain personalities on this site. When I see someone ending nearly every comment with a link to their blog or pet project, it gives me bad vibes, as if they have ulteri
8.
by refibrillator 9mo ago
Here’s a starting point: https://pmc.ncbi.nlm.nih.gov/articles/PMC5241507/#B1 TLDR: NAC is a derivative of an amino acid called cysteine, as such it is a precursor for one of the most important antioxidants in the
9.
by refibrillator 9mo ago
No disrespect but paying to verify age feels absurd, let alone putting a private company in charge of what should be an essential function of the government. How about when you turn 18 or whatever the government gives you a signed JWT that
10.
by refibrillator 9mo ago
H100 has 80 GB of HBM3 . There’s only like 37 MB of SRAM on a single chip.
11.
by refibrillator 9mo ago
Fascinatingly, the body already has a mechanism for this: fasting. One of the many beneficial side effects is rapid mucosal atrophy, decreasing villus height and crypt depth. You can find evidence of this in the literature, but it’s absurdl
12.
by refibrillator 9mo ago
This is hilarious, I don’t even want to know if it’s legit.
13.
by refibrillator 9mo ago
Love anecdotes like this! But admittedly I feel a bit lost, so please forgive my ignorance when I ask: why does choosing a subset of k integers at random require deduplication? My naive intuition is that sampling without replacement can be
14.
by refibrillator 10mo ago
One of the cooler and lesser known features of JPEG XL is a mode to losslessly transcode from JPEG while achieving ~20% space reduction. It’s reversible too because the original entropy coded bitstream is untouched. Notably GCP is rolling t
15.
by refibrillator 10mo ago
Yeah it’s pretty clearly a bot account, or at least someone who likes to copy paste from chatgpt to sound smart.
16.
by refibrillator 10mo ago
> It works better! > I strongly believe it is one of the best technologies for AI agents Do you have any quantitative evidence to support this? Sincere question. I feel it would add some much needed credibility in a space where many f
17.
by refibrillator 1y ago
Ha made me chuckle. For those wondering seriously about this, it’s not a viable optimization because weights are not readily compressible via JPEG/DCT, and there are a limited number of these units on the chip which bottlenecks through
18.
by refibrillator 1y ago
Great exposition, loved the touch of humor. Please do the backward pass when it’s published. As a fellow Tri Dao groupie and lucky duck who gets to build on Hopper/Blackwell clusters, I find it amazing how difficult it is becoming to w
19.
by refibrillator 1y ago
Well “import torch” for example will resolve certain dynamically linked symbols, which must be done first before importing your own .so code that uses libtorch and pybind11. If not you will get a super fun to debug segfault, leaving you sta
20.
by refibrillator 1y ago
Tokenization is typically done on CPU and is rarely (if ever) a bottleneck for training or inference. GPU kernels typically dominate in terms of wall clock time, the only exception might be very small models. Thus the latency of tokenizatio
21.
by refibrillator 1y ago
Hi author(s), the on-GPU interpreter approach looks like a promising path forward, have you seen this strikingly similar concurrent work? https://news.ycombinator.com/item?id=44111673 I find it curious that fundamentals of
22.
by refibrillator 1y ago
The code has few comments but gotta love when you can tell someone was having fun! https://github.com/ScalingIntelligence/tokasaurus/blob/65efb... I’m honestly impressed that a pure python implementation can
23.
by refibrillator 1y ago
> Unsloth Dynamic GGUF which, quality wise in real-world use performs very close to the original How close are we talking? I’m not calling you a liar OP, but in general I wish people perpetuating such broad claims would be more rigorous.
24.
by refibrillator 1y ago
Note to others reading along: in the last appendix page the OP paper reports DFloat11 reduces tokens/sec by ~2-3x for the Llama-3.1-8b and Qwen-2.5-14b/32b and Mistral-small-24b models (throughput penalty not reported for others).
25.
by refibrillator 1y ago
> the actual processing happens in 17B This is a common misconception of how MoE models work. To be clear, 17B parameters are activated for each token generated . In practice you will almost certainly be pulling the full 109B parameters
26.
by refibrillator 2y ago
vLLM supports MLA for Deepseek models as of 3 weeks ago. 3x higher generation throughput and 10x token memory capacity. https://github.com/vllm-project/vllm/releases/tag/v0.7.1 MHA is still faster in low
27.
by refibrillator 2y ago
Pay attention to IO bandwidth if you’re building a machine with multiple GPUs like this! In this setup the model is sharded between cards so data must be shuffled through a PCIe 3.0 x16 link which is limited to ~16 GB/s max. For refere
28.
by refibrillator 2y ago
In theory no, a network partition is indistinguishable from infinite latency. Though in common use I think “latency” tends to imply a response and measurable timing.
29.
by refibrillator 2y ago
CAP theorem is great, though it does omit latency. Which leads you to the logical extension, PACELC [1]: If there is a network Partition you must choose Availability or Consistency, Else the tradeoff is Latency vs Consistency. This offers p
30.
by refibrillator 2y ago
I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google Brain and now at Anthropic. Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep

More ›