ansk

8 ms·

1.
by ansk 6d ago
I don't know enough about the specific models they're comparing against to say this definitively, but it looks to me like they're comparing their pre-trained models with others' post-trained models. The metric upon which
2.
by ansk 2mo ago
I can see how LLMs may make compiler development seem futile when it comes to optimizing the local graph structure (ie kernel-sized chunks of the computation). But optimizing the global structure -- things like defining the boundaries betwe
3.
by ansk 2mo ago
This kind of sounds like just giving up on making the compiler better.
4.
by ansk 2mo ago
I skimmed the doc and my impression is that your second listed interpretation -- DeepSeek is pausing investment because of a leak -- is the more correct one. There's quite a bit of confidential information in the doc about the company
5.
by ansk 3mo ago
I've found breathing exercises to be effective for the duration of the exercise, but I'm more interested in the possibility of training myself to adjust my respiration patterns over sustained durations. Would it be beneficial --
6.
by ansk 3mo ago
Kind of a backwards take, both in your interpretation of this announcement and the company in general. The point of this announcement is to draw attention to the fact that the currently hyped topic is what they have been working on since th
7.
by ansk 4mo ago
I can assure you that a fully deterministic and equally effective claude is possible to build. And yes, that would mean identical prompts would yield 100% identical output 100% of the time. It would still make the occasional logical or f
8.
by ansk 4mo ago
I see what you're getting at, but determinism isn't the right word either. LLMs are fundamentally deterministic -- they are pure functions which output text as a function of the input text and the network parameters[1]. Dependin
9.
by ansk 7mo ago
The guy writing a thumbnail pipeline isn't getting petabytes (exabytes?) of storage to cache all videos from the past week in their entirety. If this quantity of data is being stored, it's being stored deliberately and at signifi
10.
by ansk 7mo ago
The other explanations here don't explain the long delay between the start of the investigation and the release of the footage. Yes, storing customer data is what we'd expect from Google and yes, the FBI can coerce Google to prov
11.
by ansk 8mo ago
The implication that OpenAI is a YC company in the same sense as the other listed companies is somewhere between misleading and dishonest. Even more distasteful to show founding teams for all the others, then just Sam for OpenAI.
12.
by ansk 11mo ago
Of all Schmidhuber's credit-attribution grievances, this is the one I am most sympathetic to. I think if he spent less time remarking on how other people didn't actually invent things (e.g. Hinton and backprop, LeCun and CNNs, et
13.
by ansk 11mo ago
I can only imagine what the Taiwanese can do in Arizona. Truly a synergy for the ages.
14.
by ansk 1y ago
My personal experience is that the cost of enduring a negative stimulus is not simply a function of the magnitude of the negative stimulus, but rather the magnitude of the negative stimulus in relation to the magnitude of all other concurre
15.
by ansk 1y ago
For future reference, if you want proper python bindings for ffmpeg* you should use pyav. * To be more precise, these are bindings for the libav* libraries that underlie ffmpeg
16.
by ansk 1y ago
And on the seventh day, God ended His work which He had done and began vibe coding the remainder of the human genome.
17.
by ansk 2y ago
Key-based attention is not attributable to the Transformer paper. First paper I can find where keys, queries, and values are distinct matrices is https://arxiv.org/abs/1703.03906 , described at the end of section 2. T
18.
by ansk 2y ago
The scientific impact of the transformer paper is large, but in my opinion the novelty is vastly overstated. The primary novelty is adapting the (already existing) dot-product attention mechanism to be multi-headed. And frankly, the singl
19.
by ansk 2y ago
Tensorflow is a lot like IBM -- it deserves praise not because it's great in its current state, but for its contributions towards advancing the broader technological front to where it is today. Tensorflow walked so JAX could run, so t
20.
by ansk 2y ago
I've seen and ignored a lot of "pytorch good, tensorflow bad" takes in my time, but this is so egregiously wrong I can't help but chime in. Facilitating graph-level optimizations has been one of the most central tenets
21.
by ansk 2y ago
I think that while researchers would agree that the empirical success of deep learning has been remarkable, they would still agree that the language used here -- "an algorithm that could really, truly learn any distribution of data (or
22.
by ansk 2y ago
> humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data) He's hand-waving around the idea presented in the Universal Approxi
23.
by ansk 2y ago
Imagine reading a math or programming textbook where each statement was true with probability 0.95.
24.
by ansk 2y ago
Is batched inference for LLMs memory bound? My understanding is that sufficiently large batched matmuls will be compute bound and flash attention has mostly removed the memory bottleneck in the attention computation. If so, the value prop
25.
by ansk 3y ago
Furthermore, each of those 16 channels would typically be mutibyte floats as opposed to single byte RGB channels. (speaking generally, haven't read the paper)
26.
by ansk 3y ago
Apparently my innuendo has not been taken well, so I'll clarify in more straightforward terms: @abidlabs is the employer of the individual running the aforementioned twitter account.
27.
by ansk 3y ago
Do you by chance have a vested interest in attesting this? A disclaimer might be appropriate if so.
28.
by ansk 3y ago
So glad the AK account exists. As a researcher, I've always wanted some guy with an econ degree and a year of ML eng to recommend me papers after glancing at them for maybe 30 seconds. I am genuinely baffled that researchers in the fi
29.
by ansk 3y ago
When I open a large pdf on arxiv (100+ MB, not uncommon for ML papers focused on hi-res image generation), there is a significant load time (10+ seconds) before anything is rendered at all other than a loading bar. Does anyone know what th
30.
by ansk 3y ago
> a new kind of AI R&D lab which creates practical end-user products based on foundational research breakthroughs This isn't new and if anything it's the de facto standard for just about every AI research lab these days. O

More ›