shenberg

6 ms·

1.
by shenberg 2mo ago
We take a lot of shortcuts when speaking, it's actually much harder to transcribe phonemes than to transcribe words, even when aware of the language being spoken. Some models have been trained for the task (e.g. look at https:/&#
2.
by shenberg 3mo ago
You always either go left or down so total 40 steps, choose 20 to be down (or 20 to be right)
3.
by shenberg 3mo ago
Some example >1B companies off the top of my head: DataDog, Sentry, Snowflake, Okta, MongoDB
4.
by shenberg 3mo ago
moondream is a beast
5.
by shenberg 3mo ago
Seems 100% AI generated and automated, the judge also seems suspect - in the first one it's actually GPT-5.5 pro which has the correct email RE: the deepseek one will match a@b.com1 as "a@b.com" while 5.5 will correctly requi
6.
by shenberg 4mo ago
11% MFU does not mean 89% of GPUs are idle, it means that they're using the GPUs ineffectively.
7.
by shenberg 7mo ago
Using existing enterprise apps probably - this solution is scalable for the vendor and it's easier to sell using existing software as-is than to start out by writing new custom tools.
8.
by shenberg 7mo ago
Mid-way I realized this was AI writing (took me a while), then I read a quote in the text about a comment that "The tragedy isn’t that they cheated; it’s that the system was designed to let them thrive for a decade before anyone bother
9.
by shenberg 7mo ago
Moshi was an amazing tech demo, building the entire stack from scratch in 6 months with a small team was an amazing show of skill: 7B text LLM data + training, emotive TTS for synth data generation (again model + data collection), synth dat
10.
by shenberg 7mo ago
Location: Paris, France (US citizen, EU resident) Remote: Yes Willing to relocate: Not until 2027 Technologies: ML / DS: PyTorch, CUDA, distributed training & inference, performance profiling/optimization (audio & speech f
11.
by shenberg 7mo ago
There are two ingredients that don't fit in the "attention-is-kernel-smoothing" as far as I can tell: positional encoding and causal masking (another way to say positional encoding, I guess) Also, Simplical attention is prett
12.
by shenberg 8mo ago
I don't understand how using group-theory language to describe number-theoretic properties provides extra insight in this case (e.g. conjecture: all perfect numbers are even is more concise than the group-theoretic description given in
13.
by shenberg 9mo ago
ssh exe.dev works
14.
by shenberg 10mo ago
The short and unsatisfying answer is that an LLM generation is a markov chain, except that instead of counting n-grams in order to generate the posterior distribution, the training process compresses the statistics into the LLM's weigh
15.
by shenberg 11mo ago
When countries like North Korea, which depends on cybercrime to fund itself, are signatories, you have to wonder whether this agreement means what its title says.
16.
by shenberg 1y ago
The reality of meetings in most places I've seen is that key stakeholders have already formed an opinion beforehand, the meeting is a place to disseminate decisions that have already been made and align the organization.
17.
by shenberg 1y ago
When I read "51% fewer false positives" followed immediately by "Median comments per pull request cut by half" it makes me wonder how many true positives they find. That's maybe unfair as my reference is automated t
18.
by shenberg 2y ago
The DeepSeek v3 model had a net training cost of >$5m for the final training run, the paper lists over 100 authors[1], meaning highly-paid engineers. This is also one of a sequence of models (v1, v2, math, coder) trained in order to buil
19.
by shenberg 2y ago
That's really not true, e.g. the wikipedia page on population transfer in the Ottoman empire[1]. This dates way back to the Assyrian and Persian empries explicitly moving conquered peoples around in their empires in order to safeguard
20.
by shenberg 2y ago
Anecdotally, a pro-audio software company I worked with had to fire 1/3 of the company when their copy-protection was cracked and sales tanked immediately afterwards, and recovered once a new copy-protection scheme was developed and ap
21.
by shenberg 2y ago
Under the leaderboard tab, if the "Solution" column has an icon, it's clickable. 2nd place solution is by Jeremy Howard (of fast.ai fame), which I'd summarize as TrueSkill Through Time (Microsoft Research paper) + some o
22.
by shenberg 2y ago
The CLIP plot (Fig. 2) is damning, however some of the generative models show flat responses in Fig. 3 (e.g. Adobe GigaGAN, DALL-E-mini). While those are on the one hand technically linear relationships, but are also exactly what we'd
23.
by shenberg 3y ago
I would have expected a sham-treatment arm to the experiment, because how do you differentiate between "an intervention 30 minutes beforehand caused improved learning" and "our specific intervention 30 minutes beforehand caus
24.
by shenberg 3y ago
I suspect that weight initializations are geared towards inputs being normal random variables with mean 0 and variance 1. Deviating from that makes the learning process unhappy.
25.
by shenberg 3y ago
"since 2020, the US has printed nearly 80% of ALL US Dollars in circulation" - I've seen this notion repeated and I assume it's a reference to M1 as published by FRED: https://fred.stlouisfed.org/series&#
26.
by shenberg 3y ago
If you're seriously suggesting that attacking unarmed civilians intentionally, killing parents in front of their children and then kidnapping the children, slaughtering defenseless party-goers, etc. is what I any resistance movement wo
27.
by shenberg 3y ago
There's a confusion this article isn't helpful with: there are physical electrons, the actual physicalparticles. They move in the metal very slowly. But, their motion propagates very quickly, and turns out that the change in moti
28.
by shenberg 3y ago
Prefixes are modifiers to specific instructions executed by the processor, e.g. to control the size of the operands or enable locking for concurrency.
29.
by shenberg 3y ago
In terms of practical algorithms, the Strassen algorithm (O(n^2.8)) is the only one that has runtime advantages for matrix sizes that aren't enormous, and even then, it's not always used because it has two non-trivial costs: reduc
30.
by shenberg 3y ago
Really reminds me of this short story by Ted Chiang: https://waldyrious.neocities.org/ted_chiang/liking-what-you-...

More ›