davmre

8 ms·

1.
by davmre 12d ago
> a team of agents completed the proof in a little under two weeks, consuming about six billion output tokens from a general-purpose internal research model roughly comparable to Claude Fable 5.1. At $50/M output tokens, this would
2.
by davmre 15d ago
The watermark lives in the entropy of sampled outputs. Typical entropy of sampled English text is about 1 bit/token, meaning that a 500-token response from a given model might have 2^500 potential outputs of roughly equal probability.
3.
by davmre 1mo ago
That depends how much text you're compressing. For example, LLM pretraining datasets are on the order of tens or hundreds of terabytes. If an LLM-based code for that data is ~twice as efficient as gzip, you could afford to transmit the
4.
by davmre 1mo ago
Any compressor actually can be used, trivially, as an autoregressive language model. Given a context (for LLMs, this would include the entire pretraining dataset, plus the prompt), you compress `context + next_token` for every possible next
5.
by davmre 3mo ago
This sounds more or less unavoidable? Decompilers are inherently security-sensitive. If you take avoiding cyberattack uplift seriously as a goal, I don't see how you get around essentially refusing to work on them. Obviously there are
6.
by davmre 5mo ago
The KV cache consists of activation vectors for every attention head at every layer of the model for every token, so it gets quite large. ChatGPT also estimates 60-100GB for full token context of an Opus-sized model: https://chat
7.
by davmre 8mo ago
Why can't it be both?
8.
by davmre 10mo ago
It's true that these are very different activities, but I think most ML researchers would agree that it's actually the creation of ImageNet that sparked the deep learning revolution. CNNs were not a novel method in 2012; the novel
9.
by davmre 1y ago
There actually has been some recent work on digitizing smell, most notably Osmo, which was founded by some ex-Google ML researchers: https://www.salon.com/2025/01/05/digital-smell-has-arrived-a...
10.
by davmre 1y ago
Even if you charge $10/hr, or whatever the market rate would be for street parking spots, you still need an enforcement mechanism to prevent people overstaying. In general, the idea of a "market rate" for any given property d
11.
by davmre 1y ago
Senators, especially committee chairmen, have quite a bit of implicit leverage, beyond the direct leverage of subpoenas or directly cutting funding to an offending agency. Any given Senator is to some extent constantly in a favor-trading ga
12.
by davmre 1y ago
If you enjoyed this article then you must watch the A Capella Science music video on the same subject: https://youtube.com/watch?v=TzN148WQ2OQ By far the catchiest song about eel mating you will encounter today.
13.
by davmre 1y ago
The public interest is in judging the trial process , not in judging the defendant. Suppose the government charges you with murder, searches your house, and finds your sex toy collection. At trial they present some elaborate thesis about h
14.
by davmre 1y ago
Regardless of the effects, I don't think the case against MS was brought with the intent to "punish" MS through the trial process. The government brought the case because it thought it could win, it did win, and a judicial re
15.
by davmre 1y ago
Sure, there's a strong public interest in having proceedings on record. US civil cases are supposed to have a presumption of openness, which the judge weighs against other interests, like protecting trade secrets, confidential business
16.
by davmre 1y ago
If there's a general standard of transparency applied to all companies, fine. There are costs to increasing transparency, but certainly you could argue for that policy. The argument that we should cheer on the use of government power t
17.
by davmre 1y ago
> The government doesn't have to win an antitrust trial in order to create competition. As the saying goes, "the process is the punishment." Regardless of what you think of Google or this case specifically, this is an argu
18.
by davmre 1y ago
Scholarly consensus is that the "Gospel of Matthew" was not written by the apostle Matthew and the "Gospel of John" was not written by the apostle John: https://en.wikipedia.org/wiki/Gospel_of_Matthe
19.
by davmre 2y ago
They're not proposing to apply tensor decomposition to an existing collection of weights. It's an architecture in which the K, V, and Q tensors are constructed as a product of factors. The model works with the factors directly and
20.
by davmre 2y ago
You're totally right there must be supervision; it's just a matter of how the term is used. "Supervised learning" for LLMs generally means the system sees a full response (eg from a human expert) as supervision. Reinforc
21.
by davmre 2y ago
Backprop itself doesn't invert the computation, but it does give you the direction for an incremental move towards the inverse (a 'nudge' as the article puts it). That is, given a sufficiently nice function f and an appropria
22.
by davmre 3y ago
Nitpicking, but for a technical audience it's worth noting that ibogaine is not at all a 'potent' psychedelic in the pharmacological sense of the term. A typical therapeutic dose is on the order of 500mg, which makes ibogaine
23.
by davmre 3y ago
> Altman was initially going to cooperate and even offered to help, until Brian Chesky & Ron Conway riled him up I don't think the article supports this. All we know is that sama appeared cooperative when the board fired him.
24.
by davmre 3y ago
Counterpoint: SSRIs were transformative for my depression. Side effects were minor and manageable (eg, Wellbutrin worked well to prevent any sexual dysfunction). I was on them for several years and had no problem tapering off. My understand
25.
by davmre 3y ago
Prophet has gotten a lot of attention since being released in 2017, I think because the idea of a fully automatic solution is very appealing to people. One of the original developers, Sean Taylor, recently posted a nice retrospective on the
26.
by davmre 3y ago
Before trying MDMA I had mostly written it off as a feel-good drug: an artificial high like cocaine or meth, useful only for escapism. Why bother chasing that sort of experience? But now having tried most of the commonly used psychedelics,
27.
Hey, why aren't we doing more research about spiritual awakening(sashachapin.substack.com)
8 points by davmre 3y ago|8 comments
28.
by davmre 7y ago
For what it's worth, Jax (github.com/google/jax) now lets you use XLA to compile Numpy code. It'd be cool to see how that would stack up in a modern comparison.
29.
by davmre 8y ago
I've been on the PhD admissions committee for the Berkeley CS PhD program. Students in many areas are, for all practical purposes, admitted by an advisor. It's not a formal arrangement in the same sense as European programs, but n
30.
by davmre 8y ago
That per-unit development cost is almost certainly including the cost of acquiring the land. Labor in SF is expensive, but not that expensive.

More ›