nilkn

6 ms·

1.
by nilkn 3d ago
Once they observe real safety risks, they're going to slow down on their own, because a major safety incident would destroy their valuations and businesses. The beauty of this principle is that it applies to everyone all over the world
2.
by nilkn 3d ago
None of these incidents involve single instances of commercially or publicly available systems. They all involve large swarms of internal models. The stuff you're describing is not the research frontier. It's really not even close
3.
by nilkn 5d ago
The work is either: (1) the act of concealment: generating an AI book that cannot be detected as AI generated; or (2) actually just writing the book yourself.
4.
by nilkn 8d ago
I think math is compressible structure. That's why we care about something like the Riemann hypothesis but, to use Tao's example, we really couldn't care less about computing the 10^10^10th digit of pi. The first compresses a
5.
by nilkn 8d ago
It's easy to come up with new open problems. It's hard to come up with new open problems that seem to teach us something fundamentally new about the world. Our current batch of problems went through a complex selection process ove
6.
by nilkn 13d ago
What's just as interesting is this morning Axiom Math announced 212 and OpenAI then appears to have rushed out their 186 announcement just 1-2 hours later followed by Astra. Did they accelerate the release of Astra itself? Not necessar
7.
by nilkn 17d ago
My most controversial opinion by far in tech circles is that I still just use a standard Windows gaming PC as my home desktop. My current machine I just bought pre-built from Microcenter, complete with a 5090 and everything. I can fire up a
8.
by nilkn 27d ago
The rank of a rational elliptic curve can be seen as a measure of how complex its arithmetic structure is. Roughly speaking, you can imagine that a curve of rank r has a substructure of dimension r. So a curve with a high rank is a pretty e
9.
by nilkn 27d ago
I don't find AGI to be a useful technical term, as nobody can agree on what it means. For instance, you used it at least five times here, but you never defined it, and I could point to intellectually credible people who would say we&#x
10.
by nilkn 28d ago
> I didn't say anything about low-value output. False. You very plainly did. You simply used the term “junk” instead. > That's baseless speculation. It might be speculation (as is much of what you’re writing), but it’s not b
11.
by nilkn 28d ago
It's not really quite that simple. You wouldn't hire Jeff Dean to do bug fixes in your mobile app. I'm sure he's capable, but I honestly doubt he'd stay interested and focused on it enough to really do a good job. T
12.
by nilkn 28d ago
It could just as easily be the opposite: it could end up being far easier to reasonably direct and evaluate the research direction and output of AI systems than human mathematicians, who are forced to specialize over decades and essentially
13.
by nilkn 28d ago
> There are vast libraries full of mathematics no one (in this discussion, anyway) has ever heard of that no one reads anymore and has never been applied to anything. And that's an issue why? It would seem to me that producing that
14.
by nilkn 28d ago
Yes, of course we'd need a way to evaluate it. I don't right now have a fully conceived answer to what that will look like. But I'm confident at least in saying we would not evaluate it, like Tao is suggesting, by only accept
15.
by nilkn 28d ago
Math that humans don't understand but nonetheless allows AI systems to develop breakthroughs in various fields of science, technology, physics, engineering, medicine, etc., would have great value to humanity even if it doesn't hel
16.
by nilkn 28d ago
We have thousands of years of precedent that suggests that breakthroughs in mathematics tend to accumulate into broader technology breakthroughs in other domains. Why does this tend to be the case, even when some of the smartest people in t
17.
by nilkn 28d ago
I believe this rule of thumb will come to fail. The combination of superhuman mathematical reasoning and synthesis in upcoming AI models plus the rapid build-out of scalable formal verification infrastructure means this exponential in math
18.
by nilkn 1mo ago
In some cases you're right, but I think that's often a symptom of mathematics in Lean being relatively immature (i.e., it will get much easier with time). Even then, verifying the statement in Lean is correct is still much easier
19.
by nilkn 1mo ago
I'm just barely old enough to have experienced a couple waves of technology being introduced and creating these sorts of life optimizations. It almost never actually improves quality of life. It just makes life more optimized, and sinc
20.
by nilkn 1mo ago
This is the Lean proof that a nonsofic group exists (34,440 lines): https://github.com/openai/ten-proofs/blob/main/NonSoficGroup... This is an extraction from that of the actual theorem statement (39 lin
21.
by nilkn 2mo ago
I find people's view is based on how much they're personally "winning" financially. If they're making money, then daily life seems more about building. If they're not making money, then the oppressive cost of l
22.
by nilkn 2mo ago
Couldn't care less. I'm not reliant on anything you just imagined I'm reliant on that supposedly doesn't exist, yet somehow I'm reliant on it. I use the Vision Pro for Mac Virtual Display, movies / entertainmen
23.
by nilkn 2mo ago
The overwhelming majority of people don't want any VR device available today at all. I'm perfectly fine being the minority for now. You say that as if you think I'd have a problem with that. That does absolutely nothing to sw
24.
by nilkn 2mo ago
For me, the economics are the exact opposite. I'd pay $1000 for a good gaming VR headset, but not $4000 -- because gaming is just a hobby that isn't really worth that much to me. Gaming VR headsets have been around for well over a
25.
by nilkn 2mo ago
Yes, this is a challenge, but I don't think it's as damning as it sounds. First, the same kind of issue exists with any kind of testing. We all know that having unit tests and integration tests doesn't mean the software is co
26.
by nilkn 2mo ago
You are free to bow out. However, we are decidedly not nitpicking over minutiae. I don't think you have understood this thread, and that's leading you to make very confident assertions that are individually correct but just not re
27.
by nilkn 2mo ago
You gave an example of Linux having value, which is not contested and not relevant. You did not give any examples showing the existence of an exponential here. Moreover, the value being discussed is long-term geopolitical leverage resulting
28.
by nilkn 2mo ago
It's well understood at this point that China's results are simply not purely the result of distillation. They're the result of significant investment with a willingness not to make a profit any time soon. That's why dis
29.
by nilkn 2mo ago
You can disagree, but simply stating disagreement isn't substantive. Linux can be enormously important while also not being subject to exponential value creation through improvements to itself.
30.
by nilkn 2mo ago
Linux is not a relevant comparison, because the fundamental capabilities of an OS do not grow exponentially with time or investment. If building full sovereign ownership of the production chain for frontier-level AI were so easy, there woul

More ›