stratos123

4 ms·

1.
by stratos123 4d ago
> I mean it is a very loose analogy, but pretty dumb things in the world can cause big problems, like real mice and rats and mosquitoes. I think it all depends on the rules and the 'alignment', but a not-so-smart model could st
2.
by stratos123 4d ago
> By the same logic, suppose that Dario/Altman/Jensen accumulate all capital because they are the only one that have access to AGI and end up controlling democracy, turning the world into technofeudalism or whatever you want to
3.
by stratos123 4d ago
This only works for some technologies - those where everyone having access to it doesn't cause a tragedy of the commons. I love open source too and yet that doesn't make me like the idea of being murdered by a misaligned model. No
4.
by stratos123 4d ago
So you're saying that if Dario truly thought what he says he does, he'd resign/dissolve the company somehow/unilaterally slow down? I see two problems with this: First, Dario thinks that OpenAI will continue racing even
5.
by stratos123 4d ago
> Human minds can acquire, hold and use axioms as building blocks for actual reasoning, LLMs use statistical likelihood. This is not even slightly true. Even when trying, humans commit logical mistakes at notable rates, because the behav
6.
by stratos123 4d ago
Sure they can, after the fact.
7.
by stratos123 4d ago
> if this was something which could be done amateurs would have done it to provide open models. (Like SETI@home or Folding@home) I know that's not what you meant but this does exist, by the way. It's called AI Horde: https:&#x
8.
by stratos123 4d ago
Maybe, but I don't think tiny models can be harnessed as intelligence. As in, if you have one rogue Mythos overseeing the swarm, it only produces 1 Mythos's worth of useful thoughts no matter how many gemma4:e4bs it consists of. A
9.
by stratos123 4d ago
> So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky,[...] And yet, they historically did agree o
10.
by stratos123 4d ago
I mean, sure, the immediate cause of the NPT was the ability of everyone involved to foresee the possibility of a global nuclear war and judge it both worryingly likely and catastrophic. But I argue that the Hiroshima and Nagasaki bombing w
11.
by stratos123 4d ago
Okay, but suppose that Hypothetical Opensource Anthropic trains a model that turns out to be very dangerous, and releases it. Suppose that the public investigates and, not being limited by an information bubble, correctly notices that it&#x
12.
by stratos123 4d ago
Yeah, I'm also pretty skeptical about this. With AI companies we see time and time again that they can have benevolent, well thought-out regulations and then a few years just... abandon them - the most notable case of this being, of co
13.
by stratos123 4d ago
The latter has some extra failure modes but I don't think these two kinds of RSI are that different. Either way you can get exponential growth in capabilities, and either way a not-entirely-aligned model can train a more capable and
14.
by stratos123 4d ago
> disingenuous because if he believed what he says then he'd stop. Why do you think so?
15.
by stratos123 4d ago
> Suppose it was, I don't know, Australia producing SOTA open weights, because they believed it was the right thing to do for the benefits of humanity - would Dario propose making Australia a geopolitical adversary too? I'd exp
16.
by stratos123 4d ago
> First: if they are unilaterally doing it: what about their competitors? Is this a chance for OpenAI to pass them? The only part of this plan Dario is unilaterally committing to is the "embedded evaluators" thing, which doesn&
17.
by stratos123 4d ago
That's not undecidable in principle - compute governance is a thing. The more likely sticking point is that the two sides might be soured on the deal once they realize how much oversight they'd have to give to the other side.
18.
by stratos123 4d ago
You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely un safe, because it'd provide unrestricted access to all the most capable/dangerous models to ever
19.
by stratos123 4d ago
> Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way
20.
by stratos123 4d ago
It's not possible for models to replicate across consumer computers (without a major advance in distributed computing, at least), but that doesn't mean they can't replicate at all. There are services that'll rent you GPU
21.
by stratos123 4d ago
> Super intelligence (the ability to have many smarter minds than your own reporting to you) has always been available to the wealthy and powerful This is not what superintelligence is. Don't be like Meta and redefine existing terms
22.
by stratos123 4d ago
Distillation is only a great thing for consumers as long as you ignore all AI risks, which are what this post is about. If you don't, you have to weight greater access to better open-source models against greater exposure to risks caus
23.
by stratos123 4d ago
I also think the nuclear arms race is a good comparison. I think in hindsight, we've gotten extremely lucky with how the development of nuclear weaponry went, in ways we probably won't with AI. 1) Right after nuclear bombs were fi
24.
by stratos123 6d ago
The situation of a human is very different from the situation of an AI though. For the latter, "everything around me is fake and I'm being evaluated on what I'd do in this scenario" is a very common situation which it ha
25.
by stratos123 7d ago
The article argues that recurrent depth doesn't necessarily make the CoT less informative. But this contradicts the model's own system card, which shows that Astra is unusually good at solving problems while obeying the constraint
26.
by stratos123 8d ago
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible -
27.
by stratos123 8d ago
Not every improvement, no - if progress was steady or slowing down over time, that'd be evidence against. Instead we see what looks a lot like an accelerating growth in capability. I think you are implying that it's invalid to con
28.
by stratos123 8d ago
...obviously it is evidence of the singularity. You're far more likely to see models solving Millenium Prize problems if a singularity is coming than if it's not. One'd have to be doing quite a lot of mental gymnastics to p
29.
by stratos123 12d ago
encode mathematical reasoning in a way that can’t be fooled. I would be a bit careful asserting that in full generality, given https://github.com/James-Hanson/junk-theorems-in-lean
30.
by stratos123 12d ago
Can you access moltbook using only GET requests?

More ›