itkovian_

5 ms·

1.
by itkovian_ 4d ago
What are you talking about - I feel like we’re living in parallel realities. If I had to go back to opus 4.5 tomorrow I’d be hugely upset and significantly slowed down
2.
by itkovian_ 4d ago
The author is trying to make the point that there are courses of action available to Dario that would achieve his stated goals but would result in Anthropic having very little impact, control or influence. And that because he’s not doing th
3.
by itkovian_ 14d ago
I can’t stand it. Very engineering-y over specified formal language around a complete lack of core understanding. Is damaging other people read this and try to learn things from it.
4.
by itkovian_ 2mo ago
Causal masking allow model to learn implicit positional embeddings. The meme that a transformer block is permutation invariant is not true.
5.
by itkovian_ 2mo ago
Intrinsic motivation/curiosity driven exploration/novelty seeking feels like it has a breakthrough paper waiting. If someone could get those methods working for LLMs, we start getting things like move37 but in math proofs and then
6.
by itkovian_ 2mo ago
the difference is all the current capex is going to durable, hard to get physical assets + things like PPAs. In your online shopping analogy, the hyperscalars are acting like Amazon in 1998
7.
by itkovian_ 2mo ago
I just don’t understand this view. This is the most significant technology ever developed. The uncertainty currently is whether it 1) has massive impact, completely altering society and the making world significantly significantly better or
8.
by itkovian_ 2mo ago
‘640kb of ram should be enough for anyone’
9.
by itkovian_ 3mo ago
Contrarian view; I think he’s right. Many of these ideas it’s almost shocking how many you can find sketched out in his old papers. To the point where I think it’s very wise to read all his work to see what hasn’t showed up yet but likely w
10.
by itkovian_ 3mo ago
You can’t run a closed llm locally. Strange to frame the dichotomy as between local and open. One begets the other.
11.
by itkovian_ 3mo ago
The entirety of Anthropic believe ai is going to eat everything, not just software, and result in major societal disruption within a year. They do not have a sliver of a doubt on this. Article has no idea, is completely wrong.
12.
by itkovian_ 3mo ago
This is called linear mode connectivity and seems to work for almost every large model. So well that in most cases it’s an explicit part of the training process; do many training ‘branches’ then merge then continue. It is not understood why
13.
by itkovian_ 3mo ago
Projects like pluralis agora solve this problem. Really what you want is the model to be collectively owned and governed, not local
14.
by itkovian_ 3mo ago
People forget the people in charge of these companies are some of the smartest people out there full stop. Far more shadowy strategy/things like this going on than people think.
15.
by itkovian_ 3mo ago
What are the odds this is partially them making the point; you were all complaining about monitoring/access/safeguards: remember we don’t have to give this to you at all. And using a us gov letter as justification for that.
16.
by itkovian_ 3mo ago
In many cases the GPs are not, at least nowhere near as much as you’d think. Obviously there are power laws here as well. Non partners, forget about it. The other thing is it is maybe the most nepotistic industry out there. Which somewhat m
17.
by itkovian_ 4mo ago
The extreme privilege of being forced to pay a major portion of all income you make, regardless of where you earn it, to the us gov indefinitely. And they make it hard for you to apply to do this. Crazy.
18.
by itkovian_ 4mo ago
The US isn’t what it used to be. It’s definitely not the best place in the world to live for quality of life, on basically any metric. The requirement of being permanently obligated to pay us taxes on global income, if you have any kind of
19.
by itkovian_ 4mo ago
The other thing people don’t understand is exponential curves are self similar. The start of an exponential looks like an exponential. People always look at and think ‘well that’s it it’s exponential now, have missed it, can’t sustain’. Nop
20.
by itkovian_ 10mo ago
The argument is that there is no incentive to carefully review a paper (I agree), however what used to occur is people would do the right thing without explicit incentives. This has totally disappeared.
21.
by itkovian_ 10mo ago
Whether it’s actually 20% or not doesn’t matter, everyone is aware the signal of the top confs is in freefall. There are also rings of reviewer fraud going on where groups of people in these niche areas all get assigned their own papers and
22.
Node-0-7.5B: A collaborative multi-participant, model-parallel pretrain(dashboard.pluralis.ai)
2 points by itkovian_ 1y ago|0 comments
23.
by itkovian_ 1y ago
These are some of the richest entities - forget about universities - just entities full stop, in the entire country.
24.
by itkovian_ 1y ago
I don’t think people understand the point sutton was making; he’s saying that general, simple systems that get better with scale tend to outperform hand engineered systems that don’t. It’s a kind of subtle point that’s implicitly saying han
25.
by itkovian_ 1y ago
I’m gonna go ahead and guess they didn’t raise 8.3b on SAFEs
26.
by itkovian_ 1y ago
Article doesn’t say jobs aren’t about to be evicerated, says this is already happening and it’s due to capitalism, a lack of consumer protections and we require more government regulation. This never made any sense to me because we don’t ha
27.
by itkovian_ 1y ago
The reason for this is it’s horrifying to consider that things like the Ukrainian war didn’t have to happen. It provides a huge amount of phycological relief to view these events as inevitable. I actually don’t think as humans are even able
28.
by itkovian_ 1y ago
I think the better analogy is if you had someone with a superhuman, but not perfect memory read a bunch of stuff, then you were allowed to talk to the person about the things they’d read, does that violate copyright? I’d say clearly no. The
29.
by itkovian_ 1y ago
Completely agree and think it’s a great summary. To summarize very succinctly; you’re chasing a moving target where the target changes based on how you move. There’s no ground truth to zero in on in value-based RL. You minimise a difference
30.
by itkovian_ 1y ago
Saying we should tokenize different modalities the same would be analogous to saying that in order to be really smart, a human has to listen with its eyes. At some point there has to be SOME modality specific preprocessing. The thing is in

More ›