maxaravind

5 ms·

1.
by maxaravind 5mo ago
The blog discusses a potential path to making LLMs self-referential and self-directed continual learners. Solving continual learning is key to AGI - and we maybe close to solving it.
2.
Language Models Are Few-Shot Learners, They Just Can't Remember(aravindjayendran.com)
2 points by maxaravind 5mo ago|1 comments
3.
by maxaravind 5mo ago
Didn't get this error before. Try now, it should be fixed.
4.
by maxaravind 5mo ago
I'd say the way to think about it is in terms of the questions you ask being in-distribution or out of distribution w.r.t the model training dataset. Consider this, if something fundamental has changed in the world after the model was
5.
by maxaravind 5mo ago
There has been a lot of talk about how continual learning might be "just and engineering challenge" and that we could have agents that continuously learn from experience by just having longer and longer context windows. Here is a
6.
by maxaravind 5mo ago
here you go: https://www.youtube.com/watch?v=Z0x99Uu4rJc
7.
by maxaravind 5mo ago
Nope. Even if context can theoretically encode arbitrary computation under fixed weights, this requires the weights to implement a usable interpreter. Random weights almost surely do not. Training is what constructs that interpreter. Withou
8.
by maxaravind 5mo ago
lol nice analogy. LLMs are frozen diamonds forged in compute. We need then to be malleable in production and change with experience.
9.
by maxaravind 5mo ago
I think both the views have their merits. In my mind the hardware vs software analogy for weights vs context holds better because in most modern computing systems, the hardware is fixed and the software changes. What the system can do effic
10.
The Irreducibility of Production Experience in Software(aravindjayendran.com)
2 points by maxaravind 5mo ago|0 comments
11.
by maxaravind 5mo ago
Author here. I spent the last weekend thinking about continual learning. A lot of people think that we can solve long term memory and learning in LLMs by simply extending the context length to infinity. I analyse a different perspective tha
12.
Context Is Software, Weights Are Hardware(aravindjayendran.com)
18 points by maxaravind 5mo ago|17 comments
13.
by maxaravind 5mo ago
One of the authors here. We ran this experiment a few weeks ago, but Anthropic’s Mythos report dropped this week and thought this would be relevant to share now. Surprisingly, we found that for repos already in the training data(pre cut off
14.
Déjà Code: How LLMs Cheat on Repos They've Seen(blogs.latentforce.ai)
1 points by maxaravind 5mo ago|1 comments
15.
by maxaravind 5mo ago
The 1-2 files per request constraint is interesting. I do the same thing, but I've started thinking of it as a symptom rather than a solution. The reason you have to constrain scope is because the agent doesn't know what it doesn&
16.
by maxaravind 5mo ago
Personal superintelligence sounds nice until you actually try to use it. We spent time yesterday arguing through an architecture decision. Today I ask the Agent to help implement it - it knows nothing about any of that. You’re effectively s