Most short-term learning/adaptation is already handled in-context. Modern context windows can hold several books worth of text - plenty for most tasks. Everybod

Most short-term learning/adaptation is already handled in-context. Modern context windows can hold several books worth of text - plenty for most tasks. Everybody is already using it to adapt models to their projects through skills/instructions/guides etc. ps. I often say that after glossary-skill next must have one is update-skill-skill that threats all .md files as live documents. Persistent weight adaptation also happens just not in real time - sessions are captured, analyzed, transformed into training data, fed into SFT/RL environments and later contribute to model updates. Takes a bit of time for the whole loop but you can't say it's not present. There's nothing fundamentally preventing real-time weight updates, ie. LoRA-style online adaptation would be one obvious approach. It's just generally not worth doing at scale. Updating a shared model centrally gives much better data efficiency, batching, evaluation, control etc. than continuously training a separate set of weights for every user/session. There is some work happening on narrowing that gap, for example Mistral has been pushing efficient LoRA-based customization, continuous pretraining, model adaptation etc. I also did play a bit with activation steering – it's super cool where you extract profile for some concepts (emotional in my case) and you have effectively toggles to control "brightness/contrast" those areas (enhancing or suppressing those activation regions from profile) injecting to the model those concepts (emotions in my case) – you can do it in real time and it's fun thing to play with.