5 ms·
"External Storage" whatever that is can not be the same as continous learning as it does not have the strong connections/capture the interdepencies of knowledge
by mhl47 6mo ago
"External Storage" whatever that is can not be the same as continous learning as it does not have the strong connections/capture the interdepencies of knowledge.
That said I think we will see more efforts also on the business side to have models that can help you build a knowledge base in some kind of standardized way that the model is trained to read. Or synthesize some sort on instructions how to navigate your knowledge base.
Currently e.g. Copilot tries to navigate a hot mess of a MS knowledge graph that is very different for each company. And due to its amnesia it has to repeat the discovery in every session. No wonder that does not work. We have to either standardize or store somewhere (model, instructions) how to find information efficiently.
- zby 6mo agoThe key to make Copilot useful is to take the limited context problem seriously enough. There are many dimensions to it: https://zby.github.io/commonplace/notes/context-efficiency-is-the-central-design-concern-in-agent-systems/ https://zby.github.io/commonplace/notes/context-efficiency-i... and it should be the starting point for designing the systems that extensively use llms.
- andai 6mo agoWell, did you solve the problem of deciding what to remember? (And I suppose, how to retrieve it? i.e. LLM can retrieve info, but if it doesn't know about something because it hasn't been retrieved yet...)
- zby 6mo agoThere are many techniques - I analyse them at my knowledge base. I wrote 'my bet is that solving ...' - if I have solved it I would not have to hedge like this.