4 ms·
No. Lora is for tuning behaviour and how the model applies what it learned in training. Teaching a model new facts is still expensive.
by irthomasthomas 2mo ago
No. Lora is for tuning behaviour and how the model applies what it learned in training. Teaching a model new facts is still expensive.
- didgeoridoo 2mo agoThere was a recent-ish paper[0] from Sakana AI about baking facts from a document corpus into a LoRA adapter. Claimed near perfect recall on very large needle-in-haystack testing. Haven’t tried it myself though. [0]: https://sakana.ai/doc-to-lora/ https://sakana.ai/doc-to-lora/
- irthomasthomas 2mo agointeresting... thanks.