17 ms·
I'm really hoping for the ability to load in different sets of trained material as embeddings/textual inversions like in Stable Diffusion. Imagine scanning in s
by edkennedy 3y ago
I'm really hoping for the ability to load in different sets of trained material as embeddings/textual inversions like in Stable Diffusion. Imagine scanning in some of your favorite philosophy and design books and throwing them with small weighting as a little flavor to your answer. The crossovers between LLM and Stable Diffusion type models (like Loras) is such a fascinating space to explore.
- alex_sf 3y agoThis already exists for LLaMA, I'm sure it'll be implemented for StableLM as well. https://github.com/lxe/simple-llm-finetuner https://github.com/lxe/simple-llm-finetuner
- pksebben 3y agogood looks on the link! I was experimenting with script writing the other day and thought "gee, I really wish I could finetune on Beckett plays specifically". I don't know if anyone else has experienced this same tipping point, but when I used to have ideas, I would look them up and discover that implementing them was probably out of scope. These days, I think "wouldn't it be cool..." and immediately stumble on a way to make it happen, by accident.
- cheald 3y agoThis has been around for GPT models for a while in the form of "soft prompts", which are rather approximate to textual inversion in the Stable Diffusion space.