5 ms·

I’ve been sidetracked with work but planning on tuning llama 65B to produce alpaca 65B, will distribute via huggingface or torrent.. FWIW running the 30B alpac

I’ve been sidetracked with work but planning on tuning llama 65B to produce alpaca 65B, will distribute via huggingface or torrent.. FWIW running the 30B alpaca-lora model quantized to 4-bit via llama.cpp has given me great results, and while I don’t expect much of an improvement from 65B at FP16, 65B will probably perform better than 30B when quantized The interesting next steps in my head are more focused around curating a better instruction-tuning dataset using GPT-4, then fine-tuning again, and integrating the LangChain project with the resulting agent

gorbypark 3y ago
Sounds great! It would be great if you could release your intermediate steps, so Alpaca-65B and then Alpaca-65B-LangChain, etc. I'd like to compare the model sizes directly when fine-tuned using the same method / dataset! I also just realized that I don't believe there's an "alpaca-native" 30B floating around, just the alpaca-lora one, so 30B would be pretty cool too (and the biggest I can run w/ llama.cpp on my MacBook).