7 ms·
What are you comparing it to? Without instruction tuning and a two character prompt "He" I am not sure why you would expect it to perform any better.
by ddren 4y ago
What are you comparing it to? Without instruction tuning and a two character prompt "He" I am not sure why you would expect it to perform any better.
- refulgentis 4y agoI was replying to a comment that said it “seems fine.” It does not seem fine. It is incomprehensible and doesn’t match the results I’ve seen from 7B through 65B. It is true that RLHF could improve it, and perhaps then this severe of optimization will seem fine.
- tbalsam 4y agoI've heard a number of people say (from earlier) that the quantization and default sampling parameters is way wacked. Honestly even running that model size alone is the big achievement here and getting the accuracy to actually reach the benchmark is the beeg next step nao, I believe. <3 :'))))
- lostmsu 4y agoIf you run a quantized 60G model and the output is worse than raw 7G model, you can throw your quantizer out.