5 ms·
This seems like a great first step to a truly open source LLM
by Jack5500 4y ago
This seems like a great first step to a truly open source LLM
- visarga 4y agoIf we can confirm the 13B model is as good as GPT3 then at least we have a target for reproduction without FB restrictions.
- lurquer 4y agoIndeed. All the weights for all the models will be available one way or the other very soon. The proprietary nature of the weights is not going to be a bottleneck for more than a month, if I had to guess. The other bottle-neck to personal use — the hardware required to run (not train from scratch) the thing - is going to be gone within the year I bet. I would assume some clever bloke is going to be able to prune the model or decrease the precision of the weights and discover you can get good-enough results with 1/10th of the memory. What happens then? Well, probably some Very Bad Things.