7 ms·
I don't trust this. The article cites semafor (https://www.semafor.com/article/03/24/2023/the-secret-history-of-elon-musk-sam-altman-and-openai https://www.sem
by euclaise 3y ago
I don't trust this. The article cites semafor (https://www.semafor.com/article/03/24/2023/the-secret-history-of-elon-musk-sam-altman-and-openai https://www.semafor.com/article/03/24/2023/the-secret-histor...), but semafor states the 1T parameter count without any source.
- brianjking 3y agoYeah, I'm highly suspicious too. Even the arxiv article from the MS researchers doesn't have specifics about the # of parameters in GPT-4.
- QuadrupleA 3y agoYeah seems spotty. Especially considering recent "chinchilla scaling" laws suggesting training set size is generally the current bottleneck, the mileage llama/alpaca gets out of 7b/13b, the huge inference cost of 1T, etc.
- lambo4bkfast 3y agoSam altman said it had 1T parameters in the Lex Fridman podcast