6 ms·
GPT 2 and 3 used the p50K right? Then GPT-4 used cl100K
by senseiV 3y ago
GPT 2 and 3 used the p50K right? Then GPT-4 used cl100K
- lifthrasiir 3y agoYeah, see [1]. [1] https://github.com/openai/tiktoken/blob/main/tiktoken/model.py https://github.com/openai/tiktoken/blob/main/tiktoken/model....