6 ms·
> Anyway, you might have more luck just writing to it in your native language. This is potentially expensive advice (at least for many mainstream options). Whe
by argee 1mo ago
> Anyway, you might have more luck just writing to it in your native language.
This is potentially expensive advice (at least for many mainstream options). Where an English word like "literature" is one token, a couple of Chinese characters that spell a word can be 4 tokens. You'll pay more for input/output and get less of a context window (per word) too.
- valleyer 1mo agoIncidentally, according to https://gpt-tokenizer.dev https://gpt-tokenizer.dev, in gpt-5, "literature" is two tokens ("liter" + "ature"), whereas "文学" is one.
- argee 1mo agoMy bad, the English word I had in mind was "technology" (技術) but I misremembered due to feeling like "tech" should be its own token.