7 ms·
From my own experiments with various models, I suspect that LLMs have distinct/partitioned cutoff dates; for example, historical literature doesn't change (Gree
by tobwen 1mo ago
From my own experiments with various models, I suspect that LLMs have distinct/partitioned cutoff dates; for example, historical literature doesn't change (Greek history, Shakespeare, Goethe), general knowledge (updated only in certain areas), technologies (updated regularly), software also remains surprisingly stable - for example, with GIT, a basic command set is sufficient to do 99% of the jobs - new features are unknown or unnecessary, and tabloid knowledge, which is always up-to-date (politics, Taylor Swift albums).
- sshh12 1mo agoI would have thought that as well but at least for coding vs world event facts I didn't see obvious differences in cutoffs
- tobwen 1mo agoIn my tests, I compiled the changelogs from various projects and checked how much functionality the model could recall over several test runs. In some cases, even with the latest models, the cutoff was around June 2024. Beyond that, it didn't recognize the features. However, it did know the cause of the last pope's death and even which hospital he was in (April 2025). However, the model did not know when Trump was sworn for the second time (Jan 2025) - but it did know THAT he was elected a second time (Nov 2024).