5 ms·
There was a step change over the last few years but that rate of improvement is not continuing. The currently known techniques seem to have hit a plateau. It's
by downWidOutaFite 2y ago
There was a step change over the last few years but that rate of improvement is not continuing. The currently known techniques seem to have hit a plateau. It's impossible to know when the next "Attention is All You Need" will materialize. It could be in 2025 or 2035.
- sdenton4 2y agoOr 2023... There's a lot of papers out there!
- FeepingCreature 2y agoo1 sort of techniques (tree search, test-time training type things) have not hit any recognizable plateau. There's still low hanging fruits all around.
- blharr 2y agoThis discussion includes o1. It's marginal improvement compared to, say, GPT-3 to GPT-4.