6 ms·
Don't forget that it's only an assumption that scaling more results in better models. There may be a ceiling to that. If that is hit and the best performing pos
by sureglymop 2mo ago
Don't forget that it's only an assumption that scaling more results in better models. There may be a ceiling to that. If that is hit and the best performing possible model can run in little VRAM, your argument here doesn't hold anymore.
In a way it is actually the same thing that makes us accept AI as working in the first place. It only needs to be good enough for human perception. The same is probably true for compute.
- oblio 2mo agoI'd argue we've already hit the ceiling. Can you truly tell the different between SOTA models from 9 months ago and those from today? There are some improvements but they're mostly marginal. Plus there is a chance the actual scaling that matters is beyond our reach. Think instead of TB models, PB or ZB models. We don't even have that kind of information. Humanity in its entire history hasn't generated 1ZB of information.
- xyzsparetimexyz 2mo ago> Can you truly tell the different between SOTA models from 9 months ago and those from today On a task that corresponds to the benchmarks, yes absolutely.
- overfeed 2mo ago> On a task that corresponds to the benchmarks, yes absolutely The very definition of diminishing returns.