6 ms·
More processing power does not make a model better. You can train models on CPUs with same result based on same model architecture and dataset. It'll just take
by fykem 3y ago
More processing power does not make a model better. You can train models on CPUs with same result based on same model architecture and dataset. It'll just take longer to get those results.
What makes models "good" is if the dataset "fits" the model architecture properly and you have given it enough time (epochs) to have a semi accurate prediction ratio (lets say 90% accurate). For image classification models I've done around ~100 epochs for 10,000 items seems to be the best certain data sets will ever get. There will at some point come a time when the continued training of the model is either underfitting/overfitting and no amount of continued training/processing power would help improve it.
- HeavyStorm 3y agoThe OP asks "per request", not training time.
- chank 3y agoAnswer is still no and still for the above reason. Compute resources are only relevant to how fast it can answer not the quality.
- pixl97 3y agoThen why does chain of thought work better than asking for short answers?
- p1esk 3y agoBecause it’s a better prompt. Works better for people too.
- famouswaffles 3y agoThat's not the only reason. More tokens = more useful compute towards making a prediction. A query with more tokens before the question is literally giving the LLM more "thinking time"
- p1esk 3y agoIt’s not clear that more tokens are better.
- famouswaffles 3y agoI think it's pretty clear https://arxiv.org/abs/2310.02226 https://arxiv.org/abs/2310.02226 I mean, i can imagine you wouldn't always need the extra compute.
- p1esk 3y agoThis paper is a great illustration of how little is understood about this question. They discovered that appending dummy tokens (ignored during both training and inference) improves performance somehow. Don’t confuse their guess as to why this might be happening with actual understanding. But in any case, this phenomenon has little to do with increasing the size of the prompt using meaningful tokens. We still have no clue if it helps or not.
- deleted 3y ago[deleted]
- famouswaffles 3y agoI just found this paper i read a while ago. Doesn't this answer the question ? The Impact of Reasoning Step Length on Large Language Models - https://arxiv.org/abs/2401.04925 https://arxiv.org/abs/2401.04925 >They discovered that appending dummy tokens (ignored during both training and inference) improves performance somehow. Don’t confuse their guess as to why this might be happening with actual understanding. More tokens is more compute time for the model to utilize, that is completely true. What they guess is that the model can utilize the extra compute for better predictions even if there's no extra information to accompany this extra "thinking time".
- p1esk 3y agoYes, more tokens means doing more compute, that much is true. The question is whether this extra compute helps or hurts. This question is yet to be answered, as far as I know. I tend to make my GPT-4 questions quite verbose, hoping it helps. This is completely orthogonal to CoT, which is simply a better prompt - it probably causes some sort of better pattern matching (again very poorly understood).
- frannyg 3y agoOk, thanks. My misconception kind of prohibited the insight of a potential (theoretical) assert statement, which is kind of what is meant by > if the [resulting] dataset "fits" the model architecture properly, right? I have too many questions. It seems unreasonable to ask away and I should instead read the studies and some books.