5 ms·
My guess is that "making cheaper and faster GPT4" was leading to the rumors that GPT4 was nerfed
by dontreact 3y ago
My guess is that "making cheaper and faster GPT4" was leading to the rumors that GPT4 was nerfed
- victorvosk 3y agoI don't care what the official line is, something changed with it. It might be the same model but the way it answers questions and the content of those answers is much more dubious now. I think how it manages its conversation context was gimped in the web chat version. I find it routinely completely forgetting things in longer conversations.
- jonplackett 3y agoI was thinking how I could optimise for longer conversations and the most obvious thing seemed to be to somehow summarise the conversation ever X messages and ask it to save what it things is important isn’t them. If they’re doing something like this that would probably explain it.
- Stagnant 3y agoIt is the text generation speed that got me suspicious. It seems like ChatGPT's GPT4 suddenly got 2-3 times faster at generating text, while the API GPT4 still generates the text at the same pace as it always has.
- hickelpickle 3y agoI was using it to experiment lazily with machine learning and midi generation via CNN-LSTMs and noticed it happening in the middle of a chat with it. The context wasn't that long and it was providing excellent insights, then randomly started telling me how to make a music game in python and lost all previous context. I tried starting a new chat and adding the back context,had a good bit of code I was providing for context, and it could hardly keep track of the provide code context over the course of a few simple questions/changes, when before it had no problem and would actually surprise with drawing from long past context relevantly. After what ever happened it would randomly assume karras was being used if having not been provided any pytorch specific code for a few messages. API doesn't have these issue, or didn't a week ago when I was using it often, but something happened with the web model, which I guess is understandable, as the main reason I was using it vs the API was cost savings.
- 93po 3y agoCopilot censored its response to my question about linked lists yesterday. Ridiculous.
- whatyesaid 3y agoBut GPT 3.5 has Turbo but nobody claims it reduced accuracy even though it's much faster. GPT 4 is now a lot faster and struggles much more. And in theory being faster should be an advantage to an extent, even if the model needs more prodding, as you gain a more seamless back and forth instead of waiting half a minute. Some people say that people think it's nerfed because when it was slow we associated it with "thinking harder" but I doubt that's the case, and the way to measure that is by loss of ability, i.e. if it is stuck more often as a result of itself and not a bad prompt.
- MacsHeadroom 3y agoTurbo is definitely a much weaker model.
- npsomaratna 3y agoNo, my experience is that GPT-3.5-turbo generates markedly poorer results than text-davinci-003 and -002.
- qeternity 3y agoAgreed, it's general consensus that 3.5-turbo is a substantially weaker model.
- sebzim4500 3y ago>But GPT 3.5 has Turbo but nobody claims it reduced accuracy even though it's much faster. Loads of people claim this, I haven't tested it myself though.
- user_named 3y agoEveryone knows that turbo is less accurate.
- whatyesaid 3y agoGuess I missed that 3.5 Turbo achieved speed by also downgrading. I didn't hear anyone complain, to be fair I only use GPT 4 because I try to use it to save time and even with 4 you have to verify. And I guess the trend is that anyone who uses it frequently uses 4 and 3.5 for basic stuff only.
- rmbyrro 3y agoMore likely to be the non-compete words, I think.