5 ms·
My primary use case for LLMs are running jobs at scale over an API, and not chat. Yes it's very slow, and it is annoying. Getting a response from GPT-5-mini f
by binarymax 1y ago
My primary use case for LLMs are running jobs at scale over an API, and not chat. Yes it's very slow, and it is annoying. Getting a response from GPT-5-mini for <Classify these 50 tokens as true or false> takes 5 seconds, compared to GPT-4o which takes about a second.
- jscheel 1y agoDoing quite a bit of that as well, but I’ve held off moving anything to gpt-5 yet. Guessing it’s a capacity issue right now.
- hoppp 1y agoIf its 5 seconds maybe you are better off renting a GPU server and running the inference where the data is, without round trips and you can use gpt-oss
- beering 1y agoThe 5 seconds delay is probably due to reasoning. Maybe try setting it to minimal? If your use case isn’t complex maybe reasoning is overkill and gpt-4.1 would suffice.