7 ms·
Show HN: ReliableGPT run 200 GPT-4 requests in parallel
reliableGPT handles GPT-4 429 rate limit errors
it optimally allocate jobs for incoming requests to maximize usage of Requests/min and Token/min limits
- nomadness 3y agoWhy not using just a fetch with retry function ?
- huac 3y agoIt is, in fact, a try-catch. https://github.com/BerriAI/reliableGPT/blob/main/reliablegpt/main.py#L14C2-L14C2 https://github.com/BerriAI/reliableGPT/blob/main/reliablegpt...
- derwiki 3y agoLooks like this will only be effective for short prompts/responses, eg if you have 4k tokens in your prompt, you can only fire 10 requests/minute with 40k token/minute rate limit