7 ms·
The inference seems to be taking very long and it gets stuck at times forever (> a few minutes). My guess is that this is happening due to the load? How fast i
by aarkay 4y ago
The inference seems to be taking very long and it gets stuck at times forever (> a few minutes). My guess is that this is happening due to the load? How fast is this when it's not under load? Is it as fast as ChatGPT?
- rushingcreek 4y agoIt is faster than ChatGPT normally. We're frantically scaling our infra right now.
- wayy 4y agoThe p50 time is around 2s end to end not under load. Bear with us as we scale our systems!
- tobr 4y agoThis might help: https://beta.sayhello.so/search?q=the+p50+time+is+around+2s+end+to+end+not+under+load.+how+do+we+scale+our+system+quickly https://beta.sayhello.so/search?q=the+p50+time+is+around+2s+... It has the code you need: INPUT X IF X > 10 THEN PRINT "X is greater than 10." ELSE IF X < 10 THEN PRINT "X is less than 10." ELSE PRINT "X is equal to 10." END IF END IF
- aarkay 4y agoGreat! All the best scaling :) I'll give it a shot again in a bit.
- rushingcreek 4y agoThanks! It's much faster now :)