19 ms·
Those are really nice numbers. With that t/s, no network latency or queueing it must feel much snappier than cloud models.
by redox99 8d ago
Those are really nice numbers. With that t/s, no network latency or queueing it must feel much snappier than cloud models.
- skolos 8d agoit is, unless it set to xhigh - it really likes generating tons of tokens for its thinking. unfortunately, for decently reliable coding results you want it on xhigh ...