7 ms·
Nothing special? I mean, inference engine might need to get some tweaks, to support whatever compute is available. But then, if you put a few terabytes of disk
by reisse 4mo ago
Nothing special?
I mean, inference engine might need to get some tweaks, to support whatever compute is available. But then, if you put a few terabytes of disk for swap, and replace RAM to bigger sticks if possible, it should work? Slowly, of course, but there is no reason it should not to.
- reverius42 4mo agoThe big difference will be measuring seconds per token instead of tokens per second.
- martijnvds 4mo agoSeconds per token is just fractional tokens per second ;)
- degamad 4mo ago> fractional Reciprocal?