5 ms·
thanks! > ditched oLlama" yeah! this is interesting. > 8.1GB per slotserve process is a lot! Is that in your control? yes, it is hard, but I agree the small
by carloslfu 14d ago
thanks!
> ditched oLlama"
yeah! this is interesting.
> 8.1GB per slotserve process is a lot! Is that in your control?
yes, it is hard, but I agree the smaller the better. I'll work on that
> If it's local and open-weight, this could be marketed this way I think.
I like this!
> what's the actual, real use case for slotserve?
I'm working rn on an app on top of it that closes the loop and is a fully local AI app, an experiment. I'll publish it as soon as it is usable!
> built a small html hello world served via Python
What did you use as a harness here?
- baristaGeek 14d agoFor the harness... just the shell. No client library. Does that answer your question?
- carloslfu 14d agothanks! in part, I was wondering how you got the code into files. I guess you copy pasted it inside a file, am I right?