6 ms·
I thought about this too, but the throughput over a network is incredibly slow. It’s not usable for interactive use.
by jmercouris 2mo ago
I thought about this too, but the throughput over a network is incredibly slow. It’s not usable for interactive use.
- i386 2mo agoThat isn’t true. llama RPC is incredibly slow but staged splits in skippy are orders of magnitude faster.
- stymaar 2mo agoThroughput is not a problem as you just share relatively small vectors (a few kilobytes in size), the key issue is network latency.
- jmercouris 2mo agoI didn't mean the throughput of the network, I meant the throughput of the calculations. Yes, the latency is the issue.