sig_kill

6 ms·

1.
by sig_kill 2mo ago
Write up here: https://meshllm.cloud/docs/pages/exo-comparison/
2.
by sig_kill 2mo ago
Yes! I’ve worked on the settings interface between our runtime and llamacpp, these are documented and available via our config.toml file
3.
by sig_kill 2mo ago
This is actually how I develop and use the mesh at home. Rather than splitting models, I aggregate disparate compute behind one endpoint, without having separate inference providers on each host and a gateway like LiteLLM
4.
by sig_kill 2mo ago
There’s a breakdown on our docs page you can check out! https://meshllm.cloud/docs/pages/exo-comparison/
5.
by sig_kill 2mo ago
Yeah, this is one area we’re struggling with due to the sheer volume of variations and conditions, but I’ve been thinking of collecting some real-time statistics around latency, prefill/decode, and model distribution… that way we can u
6.
by sig_kill 2mo ago
We didnt post it, we use a library (iroh) who featured us - so we are here answering any Q’s instead :)
7.
by sig_kill 2mo ago
It depends on the size / structure of the model… but generally you can check out the MeshLLM huggingface org for pre-computed model splits. We have a job that will take popular models, and dice them up and post on HF. When that specifi
8.
by sig_kill 2mo ago
Great question! We’ve had a lot of discussions about the direction we want to take this, and how to best generate some kind of incentive / fairness reward. And we’ve found ourselves hesitating on a direction because, at least for now,