nstogner

4 ms·

1. 1.▲
   LLM Load Balancing at Scale: Consistent Hashing with Bounded Loads(kubeai.org)

   1 points by nstogner 2y ago|1 comments
2. 2.▲
   by nstogner 2y ago

   Operating large language models (LLMs) at scale in real-world scenarios necessitates multiple backend replicas. The inter-replica load balancing algorithm greatly influences overall system performance by impacting the caching dynamics inher
3. 3.▲
   Show HN: SandboxAI – Run AI generated code in containers(github.com)

   2 points by nstogner 2y ago|0 comments
4. 4.▲
   Short visual intro to Kubernetes controllers(nickstogner.com)

   1 points by nstogner 2y ago|0 comments
5. 5.▲
   Show HN: KubeAI – Open AI on Kubernetes(github.com)

   4 points by nstogner 2y ago|0 comments
6. 6.▲
   by nstogner 3y ago

   This issue is now fixed in v0.9.0 (check "kubectl notebook --version").
7. 7.▲
   by nstogner 3y ago

   Thanks! Kubernetes provides some solid building blocks for training LLMs. I believe that what is missing is a little tooling to link together those primitives into a streamlined ML DevX.
8. 8.▲
   Show HN: Kubectl Notebook(substratus.ai)

   12 points by nstogner 3y ago|6 comments
9. 9.▲
   Stop Setting Up Software Projects to Fail(upgear.io)

   1 points by nstogner 9y ago|0 comments
10. 10.▲
    Show HN: Visualizing the 2016 Presidential Discussion in Real Time (D3js)(pulseonpolitics.com)

    4 points by nstogner 10y ago|0 comments